Effective SNOMED-CT Concept Classification from Natural Language using Knowledge Distillation
- Authors
- Kim, HyunJoo; Joe, Inwhee
- Issue Date
- Jan-2023
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Keywords
- BioBert; Classification; Knowledge distillation; SNOMED-CT
- Citation
- Lecture Notes in Networks and Systems, v.597 LNNS, pp.54 - 64
- Indexed
- SCOPUS
- Journal Title
- Lecture Notes in Networks and Systems
- Volume
- 597 LNNS
- Start Page
- 54
- End Page
- 64
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/182537
- DOI
- 10.1007/978-3-031-21438-7_4
- ISSN
- 2367-3370
- Abstract
- Recently, as natural language processing (NLP) methods have been developed a lot, research on predicting rule languages such as medical terminology systems such as Systemized Nomenclature of Medicine Clinical Term (SNOMED-CT) in natural language is becoming active. In this paper, we propose a prediction model with the SNOMED-CT code using medical natural language. Thus, natural language is encoded with the existing pre-trained model and used as a teacher model to learn a lightweight student model using knowledge distillation techniques. To improve the performance of the model, augmented data are used for learning with the augmentation technique, and performance improvement is attempted using the Teacher model in the same domain with BioBert.When the Teacher model used BioBert and the Student model used simple LSTM, the distillation results obtained an accuracy of 0.86. Only the Teacher model fine-tuned with the pre-trained model obtained a result of 0.88, but the result of the only student model, which is a simple LSTM, was 0.8695. Although we did not obtain a student model with better performance than the Teacher; however, it is useful to interpret the language of SNOMDE-CT by applying knowledge distillation (KD) to the NLP.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/182537)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.