Effective Transfer Learning with Label-Based Discriminative Feature Learning
- Authors
- Kim, Gyunyeop; Kang, Sangwoo
- Issue Date
- Mar-2022
- Publisher
- MDPI
- Keywords
- natural language processing; transfer learning; pre-training; word embedding
- Citation
- SENSORS, v.22, no.5
- Journal Title
- SENSORS
- Volume
- 22
- Number
- 5
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/83918
- DOI
- 10.3390/s22052025
- ISSN
- 1424-8220
- Abstract
- The performance of natural language processing with a transfer learning methodology has improved by applying pre-training language models to downstream tasks with a large number of general data. However, because the data used in pre-training are irrelevant to the downstream tasks, a problem occurs in that it learns general features rather than those features specific to the downstream tasks. In this paper, a novel learning method is proposed for embedding pre-trained models to learn specific features of such tasks. The proposed method learns the label features of downstream tasks through contrast learning using label embedding and sampled data pairs. To demonstrate the performance of the proposed method, we conducted experiments on sentence classification datasets and evaluated whether the features of the downstream tasks have been learned through a PCA and a clustering of the embeddings.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - IT융합대학 > 소프트웨어학과 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/83918)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.