Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble
- Authors
- Cho, Hyunsoo; Park, Choonghyun; Kang, Jaewook; Yoo, Kang Min; Kim, Tae Uk; Lee, Sang-goo
- Issue Date
- Dec-2022
- Publisher
- Association for Computational Linguistics
- Citation
- Empirical Methods in Natural Language Processing, pp.783 - 798
- Indexed
- OTHER
- Journal Title
- Empirical Methods in Natural Language Processing
- Start Page
- 783
- End Page
- 798
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/188683
- Abstract
- Out-of-distribution (OOD) detection aims todiscern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. Most recentstudies in OOD detection utilize the information from a single representation that residesin the penultimate layer to determine whetherthe input is anomalous or not. Although sucha method is straightforward, the potential ofdiverse information in the intermediate layersis overlooked. In this paper, we propose a novelframework based on contrastive learning thatencourages intermediate features to learn layerspecialized representations and assembles themimplicitly into a single representation to absorbrich information in the pre-trained languagemodel. Extensive experiments in various intentclassification and OOD datasets demonstratethat our approach is significantly more effective than other works. The source code for ourmodel is available online.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.