Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Yoo, Kang Min | - |
dc.contributor.author | Kim, Junyeob | - |
dc.contributor.author | Kim, Hyuhng Joon | - |
dc.contributor.author | Cho, Hyunsoo | - |
dc.contributor.author | Jo, Hwiyeol | - |
dc.contributor.author | Lee, Sang-Woo | - |
dc.contributor.author | Lee, Sang-goo | - |
dc.contributor.author | Kim, Tae Uk | - |
dc.date.accessioned | 2023-08-07T07:44:12Z | - |
dc.date.available | 2023-08-07T07:44:12Z | - |
dc.date.created | 2023-07-21 | - |
dc.date.issued | 2022-10 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/188896 | - |
dc.description.abstract | Despite recent explosion of interests in in-context learning, the underlying mechanism and the precise impact of the quality of demonstrations remain elusive. Intuitively, ground-truth labels should have as much impact in in-context learning (ICL) as supervised learning, but recent work reported that the input-label correspondence is significantly less important than previously thought. Intrigued by this counter-intuitive observation, we re-examine the importance of ground-truth labels in in-context learning. With the introduction of two novel metrics, namely Label-Correctness Sensitivity and Ground-truth Label Effect Ratio (GLER), we were able to conduct quantifiable analysis on the impact of ground-truth label demonstrations. Through extensive analyses, we find that the correct input-label mappings can have varying impacts on the downstream in-context learning performances, depending on the experimental configuration. Through additional studies, we identify key components, such as the verbosity of prompt templates and the language model size, as the controlling factor to achieve more noise-resilient ICL. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | Association for Computational Linguistics | - |
dc.title | Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Kim, Tae Uk | - |
dc.identifier.doi | 10.48550/arXiv.2205.12685 | - |
dc.identifier.bibliographicCitation | Empirical Methods in Natural Language Processing, pp.1 - 16 | - |
dc.relation.isPartOf | Empirical Methods in Natural Language Processing | - |
dc.citation.title | Empirical Methods in Natural Language Processing | - |
dc.citation.startPage | 1 | - |
dc.citation.endPage | 16 | - |
dc.type.rims | ART | - |
dc.type.docType | Proceeding | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | other | - |
dc.identifier.url | https://arxiv.org/abs/2205.12685 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.