Use all tokens method to improve semantic relationship learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Kihoon | - |
dc.contributor.author | Choi, Gyuho | - |
dc.contributor.author | Choi, Chang | - |
dc.date.accessioned | 2023-08-24T10:40:16Z | - |
dc.date.available | 2023-08-24T10:40:16Z | - |
dc.date.created | 2023-08-24 | - |
dc.date.issued | 2023-12 | - |
dc.identifier.issn | 0957-4174 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/88824 | - |
dc.description.abstract | Recently, research on inference methods has been actively conducted to use language models more effectively for studying natural language understanding. Inference in language models that use bidirectional encoder representations from transformers (BERT) is performed using classification tokens that convey information from the input sentences. The use of single-token inference method for inference does not involve the hidden state vector that contains relevant connection information between the words, which in turn limits the ability to infer semantic relationships. This study proposes a use all tokens (UAT) method that combines unused tokens to improve inference methods through a single token. The UAT method effectively combines hidden state vectors and ensembles the global information of sentences with the local information between words. When the Stanford natural language inference (SNLI) corpus was solved using DeBERTaV3large, compared to the existing single token inference method, the UAT method improved the precision of the neutral relationship by 4.3% (87.7% vs. 92.0%) and the recall of the entailment and contradiction relationship by an average of 2% (93.5% vs. 95.5%). The UAT method proposed in this study can be readily implemented in BERT-based language models, and it enhances the accuracy and F1-score, thereby improving the learning of semantic relationships between sentences. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | PERGAMON-ELSEVIER SCIENCE LTD | - |
dc.relation.isPartOf | EXPERT SYSTEMS WITH APPLICATIONS | - |
dc.title | Use all tokens method to improve semantic relationship learning | - |
dc.type | Article | - |
dc.type.rims | ART | - |
dc.description.journalClass | 1 | - |
dc.identifier.wosid | 001041256700001 | - |
dc.identifier.doi | 10.1016/j.eswa.2023.120911 | - |
dc.identifier.bibliographicCitation | EXPERT SYSTEMS WITH APPLICATIONS, v.233 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.scopusid | 2-s2.0-85164212023 | - |
dc.citation.title | EXPERT SYSTEMS WITH APPLICATIONS | - |
dc.citation.volume | 233 | - |
dc.contributor.affiliatedAuthor | Lee, Kihoon | - |
dc.contributor.affiliatedAuthor | Choi, Chang | - |
dc.type.docType | Article | - |
dc.subject.keywordAuthor | Natural language inference | - |
dc.subject.keywordAuthor | Pretrained language model | - |
dc.subject.keywordAuthor | Natural language understanding | - |
dc.subject.keywordAuthor | Semantic relationship | - |
dc.subject.keywordAuthor | Ensemble | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Operations Research & Management Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Operations Research & Management Science | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
1342, Seongnam-daero, Sujeong-gu, Seongnam-si, Gyeonggi-do, Republic of Korea(13120)031-750-5114
COPYRIGHT 2020 Gachon University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.