Read-All-in-Once (RAiO): Multi-layer Contextual Architecture for Long-Text Machine Reading Comprehension
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Phan, Tuan-Anh | - |
dc.contributor.author | Jung, Jason J. | - |
dc.contributor.author | Bui, Khac-Hoai Nam | - |
dc.date.accessioned | 2023-09-15T05:45:44Z | - |
dc.date.available | 2023-09-15T05:45:44Z | - |
dc.date.issued | 2023-07 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/67648 | - |
dc.description.abstract | Machine reading comprehension (MRC) is a cutting-edge technology in natural language processing (NLP), which focus on teaching machines to read and understand the meaning of texts based on the emergence of large-scale datasets and neural network models. Recently, with the successful development of pre-trained transformer models (e.g., BERT), MRC has advanced significantly, surpassing human parity in several public datasets and being applied in various NLP tasks (e.g., QA systems). Nevertheless, long document MRC is still a remain challenge since the transformer-based models are limited by the input length. For instance, several well-known pre-trained language models such as BERT and RoBERTa are limited by 512 tokens. This study aims to provide a new simple approach for long document MRC. Specifically, recent state-of-the-art models follow the architecture with two crucial stages for reading long texts in order to enable local and global context representations. In this study, we present a new architecture that is able to enrich the global information of the context with one stage by exploiting the interaction of different levels of semantic units of the context (i.e., sentence and word level). Therefore, we name the proposed model as RAiO (Read-All-in-Once) approach. For the experiment, we evaluate RAiO on two benchmark long document MRC datasets such as NewsQA and NLQuAD. Accordingly, the experiment shows promising results of the proposed approach compared with strong baselines in this research field. Author | - |
dc.format.extent | 7 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | - |
dc.title | Read-All-in-Once (RAiO): Multi-layer Contextual Architecture for Long-Text Machine Reading Comprehension | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/ACCESS.2023.3298100 | - |
dc.identifier.bibliographicCitation | IEEE Access, v.11, pp 77873 - 77879 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.wosid | 001041961900001 | - |
dc.identifier.scopusid | 2-s2.0-85165921653 | - |
dc.citation.endPage | 77879 | - |
dc.citation.startPage | 77873 | - |
dc.citation.title | IEEE Access | - |
dc.citation.volume | 11 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | Bit error rate | - |
dc.subject.keywordAuthor | Computational modeling | - |
dc.subject.keywordAuthor | Context modeling | - |
dc.subject.keywordAuthor | Data mining | - |
dc.subject.keywordAuthor | long-text machine reading comprehension | - |
dc.subject.keywordAuthor | Natural language processing | - |
dc.subject.keywordAuthor | Natural language processing | - |
dc.subject.keywordAuthor | Question answering (information retrieval) | - |
dc.subject.keywordAuthor | question-answering system | - |
dc.subject.keywordAuthor | Task analysis | - |
dc.subject.keywordAuthor | Text processing | - |
dc.subject.keywordAuthor | Transformers | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.