Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Mo, Yunho | - |
dc.contributor.author | Yoo, Joon | - |
dc.contributor.author | Kang, Sangwoo | - |
dc.date.accessioned | 2023-08-25T05:40:10Z | - |
dc.date.available | 2023-08-25T05:40:10Z | - |
dc.date.issued | 2023-07 | - |
dc.identifier.issn | 2227-7390 | - |
dc.identifier.issn | 2227-7390 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/88833 | - |
dc.description.abstract | The use of Transformer-based pre-trained language models has become prevalent in enhancing the performance of task-oriented dialogue systems. These models, which are pre-trained on large text data to grasp the language syntax and semantics, fine-tune the entire parameter set according to a specific task. However, as the scale of the pre-trained language model increases, several challenges arise during the fine-tuning process. For example, the training time escalates as the model scale grows, since the complete parameter set needs to be trained. Furthermore, additional storage space is required to accommodate the larger model size. To address these challenges, we propose a new new task-oriented dialogue system called PEFTTOD. Our proposal leverages a method called the Parameter-Efficient Fine-Tuning method (PEFT), which incorporates an Adapter Layer and prefix tuning into the pre-trained language model. It significantly reduces the overall parameter count used during training and efficiently transfers the dialogue knowledge. We evaluated the performance of PEFTTOD on the Multi-WOZ 2.0 dataset, a benchmark dataset commonly used in task-oriented dialogue systems. Compared to the traditional method, PEFTTOD utilizes only about 4% of the parameters for training, resulting in a 4% improvement in the combined score compared to the existing T5-based baseline. Moreover, PEFTTOD achieved an efficiency gain by reducing the training time by 20% and saving up to 95% of the required storage space. | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | MDPI | - |
dc.title | Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems | - |
dc.type | Article | - |
dc.identifier.wosid | 001038831600001 | - |
dc.identifier.doi | 10.3390/math11143048 | - |
dc.identifier.bibliographicCitation | MATHEMATICS, v.11, no.14 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.scopusid | 2-s2.0-85175038979 | - |
dc.citation.title | MATHEMATICS | - |
dc.citation.volume | 11 | - |
dc.citation.number | 14 | - |
dc.type.docType | Article | - |
dc.publisher.location | 스위스 | - |
dc.subject.keywordAuthor | natural language processing | - |
dc.subject.keywordAuthor | task-oriented dialogue system | - |
dc.subject.keywordAuthor | PEFT | - |
dc.subject.keywordAuthor | fine-tuning | - |
dc.subject.keywordAuthor | training efficiency | - |
dc.relation.journalResearchArea | Mathematics | - |
dc.relation.journalWebOfScienceCategory | Mathematics | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
1342, Seongnam-daero, Sujeong-gu, Seongnam-si, Gyeonggi-do, Republic of Korea(13120)031-750-5114
COPYRIGHT 2020 Gachon University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.