Task-Aware Dynamic Model Optimization for Multi-Task Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Choi, Sujin | - |
dc.contributor.author | Jin, Hyundong | - |
dc.contributor.author | Kim, Eunwoo | - |
dc.date.accessioned | 2024-01-24T05:01:16Z | - |
dc.date.available | 2024-01-24T05:01:16Z | - |
dc.date.issued | 2023 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/71327 | - |
dc.description.abstract | Multi-task learning (MTL) is a field in which a deep neural network simultaneously learns knowledge from multiple tasks. However, achieving resource-efficient MTL remains challenging due to entangled network parameters across tasks and varying task-specific complexity. Existing methods employ network compression techniques while maintaining comparable performance, but they often compress uniformly across all tasks without considering individual complexity. This can lead to suboptimal solutions due to entangled network parameters and memory inefficiency, as the parameters for each task may be insufficient or excessive. To address these challenges, we propose a framework called Dynamic Model Optimization (DMO) that dynamically allocates network parameters to groups based on task-specific complexity. This framework consists of three key steps: measuring task similarity and task difficulty, grouping tasks, and allocating parameters. This process involves the calculation of both weight and loss similarities across tasks and employs sample-wise loss as a measure of task difficulty. Tasks are grouped based on their similarities, and parameters are allocated with dynamic pruning according to task difficulty within their respective groups. We apply the proposed framework to MTL with various classification datasets. Experimental results demonstrate that the proposed approach achieves high performance while taking fewer network parameters than other MTL methods. | - |
dc.format.extent | 9 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.title | Task-Aware Dynamic Model Optimization for Multi-Task Learning | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/ACCESS.2023.3339793 | - |
dc.identifier.bibliographicCitation | IEEE ACCESS, v.11, pp 137709 - 137717 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.wosid | 001123866500001 | - |
dc.identifier.scopusid | 2-s2.0-85179830858 | - |
dc.citation.endPage | 137717 | - |
dc.citation.startPage | 137709 | - |
dc.citation.title | IEEE ACCESS | - |
dc.citation.volume | 11 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | Multi-task learning | - |
dc.subject.keywordAuthor | resource-efficient learning | - |
dc.subject.keywordAuthor | model optimization | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.