Task-Aware Dynamic Model Optimization for Multi-Task Learningopen access
- Authors
- Choi, Sujin; Jin, Hyundong; Kim, Eunwoo
- Issue Date
- 2023
- Publisher
- IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- Keywords
- Multi-task learning; resource-efficient learning; model optimization
- Citation
- IEEE ACCESS, v.11, pp 137709 - 137717
- Pages
- 9
- Journal Title
- IEEE ACCESS
- Volume
- 11
- Start Page
- 137709
- End Page
- 137717
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/71327
- DOI
- 10.1109/ACCESS.2023.3339793
- ISSN
- 2169-3536
- Abstract
- Multi-task learning (MTL) is a field in which a deep neural network simultaneously learns knowledge from multiple tasks. However, achieving resource-efficient MTL remains challenging due to entangled network parameters across tasks and varying task-specific complexity. Existing methods employ network compression techniques while maintaining comparable performance, but they often compress uniformly across all tasks without considering individual complexity. This can lead to suboptimal solutions due to entangled network parameters and memory inefficiency, as the parameters for each task may be insufficient or excessive. To address these challenges, we propose a framework called Dynamic Model Optimization (DMO) that dynamically allocates network parameters to groups based on task-specific complexity. This framework consists of three key steps: measuring task similarity and task difficulty, grouping tasks, and allocating parameters. This process involves the calculation of both weight and loss similarities across tasks and employs sample-wise loss as a measure of task difficulty. Tasks are grouped based on their similarities, and parameters are allocated with dynamic pruning according to task difficulty within their respective groups. We apply the proposed framework to MTL with various classification datasets. Experimental results demonstrate that the proposed approach achieves high performance while taking fewer network parameters than other MTL methods.
- Files in This Item
-
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/71327)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.