Resource-Efficient Multi-Task Deep Learning Using a Multi-Path Networkopen access
- Authors
- Park, S.; Lee, J.; Kim, Eunwoo
- Issue Date
- 2022
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- Computer architecture; Costs; Deep learning; Interference; Knowledge engineering; multi-path network; Multi-task learning; Multitasking; resource-efficient learning; Task analysis
- Citation
- IEEE Access, v.10, pp 32889 - 32899
- Pages
- 11
- Journal Title
- IEEE Access
- Volume
- 10
- Start Page
- 32889
- End Page
- 32899
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61899
- DOI
- 10.1109/ACCESS.2022.3161622
- ISSN
- 2169-3536
- Abstract
- Multi-task learning (MTL) improves learning efficiency compared to the single-task counterpart in that it performs multiple tasks at the same time. Due to the nature, it can achieve generalized performance as well as alleviate overfitting. However, it does not efficiently perform resource-aware inference from a single trained architecture. To address the issue, we aim to build a learning framework that minimizes the cost to infer tasks under different memory budgets. To this end, we propose a multi-path network with a self-auxiliary learning strategy. The multi-path structure contains task-specific paths in a backbone network, where a lower-level path predicts earlier with a smaller number of parameters. To alleviate the performance degradation from earlier predictions, a self-auxiliary learning strategy is presented. The self-auxiliary tasks convey task-specific knowledge to the main tasks to compensate for the performance leak. We evaluate the proposed method on an extensive set of multi-task learning scenarios, including multiple tasks learning, hierarchical learning, and curriculum learning. The proposed method outperforms existing multi-task learning competitors for most scenarios about by a margin of 1% ~ 2% accuracy on average while consuming 30% ~ 60% smaller computational cost. Author
- Files in This Item
-
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.