Auto-VirtualNet: Cost-adaptive dynamic architecture search for multi-task learning
- Authors
- Kim, E.; Ahn, C.; Oh, S.
- Issue Date
- 28-Jun-2021
- Publisher
- Elsevier B.V.
- Keywords
- Cost-adaptive learning; Dynamic architecture search; Multi-task learning; Task interference
- Citation
- Neurocomputing, v.442, pp 116 - 124
- Pages
- 9
- Journal Title
- Neurocomputing
- Volume
- 442
- Start Page
- 116
- End Page
- 124
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/44119
- DOI
- 10.1016/j.neucom.2021.02.050
- ISSN
- 0925-2312
1872-8286
- Abstract
- Multi-task learning (MTL) improves learning efficiency by solving multiple tasks simultaneously compared to multiple instances of individual learning. However, despite its benefits, there still remain several major challenges: first, negative interference can reduce the learning efficiency when the number of tasks is high or the tasks are of limited relevance. Second, exploring an optimal model structure manually is quite restricted. Last but not least, offering cost-adaptive solutions has not been addressed in the MTL regime. In spite of its notable merits, the combined problem has not been well discussed. In this work, we propose a novel MTL approach to address the combinatorial problem while minimizing memory consumption. The proposed method discovers multiple network models dynamically from a pool of candidate models, and produces a set of widely distributed solutions with respect to different computational costs for each task. For the diversity of candidate models, we modularize the given backbone architecture that generates basic building blocks and then construct a hierarchical structure based on the building blocks. The proposed method is trained to optimize both task performance and computational costs of selected models. The proposed method dynamically generates optimal networks for each task and offers significant performance improvements over existing MTL approaches in a range of experiments. © 2021 Elsevier B.V.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/44119)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.