Helpful or Harmful: Inter-task Association in Continual Learning
- Authors
- Jin, H.; Kim, Eunwoo
- Issue Date
- Oct-2022
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Keywords
- Continual learning; Model search; Task association
- Citation
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), v.13671 LNCS, pp 519 - 535
- Pages
- 17
- Journal Title
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- Volume
- 13671 LNCS
- Start Page
- 519
- End Page
- 535
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61190
- DOI
- 10.1007/978-3-031-20083-0_31
- ISSN
- 0302-9743
1611-3349
- Abstract
- When optimizing sequentially incoming tasks, deep neural networks generally suffer from catastrophic forgetting due to their lack of ability to maintain knowledge from old tasks. This may lead to a significant performance drop of the previously learned tasks. To alleviate this problem, studies on continual learning have been conducted as a countermeasure. Nevertheless, it suffers from an increase in computational cost due to the expansion of the network size or a change in knowledge that is favorably linked to previous tasks. In this work, we propose a novel approach to differentiate helpful and harmful information for old tasks using a model search to learn a current task effectively. Given a new task, the proposed method discovers an underlying association knowledge from old tasks, which can provide additional support in acquiring the new task knowledge. In addition, by introducing a sensitivity measure to the loss of the current task from the associated tasks, we find cooperative relations between tasks while alleviating harmful interference. We apply the proposed approach to both task- and class-incremental scenarios in continual learning, using a wide range of datasets from small to large scales. Experimental results show that the proposed method outperforms a large variety of continual learning approaches for the experiments while effectively alleviating catastrophic forgetting. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61190)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.