Gating Mechanism in Deep Neural Networks for Resource-Efficient Continual Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Jin, H. | - |
dc.contributor.author | Yun, K. | - |
dc.contributor.author | Kim, Eun Woo | - |
dc.date.accessioned | 2023-03-08T09:31:53Z | - |
dc.date.available | 2023-03-08T09:31:53Z | - |
dc.date.issued | 2022 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61910 | - |
dc.description.abstract | Catastrophic forgetting is well-known tendency in continual learning of a deep neural network to forget previously learned knowledge when optimizing for sequentially incoming tasks. To address the issue, several methods have been proposed in research on continual learning. However, theses methods cannot preserve the previously learned knowledge when training for a new task. Moreover, these methods are susceptible to negative interference between tasks, which may lead to catastrophic forgetting. It even becomes increasingly severe when there exists a notable gap between the domains of tasks. This paper proposes a novel method of controlling gates to select a subset of parameters learned for old tasks, which are then used to efficiently optimize a new task while avoiding negative interference. The proposed approach executes the subset of old parameters that provides positive responses by evaluating the effect when the old and new parameters are used together. The execution or skipping of old parameters through the gates is based on several responses across the network. We evaluate the proposed method in different continual learning scenarios involving image classification datasets. The proposed method outperforms other competitive methods and requires fewer parameters than the state-of-the-art methods during inference by applying the proposed gating mechanism that selectively involves a set of old parameters that provides positive prior knowledge to newer tasks. Additionally, we further prove the effectiveness of the proposed method through various analyses. Author | - |
dc.format.extent | 11 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | - |
dc.title | Gating Mechanism in Deep Neural Networks for Resource-Efficient Continual Learning | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/ACCESS.2022.3147237 | - |
dc.identifier.bibliographicCitation | IEEE Access, v.10, pp 18776 - 18786 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.wosid | 000760720400001 | - |
dc.identifier.scopusid | 2-s2.0-85124093534 | - |
dc.citation.endPage | 18786 | - |
dc.citation.startPage | 18776 | - |
dc.citation.title | IEEE Access | - |
dc.citation.volume | 10 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | Continual learning | - |
dc.subject.keywordAuthor | Feature extraction | - |
dc.subject.keywordAuthor | gating mechanism | - |
dc.subject.keywordAuthor | Interference | - |
dc.subject.keywordAuthor | Logic gates | - |
dc.subject.keywordAuthor | Neural networks | - |
dc.subject.keywordAuthor | Resource management | - |
dc.subject.keywordAuthor | resource-efficient learning | - |
dc.subject.keywordAuthor | Task analysis | - |
dc.subject.keywordAuthor | task interference | - |
dc.subject.keywordAuthor | Training | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.