Learning to Forget for Meta-Learning via Task-and-Layer-Wise Attenuation
- Authors
- Baik, Sungyong; Oh, Junghoon; Hong, Seokil; Lee, Kyoung Mu
- Issue Date
- Nov-2022
- Publisher
- IEEE COMPUTER SOC
- Keywords
- Task analysis; Optimization; Adaptation models; Attenuation; Knowledge engineering; Visualization; Neural networks; Meta-learning; few-shot learning; MAML; reinforcement learning; visual tracking
- Citation
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, v.44, no.11, pp.7718 - 7730
- Indexed
- SCIE
SCOPUS
- Journal Title
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
- Volume
- 44
- Number
- 11
- Start Page
- 7718
- End Page
- 7730
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/187270
- DOI
- 10.1109/TPAMI.2021.3102098
- ISSN
- 0162-8828
- Abstract
- Few-shot learning is an emerging yet challenging problem in which the goal is to achieve generalization from only few examples. Meta-learning tackles few-shot learning via the learning of prior knowledge shared across tasks and using it to learn new tasks. One of the most representative meta-learning algorithms is the model-agnostic meta-learning (MAML), which formulates prior knowledge as a common initialization, a shared starting point from where a learner can quickly adapt to unseen tasks. However, forcibly sharing an initialization can lead to conflicts among tasks and the compromised (undesired by tasks) location on optimization landscape, thereby hindering task adaptation. Furthermore, the degree of conflict is observed to vary not only among the tasks but also among the layers of a neural network. Thus, we propose task-and-layer-wise attenuation on the compromised initialization to reduce its adverse influence on task adaptation. As attenuation dynamically controls (or selectively forgets) the influence of the compromised prior knowledge for a given task and each layer, we name our method Learn to Forget (L2F). Experimental results demonstrate that the proposed method greatly improves the performance of the state-of-the-art MAML-based frameworks across diverse domains: few-shot classification, cross-domain few-shot classification, regression, reinforcement learning, and visual tracking.
- Files in This Item
-
Go to Link
- Appears in
Collections - ETC > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.