Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning
- Authors
- Baik, Sungyong; Choi, Myungsub; Choi, Janghoon; Kim, Heewon; Lee, Kyoung Mu
- Issue Date
- Mar-2024
- Publisher
- IEEE COMPUTER SOC
- Keywords
- Task analysis; Optimization; Mathematical models; Adaptation models; Visualization; Training; Neural networks; Few-shot learning; MAML; meta-learning; video frame interpolation; visual tracking
- Citation
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, v.46, no.3, pp 1441 - 1454
- Pages
- 14
- Journal Title
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
- Volume
- 46
- Number
- 3
- Start Page
- 1441
- End Page
- 1454
- URI
- https://scholarworks.bwise.kr/ssu/handle/2018.sw.ssu/49454
- DOI
- 10.1109/TPAMI.2023.3261387
- ISSN
- 0162-8828
1939-3539
- Abstract
- The objective of few-shot learning is to design a system that can adapt to a given task with only few examples while achieving generalization. Model-agnostic meta-learning (MAML), which has recently gained the popularity for its simplicity and flexibility, learns a good initialization for fast adaptation to a task under few-data regime. However, its performance has been relatively limited especially when novel tasks are different from tasks previously seen during training. In this work, instead of searching for a better initialization, we focus on designing a better fast adaptation process. Consequently, we propose a new task-adaptive weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can generate per-step hyperparameters for each given task: learning rate and weight decay coefficients. The experimental results validate that learning a good weight update rule for fast adaptation is the equally important component that has drawn relatively less attention in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML. Furthermore, the proposed weight-update rule is shown to consistently improve the task-adaptation capability of MAML across diverse problem domains: few-shot classification, cross-domain few-shot classification, regression, visual tracking, and video frame interpolation.
- Files in This Item
-
Go to Link
- Appears in
Collections - ETC > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.