Domain Generalization by Mutual-Information Regularization with Pre-trained Models
- Authors
- Cha, J.; Lee, Kyungjae; Park, S.; Chun, S.
- Issue Date
- Oct-2022
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Citation
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), v.13683 LNCS, pp 440 - 457
- Pages
- 18
- Journal Title
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- Volume
- 13683 LNCS
- Start Page
- 440
- End Page
- 457
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61189
- DOI
- 10.1007/978-3-031-20050-2_26
- ISSN
- 0302-9743
1611-3349
- Abstract
- Domain generalization (DG) aims to learn a generalized model to an unseen target domain using only limited source domains. Previous attempts to DG fail to learn domain-invariant representations only from the source domains due to the significant domain shifts between training and test domains. Instead, we re-formulate the DG objective using mutual information with the oracle model, a model generalized to any possible domain. We derive a tractable variational lower bound via approximating the oracle model by a pre-trained model, called Mutual Information Regularization with Oracle (MIRO). Our extensive experiments show that MIRO significantly improves the out-of-distribution performance. Furthermore, our scaling experiments show that the larger the scale of the pre-trained model, the greater the performance improvement of MIRO. Code is available at https://github.com/kakaobrain/miro. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > Department of Artificial Intelligence > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/61189)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.