Rollback Ensemble With Multiple Local Minima in Fine-Tuning Deep Learning Networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Ro, Y. | - |
dc.contributor.author | Choi, J. | - |
dc.contributor.author | Heo, B. | - |
dc.contributor.author | Choi, J.Y. | - |
dc.date.accessioned | 2023-03-08T11:10:27Z | - |
dc.date.available | 2023-03-08T11:10:27Z | - |
dc.date.issued | 2022-09 | - |
dc.identifier.issn | 2162-237X | - |
dc.identifier.issn | 2162-2388 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/62555 | - |
dc.description.abstract | Image retrieval is a challenging problem that requires learning generalized features enough to identify untrained classes, even with very few classwise training samples. In this article, to obtain generalized features further in learning retrieval data sets, we propose a novel fine-tuning method of pretrained deep networks. In the retrieval task, we discovered a phenomenon in which the loss reduction in fine-tuning deep networks is stagnated, even while weights are largely updated. To escape from the stagnated state, we propose a new fine-tuning strategy to roll back some of the weights to the pretrained values. The rollback scheme is observed to drive the learning path to a gentle basin that provides more generalized features than a sharp basin. In addition, we propose a multihead ensemble structure to create synergy among multiple local minima obtained by our rollback scheme. Experimental results show that the proposed learning method significantly improves generalization performance, achieving state-of-the-art performance on the Inshop and SOP data sets. IEEE | - |
dc.format.extent | 13 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | - |
dc.title | Rollback Ensemble With Multiple Local Minima in Fine-Tuning Deep Learning Networks | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/TNNLS.2021.3059669 | - |
dc.identifier.bibliographicCitation | IEEE Transactions on Neural Networks and Learning Systems, v.33, no.9, pp 4648 - 4660 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.wosid | 000732292700001 | - |
dc.identifier.scopusid | 2-s2.0-85102302828 | - |
dc.citation.endPage | 4660 | - |
dc.citation.number | 9 | - |
dc.citation.startPage | 4648 | - |
dc.citation.title | IEEE Transactions on Neural Networks and Learning Systems | - |
dc.citation.volume | 33 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | Deep neural network | - |
dc.subject.keywordAuthor | fine-tuning | - |
dc.subject.keywordAuthor | Generative adversarial networks | - |
dc.subject.keywordAuthor | Image retrieval | - |
dc.subject.keywordAuthor | image retrieval | - |
dc.subject.keywordAuthor | learning strategy | - |
dc.subject.keywordAuthor | Learning systems | - |
dc.subject.keywordAuthor | Neural networks | - |
dc.subject.keywordAuthor | person reidentification. | - |
dc.subject.keywordAuthor | Task analysis | - |
dc.subject.keywordAuthor | Training | - |
dc.subject.keywordAuthor | Training data | - |
dc.subject.keywordPlus | Digital storage | - |
dc.subject.keywordPlus | Learning systems | - |
dc.subject.keywordPlus | Ensemble structures | - |
dc.subject.keywordPlus | Fine-tuning methods | - |
dc.subject.keywordPlus | Generalization performance | - |
dc.subject.keywordPlus | Learning methods | - |
dc.subject.keywordPlus | Learning network | - |
dc.subject.keywordPlus | Loss reduction | - |
dc.subject.keywordPlus | State-of-the-art performance | - |
dc.subject.keywordPlus | Training sample | - |
dc.subject.keywordPlus | Deep learning | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Hardware & Architecture | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Theory & Methods | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.