Learning-Rate Annealing Methods for Deep Neural Networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Nakamura, Kensuke | - |
dc.contributor.author | Derbel, Bilel | - |
dc.contributor.author | Won, Kyoung-Jae | - |
dc.contributor.author | Hong, Byung-Woo | - |
dc.date.accessioned | 2021-09-13T03:40:10Z | - |
dc.date.available | 2021-09-13T03:40:10Z | - |
dc.date.issued | 2021-08 | - |
dc.identifier.issn | 2079-9292 | - |
dc.identifier.issn | 2079-9292 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/49038 | - |
dc.description.abstract | Deep neural networks (DNNs) have achieved great success in the last decades. DNN is optimized using the stochastic gradient descent (SGD) with learning rate annealing that overtakes the adaptive methods in many tasks. However, there is no common choice regarding the scheduled-annealing for SGD. This paper aims to present empirical analysis of learning rate annealing based on the experimental results using the major data-sets on the image classification that is one of the key applications of the DNNs. Our experiment involves recent deep neural network models in combination with a variety of learning rate annealing methods. We also propose an annealing combining the sigmoid function with warmup that is shown to overtake both the adaptive methods and the other existing schedules in accuracy in most cases with DNNs. | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | MDPI | - |
dc.title | Learning-Rate Annealing Methods for Deep Neural Networks | - |
dc.type | Article | - |
dc.identifier.doi | 10.3390/electronics10162029 | - |
dc.identifier.bibliographicCitation | ELECTRONICS, v.10, no.16 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.wosid | 000689928000001 | - |
dc.identifier.scopusid | 2-s2.0-85113751368 | - |
dc.citation.number | 16 | - |
dc.citation.title | ELECTRONICS | - |
dc.citation.volume | 10 | - |
dc.type.docType | Article | - |
dc.publisher.location | 스위스 | - |
dc.subject.keywordAuthor | learning rate annealing | - |
dc.subject.keywordAuthor | stochastic gradient descent | - |
dc.subject.keywordAuthor | image classification | - |
dc.subject.keywordPlus | OPTIMIZATION METHODS | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Physics | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Physics, Applied | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.