A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Shim, Duk-Sun | - |
dc.contributor.author | Shim, Joseph | - |
dc.date.accessioned | 2023-10-20T02:40:47Z | - |
dc.date.available | 2023-10-20T02:40:47Z | - |
dc.date.issued | 2023-11 | - |
dc.identifier.issn | 1598-6446 | - |
dc.identifier.issn | 2005-4092 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/68183 | - |
dc.description.abstract | An optimization algorithm is essential for minimizing loss (or objective) functions in machine learning and deep learning. Optimization algorithms face several challenges, one among which is to determine an appropriate learning rate. Generally, a low learning rate leads to slow convergence whereas a large learning rate causes the loss function to fluctuate around the minimum. As a hyper-parameter, the learning rate is determined in advance before parameter training, which is time-consuming. This paper proposes a modified stochastic gradient descent (mSGD) algorithm that uses a random learning rate. Random numbers are generated for a learning rate at every iteration, and the one that gives the minimum value of the loss function is chosen. The proposed mSGD algorithm can reduce the time required for determining the learning rate. In fact, the k-point mSGD algorithm can be considered as a kind of steepest descent algorithm. In a real experiment using the MNIST dataset of hand-written digits, it is demonstrated that the convergence performance of mSGD algorithm is much better than that of the SGD algorithm and slightly better than that of the AdaGrad and Adam algorithms. © 2023, ICROS, KIEE and Springer. | - |
dc.format.extent | 7 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Institute of Control, Robotics and Systems | - |
dc.title | A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning | - |
dc.type | Article | - |
dc.identifier.doi | 10.1007/s12555-022-0947-1 | - |
dc.identifier.bibliographicCitation | International Journal of Control, Automation and Systems, v.21, no.11, pp 3825 - 3831 | - |
dc.identifier.kciid | ART003012080 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.wosid | 001060184300005 | - |
dc.identifier.scopusid | 2-s2.0-85168990474 | - |
dc.citation.endPage | 3831 | - |
dc.citation.number | 11 | - |
dc.citation.startPage | 3825 | - |
dc.citation.title | International Journal of Control, Automation and Systems | - |
dc.citation.volume | 21 | - |
dc.type.docType | Article | - |
dc.publisher.location | 대한민국 | - |
dc.subject.keywordAuthor | Deep learning | - |
dc.subject.keywordAuthor | machine learning | - |
dc.subject.keywordAuthor | modified stochastic gradient descent | - |
dc.subject.keywordAuthor | random learning rate | - |
dc.subject.keywordAuthor | steepest descent algorithm | - |
dc.relation.journalResearchArea | Automation & Control Systems | - |
dc.relation.journalWebOfScienceCategory | Automation & Control Systems | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.description.journalRegisteredClass | kci | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.