An Efficient Optimization Technique for Training Deep Neural Networksopen access
- Authors
- Mehmood, Faisal; Ahmad, Shabir; Whangbo, Taeg Keun
- Issue Date
- Mar-2023
- Publisher
- MDPI
- Keywords
- machine learning; deep learning; neural network
- Citation
- MATHEMATICS, v.11, no.6
- Journal Title
- MATHEMATICS
- Volume
- 11
- Number
- 6
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/87734
- DOI
- 10.3390/math11061360
- ISSN
- 2227-7390
- Abstract
- Deep learning is a sub-branch of artificial intelligence that acquires knowledge by training a neural network. It has many applications in the field of banking, automobile industry, agriculture, and healthcare industry. Deep learning has played a significant role in solving complex tasks related to computer vision, such as image classification, natural language processing, and object detection. On the other hand, optimizers also play an intrinsic role in training the deep learning model. Recent studies have proposed many deep learning models, such as VGG, ResNet, DenseNet, and ImageNet. In addition, there are many optimizers such as stochastic gradient descent (SGD), Adam, AdaDelta, Adabelief, and AdaMax. In this study, we have selected those models that require lower hardware requirements and shorter training times, which facilitates the overall training process. We have modified the Adam based optimizers and minimized the cyclic path. We have removed an additional hyper-parameter from RMSProp and observed that the optimizer works with various models. The learning rate is set to minimum and constant. The initial weights are updated after each epoch, which helps to improve the accuracy of the model. We also changed the position of the epsilon in the default Adam optimizer. By changing the position of the epsilon, it accumulates the updating process. We used various models with SGD, Adam, RMSProp, and the proposed optimization technique. The results indicate that the proposed method is effective in achieving the accuracy and works well with the state-of-the-art architectures.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - IT융합대학 > 컴퓨터공학과 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/87734)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.