Surrogate-Assisted Hybrid-Model Estimation of Distribution Algorithm for Mixed-Variable Hyperparameters Optimization in Convolutional Neural Networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Li, Jian-Yu | - |
dc.contributor.author | Zhan, Zhi-Hui | - |
dc.contributor.author | Xu, Jin | - |
dc.contributor.author | Kwong, Sam | - |
dc.contributor.author | Zhang, Jun | - |
dc.date.accessioned | 2024-05-02T02:30:25Z | - |
dc.date.available | 2024-05-02T02:30:25Z | - |
dc.date.issued | 2023-05 | - |
dc.identifier.issn | 2162-237X | - |
dc.identifier.issn | 2162-2388 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/118933 | - |
dc.description.abstract | The performance of a convolutional neural network (CNN) heavily depends on its hyperparameters. However, finding a suitable hyperparameters configuration is difficult, challenging, and computationally expensive due to three issues, which are 1) the mixed-variable problem of different types of hyperparameters; 2) the large-scale search space of finding optimal hyperparameters; and 3) the expensive computational cost for evaluating candidate hyperparameters configuration. Therefore, this article focuses on these three issues and proposes a novel estimation of distribution algorithm (EDA) for efficient hyperparameters optimization, with three major contributions in the algorithm design. First, a hybrid-model EDA is proposed to efficiently deal with the mixed-variable difficulty. The proposed algorithm uses a mixed-variable encoding scheme to encode the mixed-variable hyperparameters and adopts an adaptive hybrid-model learning (AHL) strategy to efficiently optimize the mixed-variables. Second, an orthogonal initialization (OI) strategy is proposed to efficiently deal with the challenge of large-scale search space. Third, a surrogate-assisted multi-level evaluation (SME) method is proposed to reduce the expensive computational cost. Based on the above, the proposed algorithm is named surrogate-assisted hybrid-model EDA (SHEDA). For experimental studies, the proposed SHEDA is verified on widely used classification benchmark problems, and is compared with various state-of-the-art methods. Moreover, a case study on aortic dissection (AD) diagnosis is carried out to evaluate its performance. Experimental results show that the proposed SHEDA is very effective and efficient for hyperparameters optimization, which can find a satisfactory hyperparameters configuration for the CIFAR10, CIFAR100, and AD diagnosis with only 0.58, 0.97, and 1.18 GPU days, respectively. | - |
dc.format.extent | 15 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | IEEE Computational Intelligence Society | - |
dc.title | Surrogate-Assisted Hybrid-Model Estimation of Distribution Algorithm for Mixed-Variable Hyperparameters Optimization in Convolutional Neural Networks | - |
dc.type | Article | - |
dc.publisher.location | 미국 | - |
dc.identifier.doi | 10.1109/TNNLS.2021.3106399 | - |
dc.identifier.scopusid | 2-s2.0-85115675797 | - |
dc.identifier.wosid | 000732122600001 | - |
dc.identifier.bibliographicCitation | IEEE Transactions on Neural Networks and Learning Systems, v.34, no.5, pp 2338 - 2352 | - |
dc.citation.title | IEEE Transactions on Neural Networks and Learning Systems | - |
dc.citation.volume | 34 | - |
dc.citation.number | 5 | - |
dc.citation.startPage | 2338 | - |
dc.citation.endPage | 2352 | - |
dc.type.docType | Article | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Hardware & Architecture | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Theory & Methods | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.subject.keywordPlus | PARTICLE SWARM OPTIMIZATION | - |
dc.subject.keywordPlus | EVOLUTIONARY ALGORITHM | - |
dc.subject.keywordAuthor | Optimization | - |
dc.subject.keywordAuthor | Convolutional neural networks | - |
dc.subject.keywordAuthor | Estimation | - |
dc.subject.keywordAuthor | Computational modeling | - |
dc.subject.keywordAuthor | Brain modeling | - |
dc.subject.keywordAuthor | Probabilistic logic | - |
dc.subject.keywordAuthor | Feature extraction | - |
dc.subject.keywordAuthor | Aortic dissection (AD) diagnosis | - |
dc.subject.keywordAuthor | convolutional neural network (CNN) | - |
dc.subject.keywordAuthor | deep learning | - |
dc.subject.keywordAuthor | estimation of distribution algorithm (EDA) | - |
dc.subject.keywordAuthor | evolutionary computation (EC) | - |
dc.subject.keywordAuthor | hybrid model | - |
dc.subject.keywordAuthor | hyperparameters optimization | - |
dc.subject.keywordAuthor | mixed variable | - |
dc.identifier.url | https://ieeexplore.ieee.org/document/9540902 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
55 Hanyangdeahak-ro, Sangnok-gu, Ansan, Gyeonggi-do, 15588, Korea+82-31-400-4269 sweetbrain@hanyang.ac.kr
COPYRIGHT © 2021 HANYANG UNIVERSITY. ALL RIGHTS RESERVED.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.