Neural Tangent Kernel Analysis of Deep Narrow Neural Networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, J. | - |
dc.contributor.author | Choi, J.Y. | - |
dc.contributor.author | Ryu, E.K. | - |
dc.contributor.author | No, A. | - |
dc.date.accessioned | 2023-07-11T02:40:14Z | - |
dc.date.available | 2023-07-11T02:40:14Z | - |
dc.date.created | 2023-07-11 | - |
dc.date.issued | 2022 | - |
dc.identifier.issn | 2640-3498 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hongik/handle/2020.sw.hongik/31495 | - |
dc.description.abstract | The tremendous recent progress in analyzing the training dynamics of overparameterized neural networks has primarily focused on wide networks and therefore does not sufficiently address the role of depth in deep learning. In this work, we present the first trainability guarantee of infinitely deep but narrow neural networks. We study the infinite-depth limit of a multilayer perceptron (MLP) with a specific initialization and establish a trainability guarantee using the NTK theory. We then extend the analysis to an infinitely deep convolutional neural network (CNN) and perform brief experiments. Copyright © 2022 by the author(s) | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | ML Research Press | - |
dc.title | Neural Tangent Kernel Analysis of Deep Narrow Neural Networks | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | No, A. | - |
dc.identifier.scopusid | 2-s2.0-85138682628 | - |
dc.identifier.bibliographicCitation | Proceedings of Machine Learning Research, v.162, pp.12282 - 12351 | - |
dc.relation.isPartOf | Proceedings of Machine Learning Research | - |
dc.citation.title | Proceedings of Machine Learning Research | - |
dc.citation.volume | 162 | - |
dc.citation.startPage | 12282 | - |
dc.citation.endPage | 12351 | - |
dc.type.rims | ART | - |
dc.type.docType | Conference Paper | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
94, Wausan-ro, Mapo-gu, Seoul, 04066, Korea02-320-1314
COPYRIGHT 2020 HONGIK UNIVERSITY. ALL RIGHTS RESERVED.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.