DNN compression by ADMM-based joint pruning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Geonseok | - |
dc.contributor.author | Lee, Kichun | - |
dc.date.accessioned | 2022-07-06T08:44:08Z | - |
dc.date.available | 2022-07-06T08:44:08Z | - |
dc.date.created | 2022-01-26 | - |
dc.date.issued | 2022-03 | - |
dc.identifier.issn | 0950-7051 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/139362 | - |
dc.description.abstract | The success of deep neural networks (DNNs) has motivated pursuit of both computationally and memory efficient models for applications in resource-constrained systems such as embedded devices. In line with this trend, network pruning methods reducing redundancy in over-parameterized models are being studied actively. Previous works on this research have demonstrated the ability to learn a compact network by imposing sparsity constraints on the parameters, but most of them have difficulty not only in identifying both connections and neurons to be pruned, but also in converging to optimal solutions. We propose a systematic DNN compression method where weights and network architectures are jointly optimized. We solve the joint problem using alternating direction method of multipliers (ADMM), a powerful technique capable of handling non-convex separable programming. Additionally, we provide a holistic pruning approach, an integrated form of our method, for automatically pruning networks without specific layer-wise hyper-parameters. To verify our work, we deployed the proposed method to a variety of state-of-the-art convolutional neural networks (CNNs) on three image classification benchmark datasets: MNIST, CIFAR-10, and ImageNet. Results show that the proposed pruning method effectively compresses the network parameters and reduces the computation cost while preserving prediction accuracy. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | Elsevier B.V. | - |
dc.title | DNN compression by ADMM-based joint pruning | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Lee, Kichun | - |
dc.identifier.doi | 10.1016/j.knosys.2021.107988 | - |
dc.identifier.scopusid | 2-s2.0-85122205221 | - |
dc.identifier.wosid | 000788495000021 | - |
dc.identifier.bibliographicCitation | Knowledge-Based Systems, v.239, pp.1 - 11 | - |
dc.relation.isPartOf | Knowledge-Based Systems | - |
dc.citation.title | Knowledge-Based Systems | - |
dc.citation.volume | 239 | - |
dc.citation.startPage | 1 | - |
dc.citation.endPage | 11 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.subject.keywordPlus | Classification (of information) | - |
dc.subject.keywordPlus | Convolutional neural networks | - |
dc.subject.keywordPlus | Embedded systems | - |
dc.subject.keywordPlus | Multilayer neural networks | - |
dc.subject.keywordPlus | Network architecture | - |
dc.subject.keywordPlus | Network layers | - |
dc.subject.keywordPlus | Alternative direction method of multiplier | - |
dc.subject.keywordPlus | Computationally efficient | - |
dc.subject.keywordPlus | Memory efficient | - |
dc.subject.keywordPlus | Method of multipliers | - |
dc.subject.keywordPlus | Network compression | - |
dc.subject.keywordPlus | Neural network compression | - |
dc.subject.keywordPlus | Neural-networks | - |
dc.subject.keywordPlus | Pruning methods | - |
dc.subject.keywordPlus | Structured pruning | - |
dc.subject.keywordPlus | Unstructured pruning | - |
dc.subject.keywordPlus | Deep neural networks | - |
dc.subject.keywordAuthor | Alternative direction method of multipliers (ADMM) | - |
dc.subject.keywordAuthor | Neural network compression | - |
dc.subject.keywordAuthor | Structured pruning | - |
dc.subject.keywordAuthor | Unstructured pruning | - |
dc.identifier.url | https://www.sciencedirect.com/science/article/pii/S0950705121011047?via%3Dihub | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.