Direct Conversion: Accelerating Convolutional Neural Networks Utilizing Sparse Input Activation
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Won-Hyu | - |
dc.contributor.author | Roh, Si-Dong | - |
dc.contributor.author | Park, Sangki | - |
dc.contributor.author | Chung, Ki Seok | - |
dc.date.accessioned | 2021-07-30T05:13:23Z | - |
dc.date.available | 2021-07-30T05:13:23Z | - |
dc.date.created | 2021-05-13 | - |
dc.date.issued | 2020-10 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/3654 | - |
dc.description.abstract | The amount of computation and the number of parameters of neural networks are increasing rapidly as the depth of convolutional neural networks (CNNs) is increasing. Therefore, it is very crucial to reduce both the amount of computation and that of memory usage. The pruning method, which compresses a neural network, has been actively studied. Depending on the layer characteristics, the sparsity level of each layer varies significantly after the pruning is conducted. If weights are sparse, most results of convolution operations will be zeroes. Although several studies have proposed methods to utilize the weight sparsity to avoid carrying out meaningless operations, those studies lack consideration that input activations may also have a high sparsity level. The Rectified Linear Unit (ReLU) function is one of the most popular activation functions because it is simple and yet pretty effective. Due to properties of the ReLU function, it is often observed that the input activation sparsity level is high (up to 85%). Therefore, it is important to consider both the input activation sparsity and the weight one to accelerate CNN to minimize carrying out meaningless computation. In this paper, we propose a new acceleration method called Direct Conversion that considers the weight sparsity under the sparse input activation condition. The Direct Conversion method converts a 3D input tensor directly into a compressed format. This method selectively applies one of two different methods: a method called image to Compressed Sparse Row (im2CSR) when input activations are sparse and weights are dense; the other method called image to Compressed Sparse Overlapped Activations (im2CSOA) when both input activations and weights are sparse. Our experimental results show that Direct Conversion improves the inference speed up to 2.82× compared to the conventional method. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | IEEE Computer Society | - |
dc.title | Direct Conversion: Accelerating Convolutional Neural Networks Utilizing Sparse Input Activation | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Chung, Ki Seok | - |
dc.identifier.doi | 10.1109/IECON43393.2020.9254473 | - |
dc.identifier.scopusid | 2-s2.0-85097749059 | - |
dc.identifier.bibliographicCitation | IECON Proceedings (Industrial Electronics Conference), v.2020, no.October, pp.441 - 446 | - |
dc.relation.isPartOf | IECON Proceedings (Industrial Electronics Conference) | - |
dc.citation.title | IECON Proceedings (Industrial Electronics Conference) | - |
dc.citation.volume | 2020 | - |
dc.citation.number | October | - |
dc.citation.startPage | 441 | - |
dc.citation.endPage | 446 | - |
dc.type.rims | ART | - |
dc.type.docType | Conference Paper | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scopus | - |
dc.subject.keywordPlus | Chemical activation | - |
dc.subject.keywordPlus | Convolution | - |
dc.subject.keywordPlus | Industrial electronics | - |
dc.subject.keywordPlus | Acceleration method | - |
dc.subject.keywordPlus | Activation conditions | - |
dc.subject.keywordPlus | Activation functions | - |
dc.subject.keywordPlus | Compressed sparse row | - |
dc.subject.keywordPlus | Conventional methods | - |
dc.subject.keywordPlus | Direct conversion | - |
dc.subject.keywordPlus | Linear units | - |
dc.subject.keywordPlus | Pruning methods | - |
dc.subject.keywordPlus | Convolutional neural networks | - |
dc.subject.keywordAuthor | convolutional neural network | - |
dc.subject.keywordAuthor | embedded system | - |
dc.subject.keywordAuthor | sparsity-aware acceleration | - |
dc.identifier.url | https://ieeexplore.ieee.org/document/9254473 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.