Robust quantization of deep neural networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kim, Youngseok | - |
dc.contributor.author | Lee, Junyeol | - |
dc.contributor.author | Kim, Younghoon | - |
dc.contributor.author | Seo, Jiwon | - |
dc.date.accessioned | 2021-06-22T09:11:07Z | - |
dc.date.available | 2021-06-22T09:11:07Z | - |
dc.date.created | 2021-01-22 | - |
dc.date.issued | 2020-02 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/1521 | - |
dc.description.abstract | We studied robust quantization of deep neural networks (DNNs) for embedded devices. Existing compression techniques often generate DNNs that are sensitive to external errors. Because embedded devices may be affected by external lights and outside weather, DNNs running on those devices must be robust to such errors. For robust quantization of DNNs, we formulate an optimization problem that finds the bit width for each layer minimizing the robustness loss. To efficiently find the solution, we design a dynamic programming based algorithm, called Qed. We also propose an incremental algorithm, Q∗ that quickly finds a reasonably robust quantization and then gradually improves it. We have evaluated Qed and Q∗ with three DNN models (LeNet, AlexNet, and VGG-16) and with Gaussian random errors and realistic errors. For comparison, we also evaluate universal quantization that uses equal bit width for all layers and Deep Compression, a weight-sharing based compression technique. When tested with increasing size of errors, Qed most robustly gives correct inference output. Even if a DNN is optimized for robustness, its quantizations may not be robust unless Qed is used. Moreover, we evaluate Q∗ for its trade off in execution time and robustness. In one tenth of Qed's execution time, Q∗ gives a quantization 98% as robust as the one by Qed. © 2020 Association for Computing Machinery. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | Association for Computing Machinery, Inc | - |
dc.title | Robust quantization of deep neural networks | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Kim, Younghoon | - |
dc.identifier.doi | 10.1145/3377555.3377900 | - |
dc.identifier.scopusid | 2-s2.0-85082088697 | - |
dc.identifier.wosid | 000671030900007 | - |
dc.identifier.bibliographicCitation | CC 2020 - Proceedings of the 29th International Conference on Compiler Construction, pp.74 - 84 | - |
dc.relation.isPartOf | CC 2020 - Proceedings of the 29th International Conference on Compiler Construction | - |
dc.citation.title | CC 2020 - Proceedings of the 29th International Conference on Compiler Construction | - |
dc.citation.startPage | 74 | - |
dc.citation.endPage | 84 | - |
dc.type.rims | ART | - |
dc.type.docType | Conference Paper | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Software Engineering | - |
dc.subject.keywordPlus | Dynamic programming | - |
dc.subject.keywordPlus | Economic and social effects | - |
dc.subject.keywordPlus | Program compilers | - |
dc.subject.keywordPlus | Random errors | - |
dc.subject.keywordPlus | Bit-Width | - |
dc.subject.keywordPlus | Compression techniques | - |
dc.subject.keywordPlus | Embedded device | - |
dc.subject.keywordPlus | Gaussian random errors | - |
dc.subject.keywordPlus | Incremental algorithm | - |
dc.subject.keywordPlus | Optimization problems | - |
dc.subject.keywordPlus | Trade off | - |
dc.subject.keywordPlus | Universal quantizations | - |
dc.subject.keywordPlus | Deep neural networks | - |
dc.subject.keywordAuthor | Neural Network Quantization | - |
dc.identifier.url | https://dl.acm.org/doi/10.1145/3377555.3377900 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
55 Hanyangdeahak-ro, Sangnok-gu, Ansan, Gyeonggi-do, 15588, Korea+82-31-400-4269 sweetbrain@hanyang.ac.kr
COPYRIGHT © 2021 HANYANG UNIVERSITY. ALL RIGHTS RESERVED.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.