경량 딥러닝 모델의 초저정밀도 양자화를 위한 학습 방식의 개선
- Authors
- 김현승; 김민수; 최정욱
- Issue Date
- Nov-2020
- Publisher
- 대한전자공학회
- Citation
- 2020년도 대한전자공학회 추계학술대회 논문집, pp.601 - 604
- Indexed
- OTHER
- Journal Title
- 2020년도 대한전자공학회 추계학술대회 논문집
- Start Page
- 601
- End Page
- 604
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/192369
- Abstract
- Deep Learning Model Quantization is the most effective technique to make a model much lighter and cost efficient in terms of computation.
Above many quantization algorithms, PROFIT[1] is a specialized algorithm for sub 4-bit mobile network quantization. But this method has sudden accuracy degradation in 2-bit width precision.
In this paper, we propose a better training method to deal with this problem in 2-bit weight quantization. We adopt AIWQ, a metric for the activation’s instability induced by weight quantization [1] and make threshold value with this metric. Using threshold value, we stop training some quantized layers which have high sensitivity to weight quantization and fine-tune the rest of the quantized layers with different learning rate and scheduler. With this advanced training method, we improved 2-bit weight quantization accuracy of light deep learning models including EfficientNetB0 and MobilenetV2.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 융합전자공학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/192369)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.