Synchronization-Aware NAS for an Efficient Collaborative Inference on Mobile Platforms
- Authors
- Kang, Beom Woo; Wohn, Junho; Lee, Seongju; Park, Sunghyun; Noh, Yung-Kyun; Park, Yongjun
- Issue Date
- Jun-2023
- Publisher
- Association for Computing Machinery
- Keywords
- Model Parallelism; Neural Architecture Search; On-Device ML
- Citation
- Proceedings of the ACM SIGPLAN Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES), pp 13 - 25
- Pages
- 13
- Indexed
- SCOPUS
- Journal Title
- Proceedings of the ACM SIGPLAN Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES)
- Start Page
- 13
- End Page
- 25
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/188399
- DOI
- 10.1145/3589610.3596284
- Abstract
- Previous neural architecture search (NAS) approaches for mobile platforms have achieved great success in designing a slim-but-accurate neural network that is generally well-matched to a single computing unit such as a CPU or GPU. However, as recent mobile devices consist of multiple heterogeneous computing units, the next main challenge is to maximize both accuracy and efficiency by fully utilizing multiple available resources. We propose an ensemble-like approach with intermediate feature aggregations, namely synchronizations, for active collaboration between individual models on a mobile device. A main challenge is to determine the optimal synchronization strategies for achieving both performance and efficiency. To this end, we propose SyncNAS to automate the exploration of synchronization strategies for collaborative neural architectures that maximize utilization of heterogeneous computing units on a target device. We introduce a novel search space for synchronization strategy and apply Monte Carlo tree search (MCTS) algorithm to improve the sampling efficiency and reduce the search cost. On ImageNet, our collaborative model based on MobileNetV2 achieves 2.7% top-1 accuracy improvement within the baseline latency budget. Under the reduced target latency down to half, our model maintains higher accuracy than its baseline model, owing to the enhanced utilization and collaboration. As an impact of MCTS, SyncNAS reduces its search cost by up to 21× in searching for the optimal strategy.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/188399)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.