OPTIMUS: OPTImized matrix MUltiplication Structure for Transformer neural network accelerator
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Park, Junki | - |
dc.contributor.author | Yoon, Hyunsung | - |
dc.contributor.author | Ahn, Daehyun | - |
dc.contributor.author | Choi, Jungwook | - |
dc.contributor.author | Kim, Jae-Joon | - |
dc.date.accessioned | 2021-07-30T04:53:52Z | - |
dc.date.available | 2021-07-30T04:53:52Z | - |
dc.date.created | 2021-05-14 | - |
dc.date.issued | 2020-03 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/1988 | - |
dc.description.abstract | We present a high-performance Transformer neural network inference accelerator named OPTIMUS. OPTIMUS has several features for performance enhancement such as the redundant computation skipping method to accelerate the decoding process and the Set-Associative RCSC (SA-RCSC) sparse matrix format to maintain high utilization even when large number of MACs are used in hardware. OPTIMUS also has a flexible hardware architecture to support diverse matrix multiplications and it keeps all the intermediate computation values fully local and completely eliminate the DRAM access to achieve exceptionally fast single batch inference. It also reduces the data transfer overhead by carefully matching the data compute and load cycles. The simulation using the WMT15(EN-DE) dataset shows that latency of OPTIMUS is 41.62×, 24.23×, 16.01× smaller than that of Intel(R) i7 6900K CPU, NVIDIA Titan Xp GPU, and the baseline custom hardware, respectively. In addition, the throughput of OPTIMUS is 43.35×, 25.45× and 19.00× higher and the energy efficiency of OPTIMUS is 2393.85×, 1464× and 19.01× better than that of CPU, GPU and the baseline custom hardware, respectively. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | Machine Learning and Systems | - |
dc.title | OPTIMUS: OPTImized matrix MUltiplication Structure for Transformer neural network accelerator | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Choi, Jungwook | - |
dc.identifier.bibliographicCitation | Machine Learning and Systems 2020 (MLSys 2020), pp.1 - 16 | - |
dc.relation.isPartOf | Machine Learning and Systems 2020 (MLSys 2020) | - |
dc.citation.title | Machine Learning and Systems 2020 (MLSys 2020) | - |
dc.citation.startPage | 1 | - |
dc.citation.endPage | 16 | - |
dc.type.rims | ART | - |
dc.type.docType | Proceeding | - |
dc.description.journalClass | 3 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | other | - |
dc.identifier.url | https://proceedings.mlsys.org/paper/2020/file/903ce9225fca3e988c2af215d4e544d3-Paper.pdf | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.