W2V2-Light: A Lightweight Version of Wav2vec 2.0 for Automatic Speech Recognition
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kim, Dong-Hyun | - |
dc.contributor.author | Lee, Jae-Hong | - |
dc.contributor.author | Mo, Ji-Hwan | - |
dc.contributor.author | Chang, Joon-Hyuk | - |
dc.date.accessioned | 2022-12-20T06:24:54Z | - |
dc.date.available | 2022-12-20T06:24:54Z | - |
dc.date.created | 2022-11-02 | - |
dc.date.issued | 2022-09 | - |
dc.identifier.issn | 2308-457X | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/173086 | - |
dc.description.abstract | Wav2vec 2.0 (W2V2) has shown remarkable speech recognition performance by pre-training only with unlabeled data and fine-tuning with a small amount of labeled data. However, the practical application of W2V2 is hindered by hardware memory limitations, as it contains 317 million parameters. To address this issue, we propose W2V2-Light, a lightweight version of W2V2. We introduce two simple sharing methods to reduce the memory consumption as well as the computational costs of W2V2. Compared to W2V2, our model has 91% lesser parameters and a speedup of 1.31 times with minor degradation in downstream task performance. Moreover, by quantifying the stability of representations, we provide an empirical insight into why our model is capable of maintaining competitive performance despite the significant reduction in memory. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | International Speech Communication Association | - |
dc.title | W2V2-Light: A Lightweight Version of Wav2vec 2.0 for Automatic Speech Recognition | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Chang, Joon-Hyuk | - |
dc.identifier.doi | 10.21437/Interspeech.2022-10339 | - |
dc.identifier.scopusid | 2-s2.0-85140087147 | - |
dc.identifier.wosid | 000900724503042 | - |
dc.identifier.bibliographicCitation | Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, v.2022-September, pp.3038 - 3042 | - |
dc.relation.isPartOf | Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH | - |
dc.citation.title | Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH | - |
dc.citation.volume | 2022-September | - |
dc.citation.startPage | 3038 | - |
dc.citation.endPage | 3042 | - |
dc.type.rims | ART | - |
dc.type.docType | Proceedings Paper | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Acoustics | - |
dc.relation.journalResearchArea | Audiology & Speech-Language Pathology | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalWebOfScienceCategory | Acoustics | - |
dc.relation.journalWebOfScienceCategory | Audiology & Speech-Language Pathology | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.subject.keywordPlus | Speech recognition | - |
dc.subject.keywordPlus | Supervised learning | - |
dc.subject.keywordPlus | Speech communication | - |
dc.subject.keywordPlus | Attention alignment | - |
dc.subject.keywordPlus | Automatic speech recognition | - |
dc.subject.keywordPlus | Fine tuning | - |
dc.subject.keywordPlus | Labeled data | - |
dc.subject.keywordPlus | Parameter sharing | - |
dc.subject.keywordPlus | Pre-training | - |
dc.subject.keywordPlus | Representation learning | - |
dc.subject.keywordPlus | Semi-supervised learning | - |
dc.subject.keywordPlus | Speech recognition performance | - |
dc.subject.keywordPlus | Unlabeled data | - |
dc.subject.keywordAuthor | attention alignment | - |
dc.subject.keywordAuthor | Automatic speech recognition | - |
dc.subject.keywordAuthor | parameter sharing | - |
dc.subject.keywordAuthor | representation learning | - |
dc.subject.keywordAuthor | semi-supervised learning | - |
dc.identifier.url | https://www.isca-speech.org/archive/interspeech_2022/kim22l_interspeech.html | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.