Deep-learning-based real-time silent speech recognition using facial electromyogram recorded around eyes for hands-free interfacing in a virtual reality environment
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Cha, Ho-Seung | - |
dc.contributor.author | Chang, Won-Du | - |
dc.contributor.author | Im, Chang-Hwan | - |
dc.date.accessioned | 2023-06-01T06:59:06Z | - |
dc.date.available | 2023-06-01T06:59:06Z | - |
dc.date.created | 2022-03-07 | - |
dc.date.issued | 2022-09 | - |
dc.identifier.issn | 1359-4338 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/185827 | - |
dc.description.abstract | Speech recognition technology is a promising hands-free interfacing modality for virtual reality (VR) applications. However, it has several drawbacks, such as limited usability in a noisy environment or a public place and limited accessibility to those who cannot generate loud and clear voices. These limitations may be overcome by employing a silent speech recognition (SSR) technology utilizing facial electromyograms (fEMGs) in a VR environment. In the conventional SSR systems, however, fEMG electrodes were attached around the user's lips and neck, thereby creating new practical issues, such as the requirement of an additional wearable system besides the VR headset, necessity of a complex and time-consuming procedure for attaching the fEMG electrodes, and discomfort and limited facial muscle movements of the user. To solve these problems, we propose an SSR system using fEMGs measured by a few electrodes attached around the eyes of a user, which can also be easily incorporated into available VR headsets. To enhance the accuracy of classifying the fEMG signals recorded from limited recording locations relatively far from the phonatory organs, a deep neural network-based classification method was developed using similar fEMG data previously collected from other individuals and then transformed by dynamic positional warping. In the experiments, the proposed SSR system could classify six different fEMG patterns generated by six silently spoken words with an accuracy of 92.53%. To further demonstrate that our SSR system can be used as a hands-free control interface in practical VR applications, an online SSR system was implemented. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | SPRINGER LONDON LTD | - |
dc.title | Deep-learning-based real-time silent speech recognition using facial electromyogram recorded around eyes for hands-free interfacing in a virtual reality environment | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Im, Chang-Hwan | - |
dc.identifier.doi | 10.1007/s10055-021-00616-0 | - |
dc.identifier.scopusid | 2-s2.0-85123869037 | - |
dc.identifier.wosid | 000749126800001 | - |
dc.identifier.bibliographicCitation | Virtual Reality, v.26, no.3, pp.1047 - 1057 | - |
dc.relation.isPartOf | Virtual Reality | - |
dc.citation.title | Virtual Reality | - |
dc.citation.volume | 26 | - |
dc.citation.number | 3 | - |
dc.citation.startPage | 1047 | - |
dc.citation.endPage | 1057 | - |
dc.type.rims | ART | - |
dc.type.docType | Article; Early Access | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Imaging Science & Photographic Technology | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Interdisciplinary Applications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Software Engineering | - |
dc.relation.journalWebOfScienceCategory | Imaging Science & Photographic Technology | - |
dc.subject.keywordPlus | Deep neural networks | - |
dc.subject.keywordPlus | E-learning | - |
dc.subject.keywordPlus | Eye movements | - |
dc.subject.keywordPlus | Face recognition | - |
dc.subject.keywordPlus | Speech | - |
dc.subject.keywordPlus | Speech recognition | - |
dc.subject.keywordPlus | Virtual reality | - |
dc.subject.keywordPlus | Deep learning | - |
dc.subject.keywordPlus | Electromyo grams | - |
dc.subject.keywordPlus | Facial electromyographies | - |
dc.subject.keywordPlus | Hands-free | - |
dc.subject.keywordPlus | Human computer interfaces | - |
dc.subject.keywordPlus | Myoelectric control | - |
dc.subject.keywordPlus | Silent speech | - |
dc.subject.keywordPlus | Silent speech recognition | - |
dc.subject.keywordPlus | Speech recognition systems | - |
dc.subject.keywordPlus | Virtual-reality environment | - |
dc.subject.keywordPlus | Electrodes | - |
dc.subject.keywordAuthor | Deep learning | - |
dc.subject.keywordAuthor | Facial electromyography | - |
dc.subject.keywordAuthor | Human-computer interface | - |
dc.subject.keywordAuthor | Myoelectric control | - |
dc.subject.keywordAuthor | Silent speech recognition | - |
dc.subject.keywordAuthor | Virtual reality | - |
dc.identifier.url | https://link.springer.com/article/10.1007/s10055-021-00616-0 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.