Action-Concentrated Embedding Framework: This is your captain sign-tokening
- Authors
- Yu, Hyunwook; Shin, Suhyeon; Heo, Jungu; Shin, Hyuntaek; Kim, Hyosu; Kim, Mucheol
- Issue Date
- 2024
- Publisher
- European Language Resources Association (ELRA)
- Keywords
- short-time Fourier transform; sign language token embedding framework; sign language translation
- Citation
- 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, pp 310 - 320
- Pages
- 11
- Journal Title
- 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings
- Start Page
- 310
- End Page
- 320
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/74528
- Abstract
- Sign language is the primary communication medium for people who are deaf or have hearing loss. However, given the divergent range of sensory abilities of these individuals, there is a communication gap that needs to be addressed. In this paper, we present action-concentrated embedding (ACE), which is a novel sign token embedding framework. Additionally, to provide a more structured foundation for sign language analysis, we introduce a dedicated notation system tailored for sign language that endeavors to encapsulate the nuanced gestures and movements that are integral with sign communication. The proposed ACE approach tracks a signer's actions based on human posture estimation. Tokenizing these actions and capturing the token embedding using a short-time Fourier transform encapsulates the time-based behavioral changes. Hence, ACE offers input embedding to translate sign language into natural language sentences. When tested against a disaster sign language dataset using automated machine translation measures, ACE notably surpasses prior research in terms of translation capabilities, improving the performance by up to 5.79% for BLEU-4 and 5.46% for ROUGE-L metric. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/74528)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.