Self-sufficient framework for continuous sign language recognition
- Authors
- Jang, Youngjoon; Oh, Youngtaek; Cho, Jae Won; Kim, Myungchul; Kim, Dong Jin; Kweon, In So; Chung, Joon Son
- Issue Date
- Jun-2023
- Publisher
- IEEE Signal Processing Society
- Citation
- IEEE International Conference on Acoustics, Speech and Signal Processing, pp.1 - 5
- Indexed
- OTHER
- Journal Title
- IEEE International Conference on Acoustics, Speech and Signal Processing
- Start Page
- 1
- End Page
- 5
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/190373
- DOI
- 10.1109/ICASSP49357.2023.10095732
- Abstract
- The goal of this work is to develop self-sufficient framework for Continuous Sign Language Recognition (CSLR) that addresses key issues of sign language recognition. These include the need for complex multi-scale features such as hands, face, and mouth for understanding, and absence of frame-level annotations. To this end, we propose (1) Divide and Focus Convolution (DFConv) which extracts both manual and non-manual features without the need for additional networks or annotations, and (2) Dense Pseudo-Label Refinement (DPLR) which propagates non-spiky frame-level pseudo-labels by combining the ground truth gloss sequence labels with the predicted sequence. We demonstrate that our model achieves state-of-the-art performance among RGB-based methods on large-scale CSLR benchmarks, PHOENIX-2014 and PHOENIX-2014-T, while showing comparable results with better efficiency when compared to other approaches that use multi-modality or extra annotations.
- Files in This Item
-
Go to Link
- Appears in
Collections - ETC > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.