Dense but Efficient VideoQA for Intricate Compositional Reasoning
- Authors
- Lee, Jihyeon; Kang, Wooyoung; Kim, Eun Sol
- Issue Date
- Jan-2023
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- Algorithms: Video recognition and understanding (tracking, action recognition, etc.); Vision + language and/or other modalities
- Citation
- Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023, pp.1114 - 1123
- Indexed
- SCOPUS
- Journal Title
- Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
- Start Page
- 1114
- End Page
- 1123
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/182539
- DOI
- 10.1109/WACV56688.2023.00117
- ISSN
- 0000-0000
- Abstract
- It is well known that most of the conventional video question answering (VideoQA) datasets consist of easy questions requiring simple reasoning processes. However, long videos inevitably contain complex and compositional semantic structures along with the spatio-temporal axis, which requires a model to understand the compositional structures inherent in the videos. In this paper, we suggest a new compositional VideoQA method based on transformer architecture with a deformable attention mechanism to address the complex VideoQA tasks. The deformable attentions are introduced to sample a subset of informative visual features from the dense visual feature map to cover a temporally long range of frames efficiently. Furthermore, the dependency structure within the complex question sentences is also combined with the language embeddings to readily understand the relations among question words. Extensive experiments and ablation studies show that the suggested dense but efficient model outperforms other baselines.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/182539)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.