MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection
- Authors
- Kim, Bumsoo; Mun, Jonghwan; On, Kyoung-Woon; Shin, Minchul; Lee, Junhyun; Kim, Eun Sol
- Issue Date
- Jun-2022
- Publisher
- IEEE Computer Society
- Keywords
- Scene analysis and understanding
- Citation
- Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, v.2022-June, pp.19556 - 19565
- Indexed
- SCOPUS
- Journal Title
- Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
- Volume
- 2022-June
- Start Page
- 19556
- End Page
- 19565
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/173242
- DOI
- 10.1109/CVPR52688.2022.01897
- ISSN
- 1063-6919
- Abstract
- Human-Object Interaction (HOI) detection is the task of identifying a set of (human, object, interaction) triplets from an image. Recent work proposed transformer encoder-decoder architectures that successfully eliminated the need for many hand-designed components in HOI detection through end-to-end training. However, they are limited to single-scale feature resolution, providing suboptimal performance in scenes containing humans, objects, and their interactions with vastly different scales and distances. To tackle this problem, we propose a Multi-Scale TRansformer (MSTR) for HOI detection powered by two novel HOI-aware deformable attention modules called Dual-Entity attention and Entity-conditioned Context attention. While existing deformable attention comes at a huge cost in HOI detection performance, our proposed attention modules of MSTR learn to effectively attend to sampling points that are essential to identify interactions. In experiments, we achieve the new state-of-the-art performance on two HOI detection benchmarks.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/173242)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.