WEAKLY-SUPERVISED MOMENT RETRIEVAL NETWORK FOR VIDEO CORPUS MOMENT RETRIEVAL
- Authors
- Yoon, Sunjae; Kim, Dahyun; Hong, Ji Woo; Kim, Junyeong; Kim, Kookhoi; Yoo, Chang D.
- Issue Date
- Sep-2021
- Publisher
- IEEE
- Keywords
- Multi-modal video corpus moment retrieval; Weakly-supervised learning
- Citation
- 2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), pp 534 - 538
- Pages
- 5
- Journal Title
- 2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
- Start Page
- 534
- End Page
- 538
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/62184
- DOI
- 10.1109/ICIP42928.2021.9506218
- ISSN
- 1522-4880
- Abstract
- This paper proposes Weakly-supervised Moment Retrieval Network (WMRN) for Video Corpus Moment Retrieval (VCMR), which retrieves pertinent temporal moments related to natural language query in a large video corpus. Previous methods for VCMR require full supervision of temporal boundary information for training, which involves a labor-intensive process of annotating the boundaries in a large number of videos. To leverage this, the proposed WMRN performs VCMR in a weakly-supervised manner, where WMRN is learned without ground-truth labels but only with video and text queries. For weakly-supervised VCMR, WMRN addresses the following two limitations of prior methods: (1) Blurry attention over video features due to redundant video candidate proposals generation, (2) Insufficient learning due to weak supervision only with video-query pairs. To this end, WMRN is based on (1) Text Guided Proposal Generation (TGPG) that effectively generates text guided multi-scale video proposals in the prospective region related to query, and (2) Hard Negative Proposal Sampling (HNPS) that enhances video-language alignment via extracting negative video proposals in positive video sample for contrastive learning. Experimental results show that WMRN achieves state-of-the-art performance on TVR and DiDeMo benchmarks in the weakly-supervised setting. To validate the attainments of proposed components of WMRN, comprehensive ablation studies and qualitative analysis are conducted.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > Department of Artificial Intelligence > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/62184)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.