BaSSL: Boundary-aware Self-Supervised Learning for Video Scene Segmentation
- Authors
- Mun, Jonghwan; Shin, Minchul; Han, Gunsoo; Lee, Sangho; Ha, Seongsu; Lee, Joonseok; Kim, Eun-Sol
- Issue Date
- Mar-2023
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Keywords
- Self-supervised learning; Video scene segmentation
- Citation
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), v.13844 LNCS, pp.485 - 501
- Indexed
- SCOPUS
- Journal Title
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- Volume
- 13844 LNCS
- Start Page
- 485
- End Page
- 501
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/186008
- DOI
- 10.1007/978-3-031-26316-3_29
- ISSN
- 0302-9743
- Abstract
- Self-supervised learning has drawn attention through its effectiveness in learning in-domain representations with no ground-truth annotations; in particular, it is shown that properly designed pretext tasks bring significant performance gains for downstream tasks. Inspired from this, we tackle video scene segmentation, which is a task of temporally localizing scene boundaries in a long video, with a self-supervised learning framework where we mainly focus on designing effective pretext tasks. In our framework, given a long video, we adopt a sliding window scheme; from a sequence of shots in each window, we discover a moment with a maximum semantic transition and leverage it as pseudo-boundary to facilitate the pre-training. Specifically, we introduce three novel boundary-aware pretext tasks: 1) Shot-Scene Matching (SSM), 2) Contextual Group Matching (CGM) and 3) Pseudo-boundary Prediction (PP); SSM and CGM guide the model to maximize intra-scene similarity and inter-scene discrimination by capturing contextual relation between shots while PP encourages the model to identify transitional moments. We perform an extensive analysis to validate effectiveness of our method and achieve the new state-of-the-art on the MovieNet-SSeg benchmark. The code is available at https://github.com/kakaobrain/bassl.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/186008)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.