Step by Step: A Gradual Approach for Dense Video Captioning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Choi, Wangyu | - |
dc.contributor.author | Chen, Jiasi | - |
dc.contributor.author | Yoon, Jongwon | - |
dc.date.accessioned | 2023-07-05T05:35:04Z | - |
dc.date.available | 2023-07-05T05:35:04Z | - |
dc.date.issued | 2023-05 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/112992 | - |
dc.description.abstract | Dense video captioning aims to localize and describe events for storytelling in untrimmed videos. It is a conceptually very challenging task that requires concise, relevant, and coherent captioning based on high-quality event localization. Unlike simple temporal action localization tasks without overlapping events, dense video captioning requires detecting multiple/overlapping regions in order to branch out the video story. Most existing methods generate numerous candidate event proposals and then eliminate duplicate ones using a event proposal selection algorithm (e.g., non-maximum suppression) or generate event proposals directly through box prediction and binary classification mechanisms, similar to object detection tasks. Despite these efforts, the aforementioned approaches tend to fail to localize overlapping events into different stories, hindering high-quality captioning. In this paper, we propose SBS, a dense video captioning framework with a gradual approach that addresses the challenge of localizing overlapping events and eventually constructs high-quality captioning. SBS accurately estimates the number of explicit events for each video snippet and then detects the boundaries context/activities, which are the details for generating the event proposals. Based on both the number of events and boundaries, SBS generates the event proposals. SBS encodes the context of the event sequence and finally generates sentences describing the event proposals. Our framework is fairly effective in localizing multiple/overlapping events, thus experimental results show the state-of-the-art performance compared to the existing methods. | - |
dc.format.extent | 11 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | - |
dc.title | Step by Step: A Gradual Approach for Dense Video Captioning | - |
dc.type | Article | - |
dc.publisher.location | 미국 | - |
dc.identifier.doi | 10.1109/ACCESS.2023.3279816 | - |
dc.identifier.scopusid | 2-s2.0-85161050168 | - |
dc.identifier.wosid | 001006225900001 | - |
dc.identifier.bibliographicCitation | IEEE Access, v.11, pp 51949 - 51959 | - |
dc.citation.title | IEEE Access | - |
dc.citation.volume | 11 | - |
dc.citation.startPage | 51949 | - |
dc.citation.endPage | 51959 | - |
dc.type.docType | Article | - |
dc.description.isOpenAccess | Y | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.subject.keywordAuthor | Dense video captioning | - |
dc.subject.keywordAuthor | event captioning | - |
dc.subject.keywordAuthor | event localization | - |
dc.subject.keywordAuthor | event proposal generation | - |
dc.subject.keywordAuthor | video captioning | - |
dc.identifier.url | https://ieeexplore.ieee.org/document/10132459 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
55 Hanyangdeahak-ro, Sangnok-gu, Ansan, Gyeonggi-do, 15588, Korea+82-31-400-4269 sweetbrain@hanyang.ac.kr
COPYRIGHT © 2021 HANYANG UNIVERSITY. ALL RIGHTS RESERVED.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.