Gaining Extra Supervision via Multi-task learning for Multi-Modal Video Question Answering
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kim, Junyeong | - |
dc.contributor.author | Ma, Minuk | - |
dc.contributor.author | Kim, Kyungsu | - |
dc.contributor.author | Kim, Sungjin | - |
dc.contributor.author | Yoo, Chang D. | - |
dc.date.accessioned | 2023-03-08T15:00:03Z | - |
dc.date.available | 2023-03-08T15:00:03Z | - |
dc.date.issued | 2019-07 | - |
dc.identifier.issn | 2161-4393 | - |
dc.identifier.issn | 2161-4407 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/63728 | - |
dc.description.abstract | This paper proposes a method to gain extra supervision via multi-task learning for multi-modal video question answering. Multi-modal video question answering is an important task that aims at the joint understanding of vision and language. However, establishing large scale dataset for multi-modal video question answering is expensive and the existing benchmarks are relatively small to provide sufficient supervision. To overcome this challenge, this paper proposes a multi-task learning method which is composed of three main components: (1) multi-modal video question answering network that answers the question based on the both video and subtitle feature, (2) temporal retrieval network that predicts the time in the video clip where the question was generated from and (3) modality alignment network that solves metric learning problem to find correct association of video and subtitle modalities. By simultaneously solving related auxiliary tasks with hierarchically shared intermediate layers, the extra synergistic supervisions are provided. Motivated by curriculum learning, multi-task ratio scheduling is proposed to learn easier task earlier to set inductive bias at the beginning of the training. The experiments on publicly available dataset TVQA shows state-of-the-art results, and ablation studies are conducted to prove the statistical validity. | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | IEEE | - |
dc.title | Gaining Extra Supervision via Multi-task learning for Multi-Modal Video Question Answering | - |
dc.type | Article | - |
dc.identifier.doi | 10.48550/arXiv.1905.13540 | - |
dc.identifier.bibliographicCitation | 2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | - |
dc.description.isOpenAccess | N | - |
dc.identifier.wosid | 000530893803019 | - |
dc.identifier.scopusid | 2-s2.0-85073258430 | - |
dc.citation.title | 2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | - |
dc.identifier.url | http://dx.doi.org/10.1109/IJCNN.2019.8852087 | - |
dc.type.docType | Proceedings Paper | - |
dc.publisher.location | 미국 | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Hardware & Architecture | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.description.journalRegisteredClass | foreign | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.