Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Heo, Yu-Jung | - |
dc.contributor.author | Kim, Eun-Sol | - |
dc.contributor.author | Choi, Woo Suk | - |
dc.contributor.author | Zhang, Byoung-Tak | - |
dc.date.accessioned | 2022-10-25T07:40:35Z | - |
dc.date.available | 2022-10-25T07:40:35Z | - |
dc.date.created | 2022-10-06 | - |
dc.date.issued | 2022-09 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/172562 | - |
dc.description.abstract | Knowledge-based visual question answering (QA) aims to answer a question which requires visually-grounded external knowledge beyond image content itself. Answering complex questions that require multi-hop reasoning under weak supervision is considered as a challenging problem since i) no supervision is given to the reasoning process and ii) high-order semantics of multi-hop knowledge facts need to be captured. In this paper, we introduce a concept of hypergraph to encode high-level semantics of a question and a knowledge base, and to learn high-order associations between them. The proposed model, Hypergraph Transformer, constructs a question hypergraph and a query-aware knowledge hypergraph, and infers an answer by encoding inter-associations between two hypergraphs and intra-associations in both hypergraph itself. Extensive experiments on two knowledge-based visual QA and two knowledge-based textual QA demonstrate the effectiveness of our method, especially for multi-hop reasoning problem. Our source code is available at https://github.com/yujungheo/kbvqa-public. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | ASSOC COMPUTATIONAL LINGUISTICS-ACL | - |
dc.title | Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Kim, Eun-Sol | - |
dc.identifier.wosid | 000828702300029 | - |
dc.identifier.bibliographicCitation | PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1: (LONG PAPERS), pp.373 - 390 | - |
dc.relation.isPartOf | PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1: (LONG PAPERS) | - |
dc.citation.title | PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1: (LONG PAPERS) | - |
dc.citation.startPage | 373 | - |
dc.citation.endPage | 390 | - |
dc.type.rims | ART | - |
dc.type.docType | Proceedings Paper | - |
dc.description.journalClass | 3 | - |
dc.description.isOpenAccess | N | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Linguistics | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Interdisciplinary Applications | - |
dc.relation.journalWebOfScienceCategory | Linguistics | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.