Multimodal Neural Machine Translation With Weakly Labeled Images
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Heo, Yoonseok | - |
dc.contributor.author | Kang, Sangwoo | - |
dc.contributor.author | Yoo, Donghyun | - |
dc.date.available | 2020-02-27T07:42:55Z | - |
dc.date.created | 2020-02-05 | - |
dc.date.issued | 2019 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/2873 | - |
dc.description.abstract | Machine translation refers to a fully automated process that translates a user's input text into a target language. To improve the accuracy of machine translation, studies usually exploit not only the input text itself but also various background knowledge related to the text, such as visual information or prior knowledge. Herein, in this paper, we propose a multimodal neural machine translation system that uses both texts and their related images to translate Korean image captions into English. The data in the experiment is a set of unlabeled images only containing bilingual captions. To train the system with a supervised learning approach, we propose a weak-labeling method that selects a keyword from an image caption using feature selection methods. The keywords are used to roughly determine an image label. We also introduce an improved feature selection method using sentence clustering to select keywords that reflect the characteristics of the image captions more accurately. We found that our multimodal system achieves an improved performance compared to a text-only neural machine translation system (baseline). Furthermore, the additional images have positive impacts on addressing the issue of under-translation, where some words in a source sentence are falsely translated or not translated at all. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.relation.isPartOf | IEEE ACCESS | - |
dc.subject | MODEL | - |
dc.title | Multimodal Neural Machine Translation With Weakly Labeled Images | - |
dc.type | Article | - |
dc.type.rims | ART | - |
dc.description.journalClass | 1 | - |
dc.identifier.wosid | 000467272500001 | - |
dc.identifier.doi | 10.1109/ACCESS.2019.2911656 | - |
dc.identifier.bibliographicCitation | IEEE ACCESS, v.7, pp.54042 - 54053 | - |
dc.identifier.scopusid | 2-s2.0-85065386510 | - |
dc.citation.endPage | 54053 | - |
dc.citation.startPage | 54042 | - |
dc.citation.title | IEEE ACCESS | - |
dc.citation.volume | 7 | - |
dc.contributor.affiliatedAuthor | Kang, Sangwoo | - |
dc.type.docType | Article | - |
dc.subject.keywordAuthor | Human-computer interaction | - |
dc.subject.keywordAuthor | multi-layer neural network | - |
dc.subject.keywordAuthor | natural language processing | - |
dc.subject.keywordAuthor | image classification | - |
dc.subject.keywordAuthor | multimodal neural machine translation | - |
dc.subject.keywordAuthor | weak label | - |
dc.subject.keywordPlus | MODEL | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
1342, Seongnam-daero, Sujeong-gu, Seongnam-si, Gyeonggi-do, Republic of Korea(13120)031-750-5114
COPYRIGHT 2020 Gachon University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.