Automatically Detecting Image-Text Mismatch on Instagram with Deep Learning
- Authors
- Ha, Yui; Park, Kunwoo; Kim, Su Jung; Joo, Jungseock; Cha, Meeyoung
- Issue Date
- Nov-2020
- Publisher
- ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD
- Citation
- JOURNAL OF ADVERTISING, v.50, no.1, pp.52 - 62
- Journal Title
- JOURNAL OF ADVERTISING
- Volume
- 50
- Number
- 1
- Start Page
- 52
- End Page
- 62
- URI
- http://scholarworks.bwise.kr/ssu/handle/2018.sw.ssu/40300
- DOI
- 10.1080/00913367.2020.1843091
- ISSN
- 0091-3367
- Abstract
- Visual social media have emerged as an essential brand communication channel for advertisers and brands. The active use of hashtags has enabled advertisers to identify customers interested in their brands and better understand their consumers. However, some users post brand-incongruent content-for example, posts composed of brand-irrelevant images with brand-relevant hashtags. Such visual information mismatch can be problematic because it hinders other consumers' information search processes and advertisers' insight generation from consumer-initiated social media data. This study aims to characterize visually mismatched content in brand-related posts on Instagram and builds a visual information mismatch detection model using computer vision. We propose a machine-learning model based on three cues: image, text, and metadata. Our analysis shows the effectiveness of deep learning and the importance of combining text and image features for mismatch detection. We discuss the advantages of machine-learning methods as a novel research tool for advertising research and conclude with implications of our findings.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Information Technology > ETC > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/ssu/handle/2018.sw.ssu/40300)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.