See, caption, cluster: Large-scale image analysis using captioning and topic modeling
- Authors
- Kang, KyeongPil; Jin, Kyohoon; Jang, Soojin; Choo, Jaegul; Kim, Youngbin
- Issue Date
- Mar-2024
- Publisher
- Elsevier Ltd
- Keywords
- Clustering; Deep learning; Image captioning; Topic modeling
- Citation
- Expert Systems with Applications, v.237
- Journal Title
- Expert Systems with Applications
- Volume
- 237
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/68331
- DOI
- 10.1016/j.eswa.2023.121391
- ISSN
- 0957-4174
1873-6793
- Abstract
- Owing to the widespread use of smartphones and mobile devices and the prevalence of image-sharing social network services, the amount of image data available on the Web is soaring. Various tasks, such as image classification, detection, and segmentation, use tremendous amounts of image data to train machine learning models. Using these trained models, a visual feature representation vector can be extracted from individual images and subsequently be used in several applications, such as image retrieval, object detection, and clustering. However, despite the increasing demand for such analyses, few studies have analyzed the information summarized by such image datasets, especially for extracting topics, trends, and opinions from images generated by online communities. Therefore, we propose a novel approach to image topic modeling, which accounts for visual content as well as semantic information by leveraging the image captioning model. In addition, we propose an image–caption scoring model that measures the semantic similarity between an image and its generated caption in order to filter noisy data that obstruct analysis by obscuring the semantic meaning of topics extracted from the dataset. The results show that our proposed method assists in analyzing large-scale image datasets without the need to manually check individual images. Further experimental results show that our methods are particularly beneficial for applications such as data visualization, image retrieval, and image tag recommendation in the realm of large-scale image dataset analysis. © 2023 Elsevier Ltd
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Graduate School of Advanced Imaging Sciences, Multimedia and Film > Department of Imaging Science and Arts > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/68331)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.