Image-Based Lifelogging: User Emotion Perspectiveopen access
- Authors
- Bum, J[Bum, Junghyun]; Choo, H[Choo, Hyunseung]; Whang, JJ[Whang, Joyce Jiyoung]
- Issue Date
- 2021
- Publisher
- TECH SCIENCE PRESS
- Keywords
- Lifelog; facial expression; emotion; emotion classifier; transfer learning
- Citation
- CMC-COMPUTERS MATERIALS & CONTINUA, v.67, no.2, pp.1963 - 1977
- Indexed
- SCIE
SCOPUS
- Journal Title
- CMC-COMPUTERS MATERIALS & CONTINUA
- Volume
- 67
- Number
- 2
- Start Page
- 1963
- End Page
- 1977
- URI
- https://scholarworks.bwise.kr/skku/handle/2021.sw.skku/17730
- DOI
- 10.32604/cmc.2021.014931
- ISSN
- 1546-2218
- Abstract
- Lifelog is a digital record of an individual's daily life. It collects, records, and archives a large amount of unstructured data; therefore, techniques are required to organize and summarize those data for easy retrieval. Lifelogging has been utilized for diverse applications including healthcare, self-tracking, and entertainment, among others. With regard to the image-based lifelogging, even though most users prefer to present photos with facial expressions that allow us to infer their emotions, there have been few studies on lifelogging techniques that focus upon users' emotions. In this paper, we develop a system that extracts users' own photos from their smartphones and configures their lifelogs with a focus on their emotions. We design an emotion classifier based on convolutional neural networks (CNN) to predict the users' emotions. To train the model, we create a new dataset by collecting facial images from the CelebFaces Attributes (CelebA) dataset and labeling their facial emotion expressions, and by integrating parts of the Radboud Faces Database (RaFD). Our dataset consists of 4,715 high-resolution images. We propose Representative Emotional Data Extraction Scheme (REDES) to select representative photos based on inferring users' emotions from their facial expressions. In addition, we develop a system that allows users to easily configure diaries for a special day and summaize their lifelogs. Our experimental results show that our method is able to effectively incorporate emotions into lifelog, allowing an enriched experience.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Computing and Informatics > Computer Science and Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/skku/handle/2021.sw.skku/17730)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.