Optimizing Data Collection for Bodily Emotion Recognition: A Comparative Study

Abstract

Although there are various studies on automatic-emotion-recognition (AER), the bodily AER is scarce compared to other emotion modalities owing to limitations of research methodology. Herein, we suggest methodologies for collecting large emotional body movement data under three factors for bodily AER dataset construction: participant expertise, motion capture devices, and emotional stimuli, and compared classification accuracy using machine learning and deep learning such as convolutional neural networks, graph convolutional networks, long short-term memory and Transformer. The first study suggests that the models trained using the non-actor dataset performed better than the other model. The second study suggests that the models trained using both marker-based-MoCap and pose-estimation performed better than Kinect-MoCap and mobile-MoCap. The third study suggests that training with both word and video stimuli performed better than picture stimuli. Considering the emotion classification accuracy and accessibility, we recommend gathering bodily AER dataset using non-actors, pose-estimation, and using either word or video stimuli. The current findings may contribute to future research methodologies for bodily emotion recognition.