Emotion and Body Movement: A Comparative Study of Automatic Emotion Recognition Using Body Motions

Abstract

Emotion recognition through body movement in both real and virtual worlds is an important research topic along with facial expression and voice recognition. Computational methods to recognize emotions based on body movement have been developed to utilize skeletal data and motion capture systems, and 2D and 3D pose estimation methods have recently been proposed. Although each of these methodologies involves advantages and disadvantages, they have not been compared with same data. In this study, we collected seven types of motion data associated with specified emotional states from 25 participants, including happiness, sadness, anger, disgust, fear, surprise, and a neutral emotion. We compared three methodologies, including motion capture, 2D pose estimation, and 3D pose estimation, along with human evaluations as a baseline. The results show that measurement through motion capture showed the highest performance, and the 2D and 3D pose estimation also showed relatively high performance compared to the human evaluators' results. These findings suggest that the existing methodologies can be utilized to perform emotion recognition.