Bodily Sensation Map vs. Bodily Motion Map: Visualizing and Analyzing Emotional Body Motions

Abstract

Emotion detection using features presented in the body has been comparatively understudied compared to other emotional modalities. This study investigated and compared how emotions are revealed through bodily sensations and body movement information. We propose a novel visualization method for addressing body part activation or deactivation associated with different emotions using motion capture data (bodily motion maps; BMMs) and aim to compare its emotional features with existing methods of addressing sensation towards body activation or deactivation using the computerised self-report method (bodily sensation maps; BSMs). A user study is conducted with twenty-nine participants to gather BSMs and BMMs under seven emotional statuses (happy, sad, surprise, angry, disgust, fearful, and neutral). The results indicate that people's BMMs as well as their BSMs contain shared features that can discriminate between emotions. The asynchronies between bodily sensation and body motion existed. Furthermore, convolutional neural network (CNN)-based automatic emotion recognition outperforms humans’ emotion recognition abilities. The implications of the current study's method of comparing emotional features between bodily sensation and body motion for future emotion-related studies for mental disorders are discussed.