A Comparative Analysis Between Real Human and Virtual Human Interactions in an Academic Learning Context Using Emotion Recognition

Abstract

In today's academic scenario, understanding learners' emotional responses during academic learning is important to improve learning ability. This study provides a comparative analysis of facial emotions from interactions with both real human (RH) and virtual human (VH) in the context of online academic learning. Facial video data were collected from participants engaged in both RH and VH learning sessions. Facial landmarks were extracted using the MediaPipe Face Mesh model and six emotional states were mapped from computed action unit (AU) scores. A convolutional neural network (CNN) was trained on the FER-2013 and extended CK+ datasets to classify six facial emotional states from the acquired dataset. Emotion intensity was computed based on AU scores for each detected state. Results revealed that happiness and surprise intensities were significantly higher during VH interactions compared to RH. An ANOVA test confirmed statistically significant differences in emotional intensity between RH and VH interactions.