MAC: multimodal, attention-based cybersickness prediction modeling in virtual reality

Abstract

Cybersickness is one of the greatest barriers to the adoption of virtual reality. A growing body of research has focused on identifying the characteristics of cybersickness and finding ways to mitigate it through the utilization of data from VR content, physiological signals, and body movement, along with artificial intelligence techniques. In this work, we extend prior research on cybersickness prediction by considering the role of different data modalities. We propose a novel deep learning model named multimodal, attention-based cybersickness (MAC), which learns temporal sequences and characteristics of video flows, eye movement, head movement, and electrodermal activity. Based on data collected from 27 participants, we demonstrate the effectiveness of MAC, showing an F1-score of 0.87. Our experimental results further show not only the influences of gender and prior VR experience but also the effectiveness of the attention mechanism on model performance, emphasizing the importance of considering the characteristics of data types and users in cybersickness modeling.