Full-Face Animation for a Virtual Reality Avatar

Abstract

This research paper proposes a real-time and realistic full facial animation method for virtual reality (VR) applications. Currently, VR applications lack natural upper-face animation, which limits the immersive experiences of self-avatars. Our proposed approach combines existing lip-sync methods for the lower part of the face with a deep-learning method for the upper part. This allows us to achieve natural full-face animation with minimal latency and high computational efficiency. We demonstrate the effectiveness of our approach through experimental results and show that it is suitable for use in VR applications. Our proposed method can help to enhance the realism and immersion of self-avatars in the metaverse. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.