A Preliminary Study on Performance Evaluation of Multi-View Multi-Modal Gaze Estimation under Challenging Conditions

Abstract

In this paper, we address gaze estimation under practical and challenging conditions. Multi-view and multi-modal learning have been considered useful for various complex tasks; however, an in-depth analysis or a large-scale dataset on multi-view, multi-modal gaze estimation under a long-distance setup with a low illumination is still very limited. To address these limitations, first, we construct a dataset of images captured under challenging conditions. And we propose a simple deep learning architecture that can handle multi-view multi-modal data for gaze estimation. Finally, we conduct a performance evaluation of the proposed network with the constructed dataset to understand the effects of multiple views of a user and multi-modality (RGB, depth, and infrared). We report various findings from our preliminary experimental results and expect this would be helpful for gaze estimation studies to deal with challenging conditions.