Towards Natural and Intuitive Human-Robot Collaboration based on Goal-Oriented Human Gaze Intention Recognition

Abstract

The objective of this paper is to introduce a new method for predicting human gaze intention using a head-mounted display, with the aim of enabling natural and intuitive collaboration between humans and robots. Human eye gaze is strongly linked to cognitive processes and can facilitate communication between humans and robots. However, accurately identifying the goal-directed object through human intention remains challenging. This study focuses on developing a method to differentiate between goal and non-goal gaze by creating an area of interest (AOI) on each object through the goal-directed gaze. The Microsoft HoloLens 2 was used to simulate the robot using real-time gaze data in augmented reality (AR). The methods with and without AOI were compared through pick-and-place robot manipulation through human gaze prediction. The AOI method resulted a maximum improvement of 19% in the F1 score compared to the baseline method. The results yield strong evidence on intuitiveness and usefulness that the use of pre-defined AOI allows improved performance to predict gaze intention that has the potential to be applied in various fields, where human-robot collaboration can enhance efficiency and productivity.