Robust hand detection for augmented reality interface

Abstract

For interactive augmented reality, vision-based and hand-gesture-based interface are most desirable due to being natural and human-friendly. However, detecting hands and recognizing hand gestures in cluttered background are still challenging. Especially, if the background includes a large skin-colored region, the problem becomes more difficult. In this paper, we focus on detecting a hand reliably and propose an effective method. Our method is basically based on the assumption that a hand-forearm region (including a hand and part of a forearm) has different brightness from other skin-colored regions. Specifically, we first segment the hand-forearm region from other skin-colored regions based on the brightness difference which is represented by edges in this paper. Then, we extract the hand region from the hand-forearm region by detecting a feature point which indicates the wrist. Finally, we extract the hand by using the brightness-based segmentation which is slightly different from the hand-forearm region detection. We verify the effectiveness of our method by implementing a simple hand gesture interface based on our method and applying it to augmented reality applications.