An interface between users and virtual objects using two hands

Abstract

This paper presents a novel hand-based interface that provides natural interactions with virtual objects augmented on a user's palm. In the interface, virtual objects are easily selected by drawing predefined patterns on a palm of one hand with a finger of the other hand and recognizing the patterns. The selected virtual objects are augmented on the palm by estimating palm poses. To efficiently detect the patterns drawn with the finger, its trajectories are tracked using a wearable type of a visual marker. By estimating a wide range of palm poses and recognizing finger gestures, a user can naturally interact with the virtual objects augmented on the palm at various viewpoints. The feasibility of the interface is verified through our experimental results.