One-handed interaction with augmented virtual objects on mobile devices

Abstract

We present a one-handed approach for augmented reality and interaction on mobile devices. The proposed application considers common situations with mobile devices such as when a user's hand holds a mobile device while the other hand is free. It also supports natural augmented reality environment such as when a user interacts with augmented reality contents anytime and anywhere without special equipment such as visual markers or tags. In our approach, a virtual object is augmented on the palm of a user's free hand, as if the virtual object is just sitting on the palm, using a palm pose estimation method. The augmented virtual object reacts (e.g. moving or animation) to motions of the hand such as opening or closing the hand based on fingertip tracking. Moreover, it provides tactile interactions with the virtual object by wearing a tactile glove with vibration sensors. This paper describes how to implement the augmented reality application, and preliminary results show its potential as a new approach to mobile augmented reality interaction.