2.5D interaction space: Elevating 2D display with an over-the-top projection mapping

Abstract

This paper introduces a new 2.5D interaction space by a touch-sensitive screen along with over-the-top projection mapping. Indeed, users can simultaneously interact with virtual objects in the touch-sensitive screen and move the virtual objects with real objects (or body fingers or hands) in the elevated 2.5D space projected by the over-the-top projector in the touch-sensitive screen. Two 2D tracking devices are used to trace the real objects. Using the tracking devices, the states (position, rotation, size) of the real objects are calculated. The realistic images are then projection-mapped to the real objects using the projector installed on the top of the touch-sensitive display. We discuss three interaction mechanisms to be employed in the 2.5D interaction space: Poking, Swaying, and Rotating. A working prototype with the interaction mechanism in the 2.5D interaction space was proposed, and a large scale usability testing is being planned. ? 2018 Association of Computing Machinery.