SPATIAL POEM: A new type of experimental visual interaction in 3D virtual environment

Abstract

SPATIAL POEM is a new type of 'real-time visual interaction' expressing our own creative narrative as real-time visual by playing a musical instrument which is an emotional human behavior. There are sensors on each hole on the surface of the musical instrument. When you play it, sensors recognize that you have covered the holes. All sensors are connected to a keyboard, which means your playing behavior becomes a typing action on the keyboard. And it is programmed to spread out the visual of your words in a virtual 3D space when you play the musical instrument. The behavior when you blow the instrument, to make sounds, changes into the energy that makes you walk ahead continuously in a virtual space. It used a microphone sensor for this. After all by playing musical instrument, we get back the emotion we forgot so far, and your voice is expressed with your own visual language in virtual space.