DEVELOPMENT OF AN INTEGRATED SMART SENSOR SYSTEM FOR SOUND SYNTHESIS AND REPRODUCTION IN TELEPRESENCE

Abstract

This paper presents the development of an integrated sensor system for the telepresence system with immersive audio. The system consists of various types of sensor units and a multimedia server, which gathers, processes, and displays the data obtained from the each sensors. Three dimensional microphone arrays, IMU (Inertial Measurement Unit), depth cameras, and color camera sensors are integrated together. The paper contains a functional and a technical description of the telepresence system. It also describes the algorithm for detection of a moving sound source based on DOA (Direction of Arrival) estimation. The development of telepresence system for immersive audio with an integrated sensor technology could be beneficial in online meeting and telecommuting.