Point of Interest Detection and Visual Distance Estimation for Sensor-Rich Video

Abstract

Due to technological advances and the popularity of camera sensors, it is now straightforward for users to capture and share videos. A large number of geo-tagged photos and videos have been accumulating continuously on the web, posing a challenging problem for mining this type of media data. In one application scenario, users might desire to know what the Points of Interest (POI) are which contain important objects or places in a video. Existing solutions attempt to examine the content of the videos and recognize objects and events. This is typically time-consuming and computationally expensive and the results can be uneven. Therefore these methods face challenges when applied to large video repositories. We propose a novel technique that leverages sensor-generated meta-data (camera locations and viewing directions) which are automatically acquired as continuous streams together with the video frames. Existing smartphones can easily accommodate such integrated recording tasks. By considering a collective set of videos and leveraging the acquired auxiliary meta-data, our approach is able to detect interesting regions and objects (POIs) and their distances from the camera positions in a fully automated way. Because of its computational efficiency, the proposed method scales well and our experiments show very promising results.