조감도에서의 객체 움직임 및 환경 변화를 반영한 시공간 특징 결합 기반의 라이다 비디오 3차원 객체 검출

Abstract

This paper proposes an online 3D video object detection model based on deep learning using a sequential LiDAR point set. By utilizing sequential data as input, our proposed method aims to generate Spatio-temporal information and overcome the inherent limitations of point cloud data, such as sparsity and irregular acquisition due to distance and occlusion. The proposed method, called ST-FF, performs Spatio-temporal feature fusion between a bird"s eye view (BEV) feature maps obtained from each LiDAR point set. ST-FF first captures the movement of objects and changes in the surrounding environment over time. Based on the captured motion information, information suitable for the current feature map is selectively extracted from the past feature map. Then the final feature map is obtained by aggregating the feature map at the target frame and the extracted feature maps from the past frame. Finally, the detection head generates 3D bounding boxes for the target frame using the final feature map. Experiments were conducted on the nuScenes dataset to validate the contributions of the proposed method. Higher performance was obtained in LiDAR-based video object detection with ST-FF than 3D object detectors based on a single point set. In addition, by applying the proposed method to the 3D object detectors based on a single point set, we demonstrate that our methods are applicable to the existing LiDAR-based detectors.