Multi-Vehicle Tracking Using Heterogeneous Neural Networks for Appearance And Motion Features

Abstract

This paper presents a multi-vehicle tracking algorithm using appearance feature and motion history based on heterogeneous deep learning aimed at autonomous driving applications. Our proposed multi-vehicle tracking model follows the tracking-by-detection paradigm. To track multiple vehicles, we utilize the appearance and motion features of the target vehicles in consecutive frames. The proposed multi-vehicle tracking system employs a deep convolutional neural network, which is trained with a triplet loss minimization method to extract appearance features. The key contribution of the proposed method lies in a Long Short-Term Memory (LSTM) with a fully connected layer that accurately predicts the probability distribution of the next appearance and motion features of tracked objects. We constructed a multi-vehicle tracking dataset from various real road traffic using a camera sensor on a vehicle. To evaluate our proposed algorithm, we use several multi-target tracking datasets from the KITTI object tracking benchmark, which is a Public tracking dataset, as well as our evaluation dataset. Experimental results demonstrate that the proposed multi-vehicle tracking algorithm achieves a MOTA of 84.5% and MOTP 86.3% on the KITTI tracking dataset, and a MOTA of 81.8% and MOTP 84.8% on our evaluation dataset, an improvement of 8.6% and 9.6% over the previous methods.