Video analytics-based real-time intelligent crossing detection system (RICDS): Killer app for edge computing

Abstract

In this paper, we design and implement a video analytics based real-time intelligent crossing detection system (RICDS) as a novel safety service for the smart city. This system spans across the edge device and visualization unit, with reduced latency. To enhance object tracking in the proposed RICDS on the edge devices, which has limited computational resources, we first design an adaptive queue management-based object tracking (QMOT) scheme. This scheme adaptively controls the maximum queue size to meet the target delay requirement for the RICDS while the image frames come into the proposed queue to wait for object detection and tracking. In this sense, the maximum queue size is calculated by the target delay requirement and weighted-average of processing time, so that some redundant frames for object detection are automatically dropped due to the overflow of the maximum queue size. Moreover, we introduce the real-world-real-time (RR) tracking scheme that lightly tracks multiple objects (i.e., pedestrians in this scenario) by predicting the future position of the object and assigning unique IDs to them. Specifically, to predict the future positions of multiple objects in a lightweight manner, the possible movement boundary of every single object is determined by considering both the movement directions and movement speeds of each object to reduce search space for mapping IDs to the objects. Here, the movement directions and movement speeds are also calculated based on the real-world coordinate system after calibrating of image frame's coordinate system into the real-world one. Finally, we implement the proposed RICDS on various commercial edge devices and a visualization unit (i.e., the LED panel). Moreover, the GUI application at the remote server is provided for (i) monitoring the result of the detection and tracking and (ii) dynamically changing the configuration settings. The measurement-based experiments show that the proposed tracking scheme achieves a latency reduction of about 33%–62% compared to the benchmark schemes on the NVIDIA Xavier while exhibiting similar multi object tracking accuracy performance compared to the benchmark schemes. © 2022