Dynamic Rate Neural Acceleration Using Multiprocessing Mode Support

Abstract

Multiobject detection has become an integral component in various neural applications, such as autonomous driving and augmented reality. The system should be able to recognize and process multiple objects simultaneously. Moreover, the performance requirements for this system can be dynamically changed depending on the number of regions of interest (ROIs) in each frame. Consequently, the processing unit (PU) of the neural acceleration system should provide various inference rates. Therefore, we present a field-programmable gate array (FPGA)-based dynamic rate neural acceleration system called MultiLockOn to dynamically change the inference performance according to the number of ROIs per frame. It supports multiprocessing modes with different speeds through the introduction of novel multi-mode processing engines (PEs) comprising minimum reconfigurable interconnections across inference modes to minimize hardware overhead. The MultiLockOn system can provide an improvement of up to 4<inline-formula> <tex-math notation=LaTeX>$\times$</tex-math> </inline-formula> in the inference performance compared to that of DNNWeaver and 5.7<inline-formula> <tex-math notation=LaTeX>$\times$</tex-math> </inline-formula> compared to that of the ARM Cortex-A53 with minimum accuracy loss by supporting the multiprocessing modes.