Efficient Image Super-Resolution Using Dynamic Quality Control with Recursive Model Structures

Abstract

Nowadays, as the demand for accurate object detection (OD) applications is increasing, several attempts have been made to introduce convolutional neural network (CNN)-based super-resolution (SR) into these applications to further improve their target accuracy. OD systems require real-time processing because they are widely used in latency-critical applications such as autonomous driving, augmented reality, and surveillance cameras. However, due to their high computational and memory requirements, the introduction of SR in OD systems often makes real-time processing difficult. To reduce computation, one possible solution is to make the SR network size smaller, but this may not be the best solution because it lowers the quality of the reconstructed image. In addition, performing SR on the entire frame is inefficient because there are unnecessary background elements other than the objects that need to be detected within a single frame. Performing SR on specific regions of interest (ROIs) instead can improve the overall efficiency. Therefore, we propose an efficient dynamic quality control SR (DQC-SR) system that can dynamically adjust the inference rate according to the number of ROIs by introducing recursive and early-exit architectures into the baseline SR network. The DQC-SR network (DQC-SRNet) adopts a recursive architecture in which layers share identical parameters and an early-exit architecture that adjusts the network’s depth through bypassing paths. The recursive architecture can reduce memory consumption, while the early-exit architecture can adapt computation dynamically. With these two architectures, the DQC-SR system enables real-time SR inference on edge devices. For evaluation, we conducted example scenarios based on license plate recognition (LPR) and demonstrated the effectiveness of the DQC-SR system.