Joint learning of blind video denoising and optical flow estimation

Abstract

Many deep-learning-based image/video denoising models have been developed, and recently, several approaches for training a denoising neural network without using clean images have been proposed. However, Noise2Noise method requires paired noisy data, and obtaining them is occasionally difficult, whereas other existing models trained using unpaired noisy data deliver limited performance. Obtaining an accurate optical flow from noisy videos is also a difficult task because conventional optical flow estimation methods are primarily focused on estimating the optical flow using clean videos. This study proposes a new framework to fine-tune video denoising and optical flow estimation networks using unpaired noisy videos. These two networks are jointly trained to realize synergy; an improvement in the denoising performance increases the accuracy of the flow estimation, and an improvement in the flow-estimation performance enhances the quality of the training data for the denoiser. Our experimental results reveal that proposed approach outperforms the existing training schemes in video denoising and also provides accurate optical flows even when the videos contain a considerable amount of noise.