Comparison of optical flow image preprocessing options for state of the art deep learning models

Abstract

In recent years, many new deep learning approaches for solving optical flow estimation have been showing impressive results. But as model sizes have been becoming larger, executing them became a task that requires expensive, high end, hardware. Because most of these models use full colour RGB image pairs as an input, we test several image preprocessing methods with the goal of finding techniques that could alleviate some of these inefficiencies. We conducted these experiments using the state of the art GMA (Global Motion Aggregation) network architecture. Our results, first of all, show that optical flow can be estimated equally well with single channel greyscale images, this finding could be used to lower model sizes in general. We also find that calculating the derivative of an image in the direction of one of its axes leads to improvements in accuracy, but only in the case of less difficult optical flow data sets such as FlyingChairs and SIntel-clean. © 2022 IEEE.