Compact Representation of Light Field Data for Refocusing and Focal Stack Reconstruction Using Depth Adaptive Multi-CNN

Abstract

A light field camera can record image information of the same scene from different viewpoints. Its 4D data allow post processing among which digital refocusing is popular due to its widely practical use. However, in server-device applications, the transmission cost of the large volume of data at the server to a device is a major issue. In this paper, we propose a novel hardware-friendly refocusing representation and a depth adaptive parallel multi-CNN (DAPM) neural network to address this problem. At the server, an extended center sub-aperture image and depth map information are generated which can be used to reconstruct focal stack instead of the whole light field data. At on-chip devices, the focal stack is reconstructed by a DAPM network using the novel low-size refocusing representation data. Compared to conventional representation methods, the proposed approach shows a 32% reduction of storage size and transmission cost while retaining the same PSNR focal stack image quality. IEEE