Robust image completion and masking with application to robotic bin picking

Abstract

Automated image completion and masking have been emerged as a subject of keen interest due to their impact on image modification and interpretation. The current state-of-the-art approaches require a fixed format of missing parts and are ineffective for handling corrupted images. Besides, they focus exclusively on the image completion without taking into consideration the image masking as an inverse process of completion. This paper proposes a deep learning approach to an integrated framework of image completion and masking based on the cross-mapping generative adversarial network or CM-GAN, in short. CM-GAN offers the robustness in image completion under corruptions as well as the capability of synthesizing various masked images with arbitrary mask locations and shapes. In particular, the capability of CM-GAN in image masking is shown to be extended into the removal of unwanted backgrounds in images. We verify the superior performance of CM-GAN for image completion and masking based on extensive experiments. Furthermore, we implement a deep learning based robotic bin picking to demonstrate that the background removal capability of CM-GAN plays a key role for estimating the 3D pose of randomly filed multiple industrial parts in a bin. (C) 2020 Elsevier B.V. All rights reserved.