Detection under noise effect of tags and complex arrangement of pile with Cycle-GAN and Mask-RCNN

Abstract

Vision-based box recognition on the pallet plays the main role to provide the picking guideline in an automation system of box de-palletization utilizing robots. Nevertheless, the complexity level of the working region significantly affects to the quality of this procedure outcome. And this level is represented by factors, such as the appearance of box containing multiple types of labels and tags. Commonly, a large-scale vision dataset is required to be generated for well-training a deep learning model, which allow it to detect on diverse complexity conditions. However, a lot of effort and time will be needed to construct this dataset. This paper aims to develop a systematic image processing algorithm to remove unnecessary portion and emphasize the key features. The core of the algorithm is image transformation steps utilizing the consistent generative adversarial network (Cycle GAN) for removing main obstacles of recognition such as adhesive labels or tapes. To improve segmentation quality, the depth map-based feature extraction is proposed to emphasize required features such as boundaries of boxes. By utilizing the processed images as inputs for training Mask R-CNN model, the advanced segmentation results are obtained, and the exact position required for de-palletizing can be predicted. The superior performance of the proposed method was confirmed by predicting the picking point on the segmentation result in a total of 4000 cases that simulates the complex surface pattern and spatial arrangement of the actual de-palletizing site. © 2022 IEEE.