Progressive Multi-View Instance Matching: Occlusion-Robust Approach with Initial Segmentation Enhancement

Abstract

The growing demand for immersive media has led to active research on multi-view images. However, challenges such as occlusion and the lack of publicly available multi-view datasets hinder progress in this field. To address these issues, we introduce a novel dataset, SMIIM (Synthetic Multi-view Images for Instance Matching), designed for multi-view instance matching. We also propose a progressive matching algorithm, a three-stage process that effectively handles occlusion. Finally, we improve the image segmentation results by refining the masks generated by existing networks with our matching results. Compared to existing instance matching algorithms, our method not only provides faster processing time but also significantly enhances performance, achieving an average ID matching accuracy of 97.5% and a mean Intersection over Union (mIoU) improvement of 19.8% not only on our dataset but also on standardized datasets and real-world datasets. Multi-object matching in multi-view environments is rare, making our research a valuable contribution to this field. The SMIIM dataset will be released to facilitate further research and development in the field of multi-view image processing.