SIFNet: Free-form image inpainting using color split-inpaint-fuse approach
- Authors
- Uddin, S. M. Nadim; Jung, Yong Ju
- Issue Date
- Aug-2022
- Publisher
- ACADEMIC PRESS INC ELSEVIER SCIENCE
- Keywords
- Image inpainting; Convolutional neural network; Generative adversarial networks; Attention mechanisms; Color space decomposition
- Citation
- COMPUTER VISION AND IMAGE UNDERSTANDING, v.221
- Journal Title
- COMPUTER VISION AND IMAGE UNDERSTANDING
- Volume
- 221
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/84995
- DOI
- 10.1016/j.cviu.2022.103446
- ISSN
- 1077-3142
- Abstract
- Recent deep learning-based approaches have shown outstanding performance in generating visually plausible and refined contents for the missing regions in free-form image inpainting tasks. However, most of the existing methods employ a coarse-to-refine approach where the refinement process depends on a single coarse estimation, often leading to texture and structure inconsistencies. Though several existing methods focus on incorporating additional inputs to mitigate this problem, no learning-based studies have investigated the effects of decomposing input corrupted image into luma and chroma images and performing decoupled inpainting of the decomposed components. To this end, we propose a Split-Inpaint-Fuse Network (SIFNet), an end-to-end two-stage inpainting approach that uses a split-inpaint sub-network for separately inpainting the corrupted luma and chroma images using two decoupled branches in the coarse stage and a fusion sub-network for fusing the inpainted luma and chroma images into a refined image in the refinement stage. Additionally, we propose two attention mechanisms for the coarse stage - a progressive context module to find the patch-level feature similarity for the luma image reconstruction and a spatial-channel context module to find important spatial and channel features for the chroma image reconstruction. Experimental results reveal that our Split-Inpaint-Fuse approach outperforms the existing inpainting methods by comparative margins. In addition, extensive ablation studies confirm the effectiveness of the proposed approach, constituting modules and architectural choices.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - IT융합대학 > 소프트웨어학과 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/84995)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.