TranRF: 3D Reconstruction Approach with Effective Transferability in Context-Dependent Scenes

Abstract

Breakthroughs in neural field representations, such as Neural Radiance Fields (NeRF) and its subsequent developments have enabled 3D reconstruction in various application scenarios. However, challenges are still encountered regarding generation accuracy, visual quality, and computational efficiency. To address the challenges in 3D reconstruction, in this paper, a method based on Single-stage diffusion nerf (SSDNeRF), called TranRF, is proposed. TranRF combines Swin Transformer with multimodal data and incorporates a genetic algorithm (GA) for feature selection, which not only enhances the accuracy of 3D reconstruction but also improves feature selection efficiency. Firstly, the Swin Transformer module helps SSDNeRF to capture rich contextual information at the multi-scale level. Secondly, the ability of the model to perceive 3D geometrical information is enhanced by the introduction of multimodal data, such as depth maps and normal maps. Then, the introduction of GA in feature selection and optimization further enhances the model's automation capability and adaptability. Finally, the evaluation on the ShapenetRender dataset shows that TranRF not only matches but, in some cases, surpasses existing state-of-the-art techniques, demonstrating its capability in enhancing both precision and computational efficiency.