Abstract
Recent advancements in multi-view 3D reconstruction and novel-view synthesis, particularly through Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have greatly enhanced the fidelity and efficiency of 3D content creation. However, inpainting 3D scenes remains a challenging task due to the inherent irregularity of 3D structures and the critical need for maintaining multi-view consistency. In this work, we propose a novel 3D Gaussian inpainting framework that reconstructs complete 3D scenes by leveraging sparse inpainted views. Our framework incorporates an automatic Mask Refinement Process and region-wise Uncertainty-guided Optimization. Specifically, we refine the inpainting mask using a series of operations, including Gaussian scene filtering and back-projection, enabling more accurate localization of occluded regions and realistic boundary restoration. Furthermore, our Uncertainty-guided Fine-grained Optimization strategy, which estimates the importance of each region across multi-view images during training, alleviates multi-view inconsistencies and enhances the fidelity of fine details in the inpainted results. Comprehensive experiments conducted on diverse datasets demonstrate that our approach outperforms existing state-of-the-art methods in both visual quality and view consistency.
Abstract (translated)
最近，在多视角3D重建和新视角合成领域，特别是在使用神经辐射场（NeRF）和3D高斯点阵（3DGS）的技术进步极大地提高了三维内容创作的准确性和效率。然而，由于三维结构本身的不规则性以及维护多视图一致性的关键需求，对3D场景进行修复填充仍是一项具有挑战性的任务。 在这项工作中，我们提出了一种新颖的3D高斯修复框架，该框架通过利用稀疏的修复视图来重建完整的3D场景。我们的框架集成了自动掩码精炼过程和区域导向的不确定性指导优化策略。具体来说，我们使用一系列操作（包括高斯场景过滤和反向投影）来细化修复掩码，这使得能够更准确地定位被遮挡区域并恢复现实边界。 此外，我们在训练过程中提出了一种基于不确定性引导的细粒度优化策略，它估计了多视图图像中每个区域的重要性。这种方法减少了多视图之间的不一致性，并提高了修复结果中的细节准确性。 在多种数据集上进行的全面实验表明，我们的方法在视觉质量和视图一致性方面都优于现有的最先进的技术。
URL
https://arxiv.org/abs/2507.18023
