Abstract
Radiance field methods, such as Neural Radiance Field or 3D Gaussian Splatting, have emerged as seminal 3D representations for synthesizing realistic novel views. For practical applications, there is ongoing research on flexible scene editing techniques, among which object removal is a representative task. However, removing objects exposes occluded regions, often leading to unnatural appearances. Thus, studies have employed image inpainting techniques to replace such regions with plausible content - a task referred to as 3D scene inpainting. However, image inpainting methods produce one of many plausible completions for each view, leading to inconsistencies between viewpoints. A widely adopted approach leverages perceptual cues to blend inpainted views smoothly. However, it is prone to detail loss and can fail when there are perceptual inconsistencies across views. In this paper, we propose a novel 3D scene inpainting method that reliably produces realistic and perceptually consistent results even for complex scenes by leveraging a reference view. Given the inpainted reference view, we estimate the inpainting similarity of the other views to adjust their contribution in constructing an accurate geometry tailored to the reference. This geometry is then used to warp the reference inpainting to other views as pseudo-ground truth, guiding the optimization to match the reference appearance. Comparative evaluation studies have shown that our approach improves both the geometric fidelity and appearance consistency of inpainted scenes.
Abstract (translated)
辐射场方法，如神经辐射场或3D高斯点阵，在合成逼真的新视图方面作为重要的三维表示方式而崭露头角。在实际应用中，有关灵活场景编辑技术的研究正在进行之中，其中物体移除是一项代表性的任务。然而，移除物体会暴露被遮挡的区域，这常常会导致不自然的效果。因此，一些研究采用了图像修复技术来用合理的内容替代这些区域——这项工作被称为三维场景修复。但是，基于图像的方法为每个视图生成多种可能的完整内容之一，从而导致不同视角之间的不一致现象。一种广泛采用的方法是利用感知线索来平滑地融合修复后的视图。然而，这种方法容易丢失细节，并且当存在多视角间的感知差异时会失效。 在这篇论文中，我们提出了一种新的三维场景修复方法，它能够可靠地生成现实且在复杂场景中具有感知一致性的结果，通过利用参考视图来实现这一目标。给定修复后的参考视图后，我们会估计其他视图的修复相似度以调整它们对构建准确几何结构（针对参考）的贡献程度。随后使用该几何结构将参考视图中的修复内容转换到其他视角作为伪真实基准，并以此引导优化过程与参考视图外观相匹配。 比较评价研究显示我们的方法提升了修复场景在几何保真度和外观一致性方面的表现。
URL
https://arxiv.org/abs/2507.08434
