Abstract
This paper presents GS-RoadPatching, an inpainting method for driving scene completion by referring to completely reconstructed regions, which are represented by 3D Gaussian Splatting (3DGS). Unlike existing 3DGS inpainting methods that perform generative completion relying on 2D perspective-view-based diffusion or GAN models to predict limited appearance or depth cues for missing regions, our approach enables substitutional scene inpainting and editing directly through the 3DGS modality, extricating it from requiring spatial-temporal consistency of 2D cross-modals and eliminating the need for time-intensive retraining of Gaussians. Our key insight is that the highly repetitive patterns in driving scenes often share multi-modal similarities within the implicit 3DGS feature space and are particularly suitable for structural matching to enable effective 3DGS-based substitutional inpainting. Practically, we construct feature-embedded 3DGS scenes to incorporate a patch measurement method for abstracting local context at different scales and, subsequently, propose a structural search method to find candidate patches in 3D space effectively. Finally, we propose a simple yet effective substitution-and-fusion optimization for better visual harmony. We conduct extensive experiments on multiple publicly available datasets to demonstrate the effectiveness and efficiency of our proposed method in driving scenes, and the results validate that our method achieves state-of-the-art performance compared to the baseline methods in terms of both quality and interoperability. Additional experiments in general scenes also demonstrate the applicability of the proposed 3D inpainting strategy. The project page and code are available at: this https URL
Abstract (translated)
这篇论文介绍了GS-RoadPatching，这是一种用于驾驶场景完成的图像修复方法，通过参考完全重建区域来进行操作，这些完全重建区域由三维高斯点（3D Gaussian Splatting, 3DGS）表示。与现有的基于2D透视视角扩散或GAN模型预测缺失区域的有限外观或深度线索的生成式补全方法不同，我们的方法直接在3DGS模态下实现了替代场景修复和编辑，从而摆脱了对2D跨模式时空一致性的需求，并消除了重新训练高斯分布所需的大量时间。我们的关键见解在于驾驶场景中的高度重复性图案在其隐式的3DGS特征空间中往往共享多模态相似性，这特别适合于结构匹配以实现有效的基于3DGS的替代修复。 在实践中，我们构建了嵌入特征的3DGS场景，并引入了一种用于不同尺度下抽象局部上下文的补丁测量方法。随后，我们提出了一种结构搜索方法，在3D空间中有效找到候选补丁。最后，我们提出了一种简单而有效的替换与融合优化方法，以实现更好的视觉和谐性。 我们在多个公开数据集上进行了广泛的实验，展示了所提方法在驾驶场景中的有效性和效率，并且结果验证了我们的方法相比基线方法，在质量和互操作性方面均达到了业界领先水平。此外，一般场景中的额外实验也证明了所提出的3D修复策略的应用可行性。项目页面和代码可在以下网址获取：this https URL
URL
https://arxiv.org/abs/2509.19937
