Abstract
Vast and high-quality data are essential for end-to-end autonomous driving systems. However, current driving data is mainly collected by vehicles, which is expensive and inefficient. A potential solution lies in synthesizing data from real-world images. Recent advancements in 3D reconstruction demonstrate photorealistic novel view synthesis, highlighting the potential of generating driving data from images captured on the road. This paper introduces a novel method, I2V-GS, to transfer the Infrastructure view To the Vehicle view with Gaussian Splatting. Reconstruction from sparse infrastructure viewpoints and rendering under large view transformations is a challenging problem. We adopt the adaptive depth warp to generate dense training views. To further expand the range of views, we employ a cascade strategy to inpaint warped images, which also ensures inpainting content is consistent across views. To further ensure the reliability of the diffusion model, we utilize the cross-view information to perform a confidenceguided optimization. Moreover, we introduce RoadSight, a multi-modality, multi-view dataset from real scenarios in infrastructure views. To our knowledge, I2V-GS is the first framework to generate autonomous driving datasets with infrastructure-vehicle view transformation. Experimental results demonstrate that I2V-GS significantly improves synthesis quality under vehicle view, outperforming StreetGaussian in NTA-Iou, NTL-Iou, and FID by 45.7%, 34.2%, and 14.9%, respectively.
Abstract (translated)
大规模且高质量的数据对于端到端的自动驾驶系统至关重要。然而，目前驾驶数据主要通过车辆采集，这种方式既昂贵又效率低下。一种潜在的解决方案是从现实世界的图像中合成数据。最近在三维重建方面的进展展示了能够生成逼真的新视角图像，这凸显了从道路上捕捉的真实图像生成驾驶数据的潜力。本文介绍了一种新的方法——I2V-GS（Infrastructure to Vehicle Gaussian Splatting），该方法能将基础设施视图转换为车辆视图，并利用高斯点阵技术进行细节处理。 从稀疏的基础设施视角重建场景并在大视角变化下渲染是一个极具挑战性的问题。为此，我们采用自适应深度变形生成密集训练图像。为了进一步扩大可观察角度范围，我们采用了级联策略对变换后的图像进行修补，同时也确保不同视图之间的内容一致性和连贯性。 此外，为增强扩散模型的可靠性，我们利用跨视角信息进行了基于信心指导的优化。同时，我们还推出了RoadSight数据集，这是一个多模态、多种视角的真实场景基础设施视图数据集。 据我们所知，I2V-GS是首个能够通过基础设施到车辆视角转换生成自动驾驶数据集的框架。实验结果表明，在车辆视角下，I2V-GS在NTA-Iou（新颖视角下的IoU）、NTL-Iou（新任务下的IoU）以及FID（Frechet Inception Distance）等评价指标上显著提高了合成质量，分别优于StreetGaussian方法45.7%、34.2%和14.9%。
URL
https://arxiv.org/abs/2507.23683
