Abstract
Autonomous driving systems rely heavily on multi-view images to ensure accurate perception and robust decision-making. To effectively develop and evaluate perception stacks and planning algorithms, realistic closed-loop simulators are indispensable. While 3D reconstruction techniques such as Gaussian Splatting offer promising avenues for simulator construction, the rendered novel views often exhibit artifacts, particularly in extrapolated perspectives or when available observations are sparse. We introduce ViewMorpher3D, a multi-view image enhancement framework based on image diffusion models, designed to elevate photorealism and multi-view coherence in driving scenes. Unlike single-view approaches, ViewMorpher3D jointly processes a set of rendered views conditioned on camera poses, 3D geometric priors, and temporally adjacent or spatially overlapping reference views. This enables the model to infer missing details, suppress rendering artifacts, and enforce cross-view consistency. Our framework accommodates variable numbers of cameras and flexible reference/target view configurations, making it adaptable to diverse sensor setups. Experiments on real-world driving datasets demonstrate substantial improvements in image quality metrics, effectively reducing artifacts while preserving geometric fidelity.
Abstract (translated)
自动驾驶系统高度依赖多视角图像以确保准确的感知和稳健的决策制定。为了有效地开发和评估感知栈与规划算法，现实主义闭环模拟器不可或缺。虽然像高斯点阵（Gaussian Splatting）这样的三维重建技术为构建模拟器提供了有前景的方法，但渲染的新视角往往会出现伪影，尤其是在外推视图或可用观察数据稀疏的情况下。我们推出了ViewMorpher3D，这是一个基于图像扩散模型的多视角图像增强框架，旨在提升驾驶场景中的照片真实感和多视角一致性。 与单视角方法不同，ViewMorpher3D同时处理一组渲染视角，这些视角以相机姿态、三维几何先验知识以及时间上相邻或空间上重叠的参考视图为条件。这使得模型能够推断缺失细节，抑制渲染伪影，并确保跨视角的一致性。我们的框架支持可变数量的摄像头和灵活的参考/目标视角配置，使其适用于各种传感器设置。 在真实驾驶数据集上的实验表明，在图像质量指标上取得了显著提升，有效减少了伪影同时保持了几何保真度。
URL
https://arxiv.org/abs/2601.07540
