Abstract
Current generative models struggle to synthesize dynamic 4D driving scenes that simultaneously support temporal extrapolation and spatial novel view synthesis (NVS) without per-scene optimization. Bridging generation and novel view synthesis remains a major challenge. We present PhiGenesis, a unified framework for 4D scene generation that extends video generation techniques with geometric and temporal consistency. Given multi-view image sequences and camera parameters, PhiGenesis produces temporally continuous 4D Gaussian splatting representations along target 3D trajectories. In its first stage, PhiGenesis leverages a pre-trained video VAE with a novel range-view adapter to enable feed-forward 4D reconstruction from multi-view images. This architecture supports single-frame or video inputs and outputs complete 4D scenes including geometry, semantics, and motion. In the second stage, PhiGenesis introduces a geometric-guided video diffusion model, using rendered historical 4D scenes as priors to generate future views conditioned on trajectories. To address geometric exposure bias in novel views, we propose Stereo Forcing, a novel conditioning strategy that integrates geometric uncertainty during denoising. This method enhances temporal coherence by dynamically adjusting generative influence based on uncertainty-aware perturbations. Our experimental results demonstrate that our method achieves state-of-the-art performance in both appearance and geometric reconstruction, temporal generation and novel view synthesis (NVS) tasks, while simultaneously delivering competitive performance in downstream evaluations. Homepage is at \href{this https URL}{PhiGensis}.
Abstract (translated)
当前的生成模型在同时支持时间外推和空间新视图合成（NVS）的情况下，难以生成具有动态4D驾驶场景的数据。连接生成与新视图合成仍然是一个主要挑战。我们提出了PhiGenesis，这是一个用于4D场景生成的统一框架，它通过几何和时间一致性扩展了视频生成技术。给定多视角图像序列和摄像机参数后，PhiGenesis能够沿着目标3D轨迹产生连续的时间4D高斯散射表示。 在第一个阶段中，PhiGenesis利用带有新颖范围视图适配器的预训练视频VAE（变分自编码器），使从多视角图像进行前向传播式的4D重建成为可能。这种架构支持单帧或视频输入，并输出包括几何、语义和运动在内的完整4D场景。 在第二个阶段，PhiGenesis引入了一个基于几何引导的视频扩散模型，使用渲染的历史4D场景作为先验条件来生成未来的视图，这些未来视图根据轨迹进行调节。为了解决新视角中几何暴露偏差的问题，我们提出了Stereo Forcing，这是一种新的条件策略，在去噪过程中整合了几何不确定性。这种技术通过基于不确定性的扰动动态调整生成影响来增强时间一致性。 实验结果表明，我们的方法在外观和几何重建、时间生成和新视图合成（NVS）任务中均达到了最先进的性能，并且同时在下游评估中提供了竞争性表现。 PhiGenesis的主页位于[此处](https://this https URL "PhiGensis")。
URL
https://arxiv.org/abs/2509.20251
