Abstract
The synthesis of immersive 3D scenes from text is rapidly maturing, driven by novel video generative models and feed-forward 3D reconstruction, with vast potential in AR/VR and world modeling. While panoramic images have proven effective for scene initialization, existing approaches suffer from a trade-off between visual fidelity and explorability: autoregressive expansion suffers from context drift, while panoramic video generation is limited to low resolution. We present Stepper, a unified framework for text-driven immersive 3D scene synthesis that circumvents these limitations via stepwise panoramic scene expansion. Stepper leverages a novel multi-view 360° diffusion model that enables consistent, high-resolution expansion, coupled with a geometry reconstruction pipeline that enforces geometric coherence. Trained on a new large-scale, multi-view panorama dataset, Stepper achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches, thereby setting a new standard for immersive scene generation.
Abstract (translated)
基于新颖的视频生成模型与前馈式三维重建技术，从文本生成沉浸式三维场景的方法正迅速成熟，在增强现实/虚拟现实及世界建模领域展现出巨大潜力。尽管全景图像已被证明能有效初始化场景，但现有方法在视觉保真度与可探索性之间存在权衡：自回归扩展面临上下文漂移问题，而全景视频生成则受限于低分辨率。本文提出Stepper——一个统一的文本驱动沉浸式三维场景合成框架，通过分步全景场景扩展规避了上述限制。Stepper采用新型多视角360°扩散模型实现一致的高分辨率扩展，并结合几何重建流程确保几何连贯性。该框架在新构建的大规模多视角全景数据集上训练，实现了最先进的保真度与结构一致性，超越了先前方法，为沉浸式场景生成树立了新标杆。
URL
https://arxiv.org/abs/2603.28980
