Abstract
We introduce \textit{WonderVerse}, a simple but effective framework for generating extendable 3D scenes. Unlike existing methods that rely on iterative depth estimation and image inpainting, often leading to geometric distortions and inconsistencies, WonderVerse leverages the powerful world-level priors embedded within video generative foundation models to create highly immersive and geometrically coherent 3D environments. Furthermore, we propose a new technique for controllable 3D scene extension to substantially increase the scale of the generated environments. Besides, we introduce a novel abnormal sequence detection module that utilizes camera trajectory to address geometric inconsistency in the generated videos. Finally, WonderVerse is compatible with various 3D reconstruction methods, allowing both efficient and high-quality generation. Extensive experiments on 3D scene generation demonstrate that our WonderVerse, with an elegant and simple pipeline, delivers extendable and highly-realistic 3D scenes, markedly outperforming existing works that rely on more complex architectures.
Abstract (translated)
我们介绍了一个名为\textit{WonderVerse}的框架，这是一个简单却有效的生成可扩展三维场景的方法。与现有的依赖于迭代深度估计和图像修补方法不同，后者常常导致几何扭曲和不一致性，WonderVerse 利用了视频生成基础模型中嵌入的世界级先验知识来创建高度沉浸式且几何一致的3D环境。此外，我们提出了一种可控的3D场景扩展技术，以显著增加生成环境的规模。另外，我们引入了一个新的异常序列检测模块，该模块利用摄像机轨迹解决生成视频中的几何不一致性问题。最后，WonderVerse 兼容各种3D重建方法，既支持高效又高质量的生成。 在三维场景生成方面的广泛实验表明，我们的 WonderVerse 通过一个优雅且简单的管道，能够提供可扩展和高度逼真的3D场景，显著优于依赖更复杂架构的现有工作。
URL
https://arxiv.org/abs/2503.09160
