Abstract
Generating explorable 3D scenes from a single image is a highly challenging problem in 3D vision. Existing methods struggle to support free exploration, often producing severe geometric distortions and noisy artifacts when the viewpoint moves far from the original perspective. We introduce \textbf{One2Scene}, an effective framework that decomposes this ill-posed problem into three tractable sub-tasks to enable immersive explorable scene generation. We first use a panorama generator to produce anchor views from a single input image as initialization. Then, we lift these 2D anchors into an explicit 3D geometric scaffold via a generalizable, feed-forward Gaussian Splatting network. Instead of treating the panorama as a single image for reconstruction, we project it into multiple sparse anchor views and reformulate the reconstruction task as multi-view stereo matching, which allows us to leverage robust geometric priors learned from large-scale multi-view datasets. A bidirectional feature fusion module is used to enforce cross-view consistency, yielding an efficient and geometrically reliable scaffold. Finally, the scaffold serves as a strong prior for a novel view generator to produce photorealistic and geometrically accurate views at arbitrary cameras. By explicitly conditioning on a 3D-consistent scaffold to perform reconstruction, One2Scene works stably under large camera motions, supporting immersive scene exploration. Extensive experiments show that One2Scene substantially outperforms state-of-the-art methods in panorama depth estimation, feed-forward 360° reconstruction, and explorable 3D scene generation. Code and models will be released.
Abstract (translated)
从单张图像生成可探索的三维场景是三维视觉中的一个高度挑战性的问题。现有方法在支持自由探索时往往难以克服，在视角远离原始视点时，会产生严重的几何畸变和噪声伪影。我们引入了**One2Scene**，这是一个有效的框架，它将这一病态问题分解为三个可处理的子任务，以实现沉浸式的可探索场景生成。 首先，使用全景生成器从单张输入图像中创建锚定视图作为初始化。然后，通过一个泛化的前向高斯散射网络将这些二维锚点提升到显式的三维几何支撑结构。不同于将全景视为单一图像进行重建，我们将它投影为多个稀疏的锚定视图，并重新定义了重建任务为多视角立体匹配问题，这使我们能够利用从大规模多视角数据集中学习到的强大几何先验知识。 双向特征融合模块被用来强制执行跨视图一致性，从而生成一个高效且几何上可靠的支撑结构。最后，该支撑结构作为先验条件用于新颖视角生成器中，以便在任意相机位置生成逼真且几何准确的视图。 通过显式地基于三维一致的支撑结构进行重建，One2Scene 在面对大范围相机运动时能稳定工作，支持沉浸式的场景探索。广泛的实验表明，One2Scene 在全景深度估计、前向360°重建和可探索三维场景生成方面显著优于现有的最先进方法。代码与模型将随后发布。
URL
https://arxiv.org/abs/2602.19766
