Abstract
Acquiring detailed 3D scenes typically demands costly equipment, multi-view data, or labor-intensive modeling. Therefore, a lightweight alternative, generating complex 3D scenes from a single top-down image, plays an essential role in real-world applications. While recent 3D generative models have achieved remarkable results at the object level, their extension to full-scene generation often leads to inconsistent geometry, layout hallucinations, and low-quality meshes. In this work, we introduce 3DTown, a training-free framework designed to synthesize realistic and coherent 3D scenes from a single top-down view. Our method is grounded in two principles: region-based generation to improve image-to-3D alignment and resolution, and spatial-aware 3D inpainting to ensure global scene coherence and high-quality geometry generation. Specifically, we decompose the input image into overlapping regions and generate each using a pretrained 3D object generator, followed by a masked rectified flow inpainting process that fills in missing geometry while maintaining structural continuity. This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning. Extensive experiments across diverse scenes show that 3DTown outperforms state-of-the-art baselines, including Trellis, Hunyuan3D-2, and TripoSG, in terms of geometry quality, spatial coherence, and texture fidelity. Our results demonstrate that high-quality 3D town generation is achievable from a single image using a principled, training-free approach.
Abstract (translated)
获取详细的三维场景通常需要昂贵的设备、多视角数据或复杂的建模过程。因此，一种轻量级的方法——从单一顶视图图像生成复杂三维场景，在实际应用中扮演着重要角色。尽管最近的三维生成模型在物体级别的表现非常出色，但它们扩展到全场景生成时往往会导致不一致的几何结构、布局幻觉以及低质量的网格。为此，我们在本研究中引入了3DTown，这是一个无需训练框架，旨在从单一顶视图图像合成真实且连贯的三维场景。 我们的方法基于两个原则：区域化生成以提高二维到三维的一致性和分辨率，以及空间感知的三维修复填充来确保全局场景的一致性及高质量几何结构生成。具体而言，我们将输入图像分解为重叠的区域，并使用预训练的三维对象生成器生成每个区域；随后进行掩码修正流修复填充过程，以填补缺失的几何信息并保持结构连续性。 这种模块化设计允许我们克服分辨率瓶颈问题，并在无需三维监督或微调的情况下保存空间结构。通过在各种场景中进行广泛的实验表明，3DTown 在几何质量、空间一致性以及纹理保真度方面均优于当前最佳基线模型（包括 Trellis, Hunyuan3D-2 和 TripoSG）。 我们的研究成果展示了基于单一图像生成高质量三维城镇的可行性，并且采用了一种无需训练的原理性方法。
URL
https://arxiv.org/abs/2505.15765
