Abstract
Recent advancements in generative models have enabled 3D urban scene generation from satellite imagery, unlocking promising applications in gaming, digital twins, and beyond. However, most existing methods rely heavily on neural rendering techniques, which hinder their ability to produce detailed 3D structures on a broader scale, largely due to the inherent structural ambiguity derived from relatively limited 2D observations. To address this challenge, we propose Sat2City, a novel framework that synergizes the representational capacity of sparse voxel grids with latent diffusion models, tailored specifically for our novel 3D city dataset. Our approach is enabled by three key components: (1) A cascaded latent diffusion framework that progressively recovers 3D city structures from satellite imagery, (2) a Re-Hash operation at its Variational Autoencoder (VAE) bottleneck to compute multi-scale feature grids for stable appearance optimization and (3) an inverse sampling strategy enabling implicit supervision for smooth appearance this http URL overcome the challenge of collecting real-world city-scale 3D models with high-quality geometry and appearance, we introduce a dataset of synthesized large-scale 3D cities paired with satellite-view height maps. Validated on this dataset, our framework generates detailed 3D structures from a single satellite image, achieving superior fidelity compared to existing city generation models.
Abstract (translated)
最近在生成模型领域的进展使得从卫星图像生成三维城市场景成为可能，这为游戏、数字孪生等领域开启了新的应用前景。然而，大多数现有方法严重依赖于神经渲染技术，这种技术限制了它们在更广泛范围内产生详细三维结构的能力，主要是由于相对有限的二维观测导致的基本结构模糊性。为了应对这一挑战，我们提出了Sat2City，这是一种新颖的框架，它将稀疏体素网格的表现能力与潜在扩散模型相结合，专为我们的新型3D城市数据集设计。 该方法由三个关键组件组成： 1. 一个级联式潜在扩散框架，逐步从卫星图像中恢复三维城市结构。 2. 在其变分自编码器（VAE）瓶颈处进行的Re-Hash操作，用于计算多尺度特征网格以实现稳定的外观优化。 3. 一种反向采样策略，通过隐性监督使外观平滑化。 为了克服收集具有高质量几何和外观的真实世界城市规模三维模型这一挑战，我们引入了一个由合成大规模3D城市及其卫星视图高度地图组成的新型数据集。在该数据集上验证后，我们的框架能够从单张卫星图像生成详细的三维结构，并且与现有城市生成模型相比，在保真度方面表现更佳。
URL
https://arxiv.org/abs/2507.04403
