Abstract
In this paper, we explore the task of generating expansive outdoor scenes, ranging from castles to high-rises. Unlike indoor scene generation, which has been a primary focus of prior work, outdoor scene generation presents unique challenges, including wide variations in scene heights and the need for a method capable of rapidly producing large landscapes. To address this, we propose an efficient approach that encodes scene chunks as uniform vector sets, offering better compression and performance than the spatially structured latents used in prior methods. Furthermore, we train an explicit outpainting model for unbounded generation, which improves coherence compared to prior resampling-based inpainting schemes while also speeding up generation by eliminating extra diffusion steps. To facilitate this task, we curate NuiScene43, a small but high-quality set of scenes, preprocessed for joint training. Notably, when trained on scenes of varying styles, our model can blend different environments, such as rural houses and city skyscrapers, within the same scene, highlighting the potential of our curation process to leverage heterogeneous scenes for joint training.
Abstract (translated)
在这篇论文中，我们探讨了生成广阔户外场景的任务，这些场景从城堡到高层建筑不等。与先前研究主要关注的室内场景生成不同，户外场景生成面临着独特的挑战，包括场景高度变化大以及需要一种能够快速生产大规模景观的方法。为了解决这些问题，我们提出了一种高效的方法，该方法将场景块编码成均匀向量集合，相比之前方法使用的空间结构化潜在变量，这种方法提供了更好的压缩和性能表现。 此外，我们还训练了一个显式的“outpainting”模型用于无边界生成，与先前基于重采样的填充方案相比，这种方案能够提高连贯性，并通过消除额外的扩散步骤加快了生成速度。为了便于这项任务，我们整理了NuiScene43，这是一个虽小但质量高的一组场景集，经过预处理以进行联合训练。 值得注意的是，在不同风格场景上训练时，我们的模型可以在同一场景中融合不同的环境，如乡村房屋和城市摩天大楼，这突显出我们的数据整理过程能够利用异构场景进行联合训练的潜力。
URL
https://arxiv.org/abs/2503.16375
