Abstract
Outdoor 3D semantic scene generation produces realistic and semantically rich environments for applications such as urban simulation and autonomous driving. However, advances in this direction are constrained by the absence of publicly available, well-annotated datasets. We introduce SketchSem3D, the first large-scale benchmark for generating 3D outdoor semantic scenes from abstract freehand sketches and pseudo-labeled annotations of satellite images. SketchSem3D includes two subsets, Sketch-based SemanticKITTI and Sketch-based KITTI-360 (containing LiDAR voxels along with their corresponding sketches and annotated satellite images), to enable standardized, rigorous, and diverse evaluations. We also propose Cylinder Mamba Diffusion (CymbaDiff) that significantly enhances spatial coherence in outdoor 3D scene generation. CymbaDiff imposes structured spatial ordering, explicitly captures cylindrical continuity and vertical hierarchy, and preserves both physical neighborhood relationships and global context within the generated scenes. Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization. The code and dataset will be available at this https URL
Abstract (translated)
户外三维语义场景生成旨在为城市模拟和自动驾驶等应用创建现实且富含语义信息的环境。然而，这一领域的进步受到公开可用、标注完善的数据库缺失的限制。为此，我们推出了SketchSem3D，这是首个用于从抽象的手绘草图和卫星图像伪标签注释生成大规模三维户外语义场景的基准数据集。SketchSem3D包含两个子集：基于草图的SemanticKITTI和基于草图的KITTI-360（包括激光雷达体素及其对应的草图和标注过的卫星图像），以支持标准、严格且多样化的评估。此外，我们还提出了Cylinder Mamba Diffusion (CymbaDiff)，这一方法显著提升了户外三维场景生成的空间一致性。CymbaDiff施加了结构化空间顺序，并明确捕捉了圆柱连续性和垂直层次关系，在生成的场景中同时保留物理邻域关系和全局背景信息。在SketchSem3D上的广泛实验表明，CymbaDiff实现了更优的语义一致性和空间现实感，以及跨数据集的泛化能力。相关代码和数据集将在以下链接发布：[此网址] (请根据实际发布的地址替换该占位符)
URL
https://arxiv.org/abs/2510.13245
