Abstract
The completion, extension, and generation of 3D semantic scenes are an interrelated set of capabilities that are useful for robotic navigation and exploration. Existing approaches seek to decouple these problems and solve them oneoff. Additionally, these approaches are often domain-specific, requiring separate models for different data distributions, e.g. indoor vs. outdoor scenes. To unify these techniques and provide cross-domain compatibility, we develop a single framework that can perform scene completion, extension, and generation in both indoor and outdoor scenes, which we term Octree Latent Semantic Diffusion. Our approach operates directly on an efficient dual octree graph latent representation: a hierarchical, sparse, and memory-efficient occupancy structure. This technique disentangles synthesis into two stages: (i) structure diffusion, which predicts binary split signals to construct a coarse occupancy octree, and (ii) latent semantic diffusion, which generates semantic embeddings decoded by a graph VAE into voxellevel semantic labels. To perform semantic scene completion or extension, our model leverages inference-time latent inpainting, or outpainting respectively. These inference-time methods use partial LiDAR scans or maps to condition generation, without the need for retraining or finetuning. We demonstrate highquality structure, coherent semantics, and robust completion from single LiDAR scans, as well as zero-shot generalization to out-of-distribution LiDAR data. These results indicate that completion-through-generation in a dual octree graph latent space is a practical and scalable alternative to regression-based pipelines for real-world robotic perception tasks.
Abstract (translated)
三维语义场景的完成、扩展和生成是一系列相互关联的能力，这些能力对于机器人导航与探索非常有用。现有方法试图将这些问题解耦，并单独解决每一个问题。此外，这些方法往往是特定领域的，需要为不同的数据分布（如室内场景与室外场景）分别构建模型。为了统一这些技术并提供跨域兼容性，我们开发了一个单一的框架，能够在室内和室外场景中执行场景完成、扩展和生成任务，我们将该框架命名为Octree Latent Semantic Diffusion。 我们的方法直接在一种高效且稀疏的记忆节约型占用结构——双八叉树图隐式表示上操作。这种技术将合成过程分为两个阶段：(i) 结构扩散，预测二进制分割信号以构建一个粗略的占用八叉树；(ii) 隐含语义扩散，生成由图VAE解码成体素级别语义标签的语义嵌入。 为了执行语义场景完成或扩展任务，我们的模型利用了推断时的隐式插值（完成）和外推（扩展）。这些推断方法使用部分LiDAR扫描或地图作为条件进行生成，而无需重新训练或微调。我们展示了从单次LiDAR扫描中获得高质量结构、连贯语义以及稳健完成能力的能力，并且还能够零样本泛化到分布外的LiDAR数据。 这些结果表明，在双八叉树图隐式空间中的通过生成进行场景完成是一种实用而可扩展的方法，可以作为基于回归的工作流管道在现实世界的机器人感知任务中的一种替代方案。
URL
https://arxiv.org/abs/2509.16483
