Abstract
Existing solutions for 3D semantic occupancy prediction typically treat the task as a one-shot 3D voxel-wise segmentation perception problem. These discriminative methods focus on learning the mapping between the inputs and occupancy map in a single step, lacking the ability to gradually refine the occupancy map and the reasonable scene imaginative capacity to complete the local regions somewhere. In this paper, we introduce OccGen, a simple yet powerful generative perception model for the task of 3D semantic occupancy prediction. OccGen adopts a ''noise-to-occupancy'' generative paradigm, progressively inferring and refining the occupancy map by predicting and eliminating noise originating from a random Gaussian distribution. OccGen consists of two main components: a conditional encoder that is capable of processing multi-modal inputs, and a progressive refinement decoder that applies diffusion denoising using the multi-modal features as conditions. A key insight of this generative pipeline is that the diffusion denoising process is naturally able to model the coarse-to-fine refinement of the dense 3D occupancy map, therefore producing more detailed predictions. Extensive experiments on several occupancy benchmarks demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods. For instance, OccGen relatively enhances the mIoU by 9.5%, 6.3%, and 13.3% on nuScenes-Occupancy dataset under the muli-modal, LiDAR-only, and camera-only settings, respectively. Moreover, as a generative perception model, OccGen exhibits desirable properties that discriminative models cannot achieve, such as providing uncertainty estimates alongside its multiple-step predictions.
Abstract (translated)
目前针对3D语义占有预测的解决方案通常将其视为一次性的3D体素级分割感知问题。这些判别方法集中精力在单步内学习输入与占有图之间的映射，而缺乏逐步精炼占有图和完成局部区域的合理场景想象能力。在本文中，我们介绍了OccGen，一个简单而强大的3D语义占有预测生成感知模型。OccGen采用了一种“噪声到占有”的生成范式，通过预测和消除源于随机高斯分布的噪声来逐步精炼占有图。OccGen由两个主要组件组成：一个条件编码器，能够处理多模态输入，和一个 progressive refinement decoder，通过多模态特征应用扩散去噪。这一生成流程的关键见解是，扩散去噪过程自然能够建模出密集3D占有图的粗细精炼，从而产生更详细的预测。在多个占有基准测试中进行的广泛实验证明，与最先进的方法相比，所提出的方法具有有效性。例如，在nuScenes-Occupancy数据集上，OccGen分别相对增强了mIoU by 9.5%，6.3%和13.3%。此外，作为生成感知模型，OccGen表现出判别模型无法实现的一些有价值的特性，例如在多次预测中提供不确定性估计。
URL
https://arxiv.org/abs/2404.15014
