Abstract
Generating high-fidelity, controllable, and annotated training data is critical for autonomous driving. Existing methods typically generate a single data form directly from a coarse scene layout, which not only fails to output rich data forms required for diverse downstream tasks but also struggles to model the direct layout-to-data distribution. In this paper, we introduce UniScene, the first unified framework for generating three key data forms - semantic occupancy, video, and LiDAR - in driving scenes. UniScene employs a progressive generation process that decomposes the complex task of scene generation into two hierarchical steps: (a) first generating semantic occupancy from a customized scene layout as a meta scene representation rich in both semantic and geometric information, and then (b) conditioned on occupancy, generating video and LiDAR data, respectively, with two novel transfer strategies of Gaussian-based Joint Rendering and Prior-guided Sparse Modeling. This occupancy-centric approach reduces the generation burden, especially for intricate scenes, while providing detailed intermediate representations for the subsequent generation stages. Extensive experiments demonstrate that UniScene outperforms previous SOTAs in the occupancy, video, and LiDAR generation, which also indeed benefits downstream driving tasks.
Abstract (translated)
生成高保真、可控且带有标注的训练数据对于自动驾驶至关重要。现有的方法通常直接从粗略的场景布局生成单一的数据形式，这不仅无法输出多样化的下游任务所需的丰富数据类型，还难以模拟直接从布局到数据分布的过程。在本文中，我们介绍了UniScene，这是首个用于生成驾驶场景中三种关键数据形式——语义占用、视频和激光雷达（LiDAR）数据的统一框架。UniScene采用了一个渐进式生成过程，将复杂的场景生成任务分解为两个分层步骤：(a) 首先从定制的场景布局生成一个富含语义和几何信息的元场景表示——即语义占用，并且(b) 在此基础上分别通过两种新颖的基于高斯联合渲染和先验指导稀疏建模的数据转移策略，生成视频和LiDAR数据。这种以占用为中心的方法减轻了生成复杂场景时的压力，同时为后续生成阶段提供了详细的中间表示形式。广泛的实验表明，UniScene在语义占用、视频和LiDAR数据的生成方面超越了之前的最优技术，并确实对下游驾驶任务产生了积极影响。
URL
https://arxiv.org/abs/2412.05435
