Abstract
Synthesis of diverse driving scenes serves as a crucial data augmentation technique for validating the robustness and generalizability of autonomous driving systems. Current methods aggregate high-definition (HD) maps and 3D bounding boxes as geometric conditions in diffusion models for conditional scene generation. However, implicit inter-condition dependency causes generation failures when control conditions change independently. Additionally, these methods suffer from insufficient details in both semantic and structural aspects. Specifically, brief and view-invariant captions restrict semantic contexts, resulting in weak background modeling. Meanwhile, the standard denoising loss with uniform spatial weighting neglects foreground structural details, causing visual distortions and blurriness. To address these challenges, we propose DrivePTS, which incorporates three key innovations. Firstly, our framework adopts a progressive learning strategy to mitigate inter-dependency between geometric conditions, reinforced by an explicit mutual information constraint. Secondly, a Vision-Language Model is utilized to generate multi-view hierarchical descriptions across six semantic aspects, providing fine-grained textual guidance. Thirdly, a frequency-guided structure loss is introduced to strengthen the model's sensitivity to high-frequency elements, improving foreground structural fidelity. Extensive experiments demonstrate that our DrivePTS achieves state-of-the-art fidelity and controllability in generating diverse driving scenes. Notably, DrivePTS successfully generates rare scenes where prior methods fail, highlighting its strong generalization ability.
Abstract (translated)
合成多样化的驾驶场景是验证自动驾驶系统鲁棒性和泛化能力的关键数据增强技术。目前的方法在扩散模型中聚合高精度地图和3D边界框作为几何条件来生成条件场景。然而，隐含的跨条件依赖性导致当控制条件独立变化时产生生成失败的问题。此外，这些方法在语义和结构细节方面存在不足。具体来说，简短且视图不变的描述限制了语义上下文，从而弱化了背景建模效果。同时，标准的空间均匀加权去噪损失忽略了前景结构细节，导致视觉失真和模糊。 为了解决这些问题，我们提出了DrivePTS方法，该方法集成了三个关键创新点。首先，我们的框架采用了一种渐进式学习策略来缓解几何条件之间的相互依赖性，并通过显式的互信息约束加以强化。其次，利用视觉-语言模型生成六个语义方面的多视角层次化描述，提供精细的文本指导。第三，引入了基于频率引导的结构损失以增强模型对高频元素的敏感度，提高前景结构的一致性和真实性。 广泛的实验表明，我们的DrivePTS在生成多样化驾驶场景时达到了最先进的保真度和可控性。特别值得注意的是，DrivePTS成功地生成了一些此前方法无法处理的罕见场景，这突显了其强大的泛化能力。
URL
https://arxiv.org/abs/2602.22549
