Abstract
Realistic and diverse multi-agent driving scenes are crucial for evaluating autonomous vehicles, but safety-critical events which are essential for this task are rare and underrepresented in driving datasets. Data-driven scene generation offers a low-cost alternative by synthesizing complex traffic behaviors from existing driving logs. However, existing models often lack controllability or yield samples that violate physical or social constraints, limiting their usability. We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling from a scene generation model. OMEGA re-anchors each reverse diffusion step via constrained optimization, steering the generation towards physically plausible and behaviorally coherent trajectories. Building on this framework, we formulate ego-attacker interactions as a game-theoretic optimization in the distribution space, approximating Nash equilibria to generate realistic, safety-critical adversarial scenarios. Experiments on nuPlan and Waymo show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes from 32.35% to 72.27% for free exploration capabilities, and from 11% to 80% for controllability-focused generation. Our approach can also generate $5\times$ more near-collision frames with a time-to-collision under three seconds while maintaining the overall scene realism.
Abstract (translated)
真实且多样化的多代理驾驶场景对于评估自动驾驶车辆至关重要，但这些场景中关键的安全事件（对任务来说是必不可少的）在驾驶数据集中非常罕见并且代表性不足。基于数据驱动的场景生成提供了一种低成本的选择，通过从现有的驾驶日志中合成复杂的交通行为来实现这一点。然而，现有模型往往缺乏可控性或产生违反物理和社会约束的样本，限制了它们的应用。 我们提出了OMEGA框架，这是一个优化引导、无需训练的框架，在基于扩散的方法进行场景生成采样过程中强制执行结构一致性和交互意识。通过在每个反向扩散步骤中应用有约束条件的优化方法，OMEGA重新定位每次迭代，使生成的过程朝着物理上合理且行为连贯的方向前进。 在此框架基础上，我们将自我车辆与攻击者之间的互动作为分布空间中的博弈论优化问题来处理，逼近纳什均衡以生成现实且具有安全挑战性的对抗性场景。在nuPlan和Waymo数据集上的实验表明，OMEGA提高了生成的场景的真实感、一致性和可控性，在自由探索能力方面将物理和行为上有效场景的比例从32.35%提升至72.27%，而在聚焦于可控性生成的情况下，这一比例则从11%大幅提升到80%。此外，我们的方法还能在保持整体场景真实性的前提下，产生五倍数量的接近碰撞帧，这些帧的时间距离（time-to-collision）小于三秒。 通过这样的改进，OMEGA不仅提高了合成驾驶场景的质量和真实性，还为自动驾驶车辆的安全性和性能评估提供了更有价值的数据。
URL
https://arxiv.org/abs/2512.07661
