Abstract
With the rapid advancement of autonomous driving technology, a lack of data has become a major obstacle to enhancing perception model accuracy. Researchers are now exploring controllable data generation using world models to diversify datasets. However, previous work has been limited to studying image generation quality on specific public datasets. There is still relatively little research on how to build data generation engines for real-world application scenes to achieve large-scale data generation for challenging scenes. In this paper, a simulator-conditioned scene generation engine based on world model is proposed. By constructing a simulation system consistent with real-world scenes, simulation data and labels, which serve as the conditions for data generation in the world model, for any scenes can be collected. It is a novel data generation pipeline by combining the powerful scene simulation capabilities of the simulation engine with the robust data generation capabilities of the world model. In addition, a benchmark with proportionally constructed virtual and real data, is provided for exploring the capabilities of world models in real-world scenes. Quantitative results show that these generated images significantly improve downstream perception models performance. Finally, we explored the generative performance of the world model in urban autonomous driving scenarios. All the data and code will be available at this https URL.
Abstract (translated)
随着自动驾驶技术的迅速发展，数据不足已成为提高感知模型精度的主要障碍。研究人员正探索使用世界模型进行可控的数据生成以丰富数据集。然而，之前的工作主要集中在特定公开数据集上的图像生成质量研究上，对于如何构建适用于真实应用场景的数据生成引擎以实现大规模挑战性场景下的数据生成的研究还相对较少。 本文提出了一种基于世界模型的仿真条件化场景生成引擎。通过构建与现实场景一致的仿真系统，可以收集任何场景所需的模拟数据和标签作为在世界模型中进行数据生成的条件。这是一种创新性的数据生成流程，它结合了仿真引擎强大的场景模拟能力和世界模型的数据生成能力。 此外，本文还提供了一个由虚拟数据和实际数据按比例构建的基准测试平台，用以探索世界模型在现实场景中的能力。定量结果表明，这些生成的图像显著提升了下游感知模型的表现性能。最后，我们研究了世界模型在城市自动驾驶场景下的生成性能。所有数据及代码将在该网址提供：[此 URL]
URL
https://arxiv.org/abs/2503.13952
