Abstract
We present DriveGen3D, a novel framework for generating high-quality and highly controllable dynamic 3D driving scenes that addresses critical limitations in existing methodologies. Current approaches to driving scene synthesis either suffer from prohibitive computational demands for extended temporal generation, focus exclusively on prolonged video synthesis without 3D representation, or restrict themselves to static single-scene reconstruction. Our work bridges this methodological gap by integrating accelerated long-term video generation with large-scale dynamic scene reconstruction through multimodal conditional control. DriveGen3D introduces a unified pipeline consisting of two specialized components: FastDrive-DiT, an efficient video diffusion transformer for high-resolution, temporally coherent video synthesis under text and Bird's-Eye-View (BEV) layout guidance; and FastRecon3D, a feed-forward reconstruction module that rapidly builds 3D Gaussian representations across time, ensuring spatial-temporal consistency. Together, these components enable real-time generation of extended driving videos (up to $424\times800$ at 12 FPS) and corresponding dynamic 3D scenes, achieving SSIM of 0.811 and PSNR of 22.84 on novel view synthesis, all while maintaining parameter efficiency.
Abstract (translated)
我们介绍了DriveGen3D，这是一个用于生成高质量、高度可控的动态3D驾驶场景的新颖框架，解决了现有方法中的关键限制。当前的驾驶场景合成方法要么在长时间生成时面临计算需求过高的问题，要么仅专注于没有3D表示的长时间视频合成都市仿真中常见的情况，或者局限于静态单一场景重建。 我们的工作通过结合加速长期视频生成和大规模动态场景重构来弥合这些方法论之间的差距，并且支持多模态条件控制。DriveGen3D引入了一个统一的工作流程，包括两个专门化的组件：FastDrive-DiT是一个高效的视频扩散变压器，在文本和鸟瞰图（BEV）布局引导下进行高分辨率、时间连贯的视频合成；以及FastRecon3D，这是一个前馈重构模块，可以快速构建时序中的三维高斯表示，并保证空间-时间一致性。这两个组件共同实现实时生成扩展驾驶视频（高达424x800像素，12 FPS）和相应的动态3D场景，在新视角合成中达到SSIM为0.811、PSNR为22.84的性能指标，同时保持参数效率。
URL
https://arxiv.org/abs/2510.15264
