Abstract
Scalable and reliable evaluation is increasingly critical in the end-to-end era of autonomous driving, where vision--language--action (VLA) policies directly map raw sensor streams to driving actions. Yet, current evaluation pipelines still rely heavily on real-world road testing, which is costly, biased toward limited scenario coverage, and difficult to reproduce. These challenges motivate a real-world simulator that can generate realistic future observations under proposed actions, while remaining controllable and stable over long horizons. We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space. Given synchronized multi-view camera history and a future action sequence, X-World generates future multi-camera video streams that follow the commanded actions. To ensure reproducible and editable scene rollouts, X-World further supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control (e.g., weather and time of day). Beyond world simulation, X-World also enables video style transfer by conditioning on appearance prompts while preserving the underlying action and scene dynamics. At the core of X-World is a multi-view latent video generator designed to explicitly encourage cross-view geometric consistency and temporal coherence under diverse control signals. Experiments show that X-World achieves high-quality multi-view video generation with (i) strong view consistency across cameras, (ii) stable temporal dynamics over long rollouts, and (iii) high controllability with strict action following and faithful adherence to optional scene controls. These properties make X-World a practical foundation for scalable and reproducible evaluation.
Abstract (translated)
可扩展且可靠的评估在自动驾驶端到端时代日益关键，其中视觉-语言-动作（VLA）策略直接将原始传感器流映射为驾驶动作。然而当前评估流程仍高度依赖真实路测，该方法成本高昂、场景覆盖有限且难以复现。这些挑战催生了对真实世界模拟器的需求——它需能在提议动作下生成逼真的未来观测，同时保持长期可控性与稳定性。我们提出X-World，一个动作条件的多摄像头生成式世界模型，直接在视频空间中模拟未来观测。给定同步多视角摄像头历史与未来动作序列，X-World可生成遵循指令动作的未来多摄像头视频流。为确保场景推演的可复现与可编辑性，X-World进一步支持对动态交通参与者与静态道路元素的可选控制，并保留文本提示接口以实现外观级控制（如天气与时段）。除世界模拟外，X-World还能通过外观提示条件实现视频风格迁移，同时保持底层动作与场景动态。X-World的核心是一个多视角隐空间视频生成器，专为在不同控制信号下显式促进跨视角几何一致性与时间连贯性而设计。实验表明，X-World实现了高质量多视角视频生成，具备：（i）摄像头间强视角一致性；（ii）长期推演中稳定的时间动态；（iii）通过严格动作跟随与忠实执行可选场景控制实现的高可控性。这些特性使X-World成为可扩展、可复现评估的实用基础。
URL
https://arxiv.org/abs/2603.19979
