Abstract
We present UniFuture, a simple yet effective driving world model that seamlessly integrates future scene generation and perception within a single framework. Unlike existing models focusing solely on pixel-level future prediction or geometric reasoning, our approach jointly models future appearance (i.e., RGB image) and geometry (i.e., depth), ensuring coherent predictions. Specifically, during the training, we first introduce a Dual-Latent Sharing scheme, which transfers image and depth sequence in a shared latent space, allowing both modalities to benefit from shared feature learning. Additionally, we propose a Multi-scale Latent Interaction mechanism, which facilitates bidirectional refinement between image and depth features at multiple spatial scales, effectively enhancing geometry consistency and perceptual alignment. During testing, our UniFuture can easily predict high-consistency future image-depth pairs by only using the current image as input. Extensive experiments on the nuScenes dataset demonstrate that UniFuture outperforms specialized models on future generation and perception tasks, highlighting the advantages of a unified, structurally-aware world model. The project page is at this https URL.
Abstract (translated)
我们介绍了UniFuture，这是一种简单而有效的驾驶世界模型，它在一个单一框架内无缝整合了未来场景生成和感知。与现有的仅专注于像素级未来预测或几何推理的模型不同，我们的方法同时建模未来的外观（即RGB图像）和几何结构（即深度），确保一致的预测结果。具体来说，在训练阶段，我们首先引入了一种双潜在共享方案，该方案在一个共享的潜在空间中传输图像和深度序列，使两种模式都能从共同特征学习中获益。此外，我们还提出了一种多尺度潜互动机制，这一机制促进了在多个空间尺度下图象与深度特征之间的双向细化过程，有效增强了几何一致性及感知对齐。 测试阶段，UniFuture可以通过仅使用当前图像作为输入来轻易预测高一致性的未来图像-深度配对。在nuScenes数据集上的广泛实验表明，UniFuture在未来的生成和感知任务中超越了专门化的模型，突显出统一的、结构意识强的世界模型的优势。项目的网页链接如下：[此URL]（请将方括号中的文字替换为实际网址）。
URL
https://arxiv.org/abs/2503.13587
