Abstract
Robotic world models are a promising paradigm for forecasting future environment states, yet their inference speed and the physical plausibility of generated trajectories remain critical bottlenecks, limiting their real-world applications. This stems from the redundancy of the prevailing frame-to-frame generation approach, where the model conducts costly computation on similar frames, as well as neglecting the semantic importance of key transitions. To address this inefficiency, we propose KeyWorld, a framework that improves text-conditioned robotic world models by concentrating transformers computation on a few semantic key frames while employing a lightweight convolutional model to fill the intermediate frames. Specifically, KeyWorld first identifies significant transitions by iteratively simplifying the robot's motion trajectories, obtaining the ground truth key frames. Then, a DiT model is trained to reason and generate these physically meaningful key frames from textual task descriptions. Finally, a lightweight interpolator efficiently reconstructs the full video by inpainting all intermediate frames. Evaluations on the LIBERO benchmark demonstrate that KeyWorld achieves a 5.68$\times$ acceleration compared to the frame-to-frame generation baseline, and focusing on the motion-aware key frames further contributes to the physical validity of the generated videos, especially on complex tasks. Our approach highlights a practical path toward deploying world models in real-time robotic control and other domains requiring both efficient and effective world models. Code is released at this https URL.
Abstract (translated)
机器人世界的模型是一种有前景的预测未来环境状态的方法，然而它们的推理速度和生成轨迹的物理合理性仍然存在关键瓶颈，限制了其在现实世界中的应用。这源于当前流行的帧对帧生成方法的冗余性，在这种方法中，模型会在相似的帧上进行昂贵的计算，并且忽略了关键过渡过程中的语义重要性。为了应对这种低效率问题，我们提出了KeyWorld框架，通过将Transformer计算集中在少数几个语义关键帧上，并使用轻量级卷积模型填充中间帧来改进基于文本条件的机器人世界模型。 具体来说，KeyWorld首先通过迭代简化机器人的运动轨迹来识别重要的过渡过程，获得真正的关键帧。然后训练一个DiT模型，从任务描述的文字中推理并生成这些具有物理意义的关键帧。最后，使用轻量级插值器高效地重建完整视频，通过填充所有中间帧进行重构。 在LIBERO基准测试上的评估显示，与帧对帧的生成基线相比，KeyWorld实现了5.68倍的速度提升，并且专注于运动感知关键帧进一步提升了所生成视频的物理合理性，尤其是在复杂任务中表现尤为突出。我们的方法强调了一条实用路径，将世界模型应用于实时机器人控制及其他需要高效和有效世界模型的领域。 代码可以在该链接处获取（原文中的具体链接未给出）。
URL
https://arxiv.org/abs/2509.21027
