Abstract
While a general embodied agent must function as a unified system, current methods are built on isolated models for understanding, world modeling, and control. This fragmentation prevents unifying multimodal generative capabilities and hinders learning from large-scale, heterogeneous data. In this paper, we propose Motus, a unified latent action world model that leverages existing general pretrained models and rich, sharable motion information. Motus introduces a Mixture-of-Transformer (MoT) architecture to integrate three experts (i.e., understanding, video generation, and action) and adopts a UniDiffuser-style scheduler to enable flexible switching between different modeling modes (i.e., world models, vision-language-action models, inverse dynamics models, video generation models, and video-action joint prediction models). Motus further leverages the optical flow to learn latent actions and adopts a recipe with three-phase training pipeline and six-layer data pyramid, thereby extracting pixel-level "delta action" and enabling large-scale action pretraining. Experiments show that Motus achieves superior performance against state-of-the-art methods in both simulation (a +15% improvement over X-VLA and a +45% improvement over Pi0.5) and real-world scenarios(improved by +11~48%), demonstrating unified modeling of all functionalities and priors significantly benefits downstream robotic tasks.
Abstract (translated)
虽然通用具身代理必须作为一个统一系统运行，但目前的方法是基于用于理解、世界建模和控制的孤立模型构建的。这种碎片化阻碍了多模态生成能力的一体化，并且妨碍从大规模异构数据中学习。在本文中，我们提出了Motus，这是一种统一的动作潜变量世界模型，它利用现有的通用预训练模型和丰富的可共享运动信息。Motus引入了一种Transformer混合架构（MoT），以整合三个专家领域（即理解、视频生成和动作）并采用类似UniDiffuser的调度器来实现在不同建模模式之间的灵活切换（例如：世界模型、视觉-语言-行动模型、逆动力学模型、视频生成模型以及视频-行动联合预测模型）。此外，Motus利用光学流学习潜变量动作，并采用了包含三个阶段训练流程和六层数据金字塔的配方方法，从而提取像素级别的“delta动作”并实现大规模的动作预训练。实验表明，在模拟环境（相对于X-VLA性能提高15%，相较于Pi0.5则提高了45%）以及真实世界场景中（性能提升范围为+11至+48%），Motus的表现优于最先进的方法，证明了一体化模型能够统一所有功能和先验知识，显著地有利于下游的机器人任务。
URL
https://arxiv.org/abs/2512.13030
