Abstract
To what extent do vision-and-language foundation models possess a realistic world model (observation $\times$ action $\rightarrow$ observation) and a dynamics model (observation $\times$ observation $\rightarrow$ action), when actions are expressed through language? While open-source foundation models struggle with both, we find that fine-tuning them to acquire a dynamics model through supervision is significantly easier than acquiring a world model. In turn, dynamics models can be used to bootstrap world models through two main strategies: 1) weakly supervised learning from synthetic data and 2) inference time verification. Firstly, the dynamics model can annotate actions for unlabelled pairs of video frame observations to expand the training data. We further propose a new objective, where image tokens in observation pairs are weighted by their importance, as predicted by a recognition model. Secondly, the dynamics models can assign rewards to multiple samples of the world model to score them, effectively guiding search at inference time. We evaluate the world models resulting from both strategies through the task of action-centric image editing on Aurora-Bench. Our best model achieves a performance competitive with state-of-the-art image editing models, improving on them by a margin of $15\%$ on real-world subsets according to GPT4o-as-judge, and achieving the best average human evaluation across all subsets of Aurora-Bench.
Abstract (translated)
当动作通过语言表达时，视觉和语言基础模型在多大程度上具备现实世界的模型（观察 $\times$ 动作 $\rightarrow$ 观察）和动态模型（观察 $\times$ 观察 $\rightarrow$ 动作）？开源基础模型在这两方面都存在困难，但我们发现通过监督来微调它们以获取动态模型比获取世界模型要容易得多。反过来，可以通过两种主要策略利用动态模型来启动世界模型：1）从合成数据进行弱监督学习；2）在推理时间验证。首先，动态模型可以为未标记的视频帧观察对分配动作标签，以此扩展训练数据集。此外，我们还提出了一种新目标，在此目标中，通过识别模型预测的重要性加权观测配对中的图像令牌。其次，动态模型还可以将奖励分配给世界模型的多个样本进行评分，从而有效指导推理时间搜索。 我们在Aurora-Bench上通过对以行动为中心的图像编辑任务来评估两种策略产生的世界模型性能。我们最好的模型在与最先进的图像编辑模型的竞争中表现出具有竞争力的表现，在GPT4o-as-judge根据现实世界的子集评估时提高了15%，并且在整个Aurora-Bench所有子集中实现了最佳的人均评价结果。
URL
https://arxiv.org/abs/2506.06006
