Abstract
We introduce World Consistency Score (WCS), a novel unified evaluation metric for generative video models that emphasizes internal world consistency of the generated videos. WCS integrates four interpretable sub-components - object permanence, relation stability, causal compliance, and flicker penalty - each measuring a distinct aspect of temporal and physical coherence in a video. These submetrics are combined via a learned weighted formula to produce a single consistency score that aligns with human judgments. We detail the motivation for WCS in the context of existing video evaluation metrics, formalize each submetric and how it is computed with open-source tools (trackers, action recognizers, CLIP embeddings, optical flow), and describe how the weights of the WCS combination are trained using human preference data. We also outline an experimental validation blueprint: using benchmarks like VBench-2.0, EvalCrafter, and LOVE to test WCS's correlation with human evaluations, performing sensitivity analyses, and comparing WCS against established metrics (FVD, CLIPScore, VBench, FVMD). The proposed WCS offers a comprehensive and interpretable framework for evaluating video generation models on their ability to maintain a coherent "world" over time, addressing gaps left by prior metrics focused only on visual fidelity or prompt alignment.
Abstract (translated)
我们介绍了一种新颖的统一评估指标——世界一致性得分（World Consistency Score，WCS），用于评价生成视频模型。该指标特别强调了所生成视频内部的世界一致性。WCS集成了四个可解释的子组件：物体恒常性、关系稳定性、因果合规性和闪烁惩罚，每个子组件分别衡量视频在时间和物理连贯性方面的不同方面。这些子度量通过一个基于学习权重公式组合起来，产生一个单一的一致性得分，并且该得分能够与人类判断保持一致。 我们详细阐述了WCS相对于现有视频评估指标的动机、形式化定义每个子度量以及如何使用开源工具（跟踪器、动作识别器、CLIP嵌入、光流）进行计算。此外，还描述了如何通过利用人类偏好数据来训练WCS组合权重。 我们也概述了一个实验验证蓝图：使用VBench-2.0、EvalCrafter和LOVE等基准测试来检验WCS与人类评价的相关性；执行敏感性分析，并将WCS与其他已建立的度量（如FVD、CLIPScore、VBench、FVMD）进行比较。 提出的WCS提供了一个全面且可解释的框架，用于评估视频生成模型在其维持随时间变化的一致“世界”方面的能力。这弥补了之前仅关注视觉保真度或指令对齐的指标所留下的不足之处。
URL
https://arxiv.org/abs/2508.00144
