Abstract
Video foundation models aim to integrate video understanding, generation, editing, and instruction following within a single framework, making them a central direction for next-generation multimodal systems. However, existing evaluation benchmarks remain fragmented and limited in scope, as they each target a single task, rely on task-specific metrics, and typically use short or simple video clips. As a result, they do not capture the unified capabilities that these models are designed to deliver. To address this gap, we introduce UniVBench, a benchmark purpose-built for evaluating video foundation models across four core abilities: video understanding, video generation, video editing, and a newly proposed task, video reconstruction, which assesses how faithfully a model can reproduce video content it has encountered. Our benchmark substantially expands the complexity of evaluation by incorporating 200 high-quality, diverse and multi-shot videos, each paired with detailed captions, multi-format editing instructions, and reference images. All videos are human-created and carefully validated, offering richer cinematic information than prior benchmarks. In addition, we develop a unified agentic evaluation system (UniV-Eval) that standardizes prompting, instruction parsing, and scoring across all tasks, enabling fair, scalable, and reproducible comparisons of unified video models. By grounding evaluation in instruction-based multi-shot video tasks, UniVBench provides the first framework for measuring the integrated capabilities that video foundation models aim to achieve. Extensive human annotations ensure our evaluation aligns with human judgment, enabling rigorous assessment and accelerating progress toward robust video intelligence.
Abstract (translated)
视频基础模型的目标是将视频理解、生成、编辑和指令执行整合到一个单一的框架中，使之成为下一代多模态系统的核心发展方向。然而，现有的评估基准仍然分散且范围有限，因为它们各自针对单个任务，依赖特定于该任务的指标，并通常使用短小或简单的视频片段。因此，这些基准无法捕捉模型设计所要实现的统一能力。 为了填补这一空白，我们引入了UniVBench，这是一个专门用于评价视频基础模型在四大核心能力方面的基准：视频理解、视频生成、视频编辑和一项新提出的任务——视频重建，这项任务评估的是模型再现其遇到过的视频内容的忠实度。我们的基准通过纳入200部高质量、多样化且包含多镜头的视频来大幅扩展了评估的复杂性，每部视频都配有一系列详细的字幕、多种格式的编辑指令和参考图像。所有视频均由人类创作并经过精心验证，提供了比以往基准更加丰富的电影信息。 此外，我们开发了一个统一代理评估系统（UniV-Eval），标准化了所有任务中的提示生成、指令解析和评分过程，从而实现了公平、可扩展且可重复的统一视频模型比较。通过基于指令多镜头视频任务进行评价，UniVBench首次提供了一种测量视频基础模型所追求的整体能力的方法。 广泛的人员注释确保我们的评估与人类判断一致，使严格的评估成为可能，并加快向稳健视频智能发展的步伐。
URL
https://arxiv.org/abs/2602.21835
