Abstract
The rapid progress of Large Language Models (LLMs) has spurred growing interest in Multi-modal LLMs (MLLMs) and motivated the development of benchmarks to evaluate their perceptual and comprehension abilities. Existing benchmarks, however, are limited to static images or single videos, overlooking the complex interactions across multiple videos. To address this gap, we introduce the Multi-Video Perception Evaluation Benchmark (MVPBench), a new benchmark featuring 14 subtasks across diverse visual domains designed to evaluate models on extracting relevant information from video sequences to make informed decisions. MVPBench includes 5K question-answering tests involving 2.7K video clips sourced from existing datasets and manually annotated clips. Extensive evaluations reveal that current models struggle to process multi-video inputs effectively, underscoring substantial limitations in their multi-video comprehension. We anticipate MVPBench will drive advancements in multi-video perception.
Abstract (translated)
大语言模型（LLMs）的快速发展激发了人们对多模态大模型（MLLMs）日益浓厚的兴趣，并推动了用于评估其感知与理解能力的基准测试的开发。然而，现有基准测试局限于静态图像或单一视频，忽略了跨多个视频的复杂交互。为弥补这一空白，我们推出了多视频感知评估基准（MVPBench），这是一个涵盖多样视觉领域14个子任务的新基准，旨在评估模型从视频序列中提取相关信息以做出明智决策的能力。MVPBench包含5K个问答测试，涉及2.7K个来自现有数据集及手动标注的视频片段。广泛评估表明，当前模型难以有效处理多视频输入，凸显了其多视频理解能力的重大局限。我们期待MVPBench能推动多视频感知领域的发展。
URL
https://arxiv.org/abs/2603.22756
