Abstract
Recent video multimodal large language models achieve impressive results across various benchmarks. However, current evaluations suffer from two critical limitations: (1) inflated scores can mask deficiencies in fine-grained visual understanding and reasoning, and (2) answer correctness is often measured without verifying whether models identify the precise spatio-temporal evidence supporting their predictions. To address this, we present VideoZeroBench, a hierarchical benchmark designed for challenging long-video question answering that rigorously verifies spatio-temporal evidence. It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence. To disentangle answering generation, temporal grounding, and spatial grounding, we introduce a five-level evaluation protocol that progressively tightens evidence requirements. Experiments show that even Gemini-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting (Level-3). When grounding constraints are imposed, performance drops sharply: No model exceeds 1% accuracy when both correct answering and accurate spatio-temporal localization are required (Level-5), with most failing to achieve any correct grounded predictions. These results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning, revealing that grounded video understanding remains a bottleneck for long-video QA. We further analyze performance across minimal evidence spans, atomic abilities, and inference paradigms, providing insights for future research in grounded video reasoning. The benchmark and code will be made publicly available.
Abstract (translated)
近期视频多模态大语言模型在各种基准测试中取得了令人瞩目的成果。然而，当前评估存在两个关键局限：(1) 虚高的分数可能掩盖模型在细粒度视觉理解与推理方面的缺陷；(2) 答案正确性的判定往往未验证模型是否识别出支撑其预测的精确时空证据。为解决此问题，我们提出了 VideoZeroBench——一个面向挑战性长视频问答的层级化基准，可严格验证时空证据。该基准包含13个领域中500个手工标注的问题，并配以时间区间和空间边界框作为证据。为解耦答案生成、时序定位与空间定位，我们设计了五级评估协议，逐步收紧证据要求。实验表明，即使在标准端到端问答设置（Level-3）下，Gemini-3-Pro 正确回答的问题比例仍不足17%。当施加定位约束时，性能急剧下降：在同时要求答案正确与时空定位准确（Level-5）的条件下，所有模型准确率均未超过1%，多数模型甚至无法产生任何正确的定位预测。这些结果揭示了表面答案正确性与真实基于证据的推理之间存在显著差距，表明基于定位的视频理解仍是长视频问答的瓶颈。我们还针对最小证据跨度、原子能力及推理范式进行了性能分析，为未来基于定位的视频推理研究提供见解。该基准及代码将公开发布。
URL
https://arxiv.org/abs/2604.01569
