Abstract
Multimodal large language models (MLLMs) have demonstrated impressive general competence in video understanding, yet their reliability for real-world Video Anomaly Detection (VAD) remains largely unexplored. Unlike conventional pipelines relying on reconstruction or pose-based cues, MLLMs enable a paradigm shift: treating anomaly detection as a language-guided reasoning task. In this work, we systematically evaluate state-of-the-art MLLMs on the ShanghaiTech and CHAD benchmarks by reformulating VAD as a binary classification task under weak temporal supervision. We investigate how prompt specificity and temporal window lengths (1s--3s) influence performance, focusing on the precision--recall trade-off. Our findings reveal a pronounced conservative bias in zero-shot settings; while models exhibit high confidence, they disproportionately favor the 'normal' class, resulting in high precision but a recall collapse that limits practical utility. We demonstrate that class-specific instructions can significantly shift this decision boundary, improving the peak F1-score on ShanghaiTech from 0.09 to 0.64, yet recall remains a critical bottleneck. These results highlight a significant performance gap for MLLMs in noisy environments and provide a foundation for future work in recall-oriented prompting and model calibration for open-world surveillance, which demands complex video understanding and reasoning.
Abstract (translated)
多模态大型语言模型（MLLMs）在视频理解方面展示了令人印象深刻的通用能力，但它们在现实世界的视频异常检测（VAD）中的可靠性仍然鲜有研究。与传统的依赖于重建或基于姿势的线索的方法不同，MLLMs实现了范式转变：将异常检测视为一种由语言引导的推理任务。在这项工作中，我们通过重新定义VAD为一个带有弱时间监督的二分类任务，在ShanghaiTech和CHAD基准上系统地评估了最先进的MLLMs。我们探讨了提示具体性和不同时间窗口长度（1秒至3秒）对性能的影响，并重点关注精度与召回率之间的权衡。我们的发现揭示了一个明显的保守倾向：在零样本设置下，尽管模型表现出高度自信，但它们倾向于过多地偏向“正常”类别，从而导致高精度但低召回率的问题，限制了其实际应用价值。我们展示了针对特定类别的指令可以显著改变这种决策边界，在ShanghaiTech上的峰值F1分数从0.09提升到了0.64，尽管如此，召回仍然是一个关键瓶颈。这些结果突显了MLLMs在嘈杂环境中的性能差距，并为未来的研究提供了基础：通过侧重于提高召回率的提示设计和模型校准，在开放世界的监控领域中实现复杂视频理解和推理的需求。
URL
https://arxiv.org/abs/2603.04727
