Abstract
Video Anomaly Understanding (VAU) extends traditional Video Anomaly Detection (VAD) by not only localizing anomalies but also describing and reasoning about their context. Existing VAU approaches often rely on fine-tuned multimodal large language models (MLLMs) or external modules such as video captioners, which introduce costly annotations, complex training pipelines, and high inference overhead. In this work, we introduce PrismVAU, a lightweight yet effective system for real-time VAU that leverages a single off-the-shelf MLLM for anomaly scoring, explanation, and prompt optimization. PrismVAU operates in two complementary stages: (1) a coarse anomaly scoring module that computes frame-level anomaly scores via similarity to textual anchors, and (2) an MLLM-based refinement module that contextualizes anomalies through system and user prompts. Both textual anchors and prompts are optimized with a weakly supervised Automatic Prompt Engineering (APE) framework. Extensive experiments on standard VAD benchmarks demonstrate that PrismVAU delivers competitive detection performance and interpretable anomaly explanations -- without relying on instruction tuning, frame-level annotations, and external modules or dense processing -- making it an efficient and practical solution for real-world applications.
Abstract (translated)
视频异常理解（Video Anomaly Understanding，简称VAU）在传统的视频异常检测（Video Anomaly Detection，简称VAD）基础上更进一步，不仅定位了异常事件，还描述并解释了这些异常的上下文。现有的VAU方法往往依赖于微调过的多模态大规模语言模型（Multimodal Large Language Models，简称MLLMs）或外部模块如视频字幕生成器，这引入了昂贵的数据标注、复杂的训练流程和高推理成本。 在此项工作中，我们提出了PrismVAU，这是一个轻量级且高效的系统，适用于实时的VAU任务。该系统利用单一现成的多模态大规模语言模型（MLLM）来完成异常评分、解释以及提示优化。PrismVAU通过两个互补阶段实现其功能：(1) 一个粗略的异常评分模块，它计算基于与文本锚点相似度的帧级异常分数；(2) 一个基于多模态大规模语言模型（MLLM）的细化模块，该模块通过系统和用户提示来上下文化异常。这些文本锚点和提示都经过弱监督自动提示工程（Automatic Prompt Engineering, APE）框架进行优化。 在标准VAD基准上的广泛实验表明，PrismVAU能够提供竞争力强的检测性能以及可解释性的异常解释——并且无需依赖指令调优、帧级标注或外部模块及密集处理流程。这使得PrismVAU成为实际应用中一个高效且实用的解决方案。
URL
https://arxiv.org/abs/2601.02927
