Abstract
Weakly supervised multimodal video anomaly detection has gained significant attention, yet the potential of the text modality remains under-explored. Text provides explicit semantic information that can enhance anomaly characterization and reduce false alarms. However, extracting effective text features is challenging due to the inability of general-purpose language models to capture anomaly-specific nuances and the scarcity of relevant descriptions. Furthermore, multimodal fusion often suffers from redundancy and imbalance. To address these issues, we propose a novel text-guided framework. First, we introduce an in-context learning-based multi-stage text augmentation mechanism to generate high-quality anomaly text samples for fine-tuning the text feature extractor. Second, we design a multi-scale bottleneck Transformer fusion module that uses compressed bottleneck tokens to progressively integrate information across modalities, mitigating redundancy and imbalance. Experiments on UCF-Crime and XD-Violence demonstrate state-of-the-art performance.
Abstract (translated)
弱监督多模态视频异常检测已经引起了广泛关注，然而文本模态的潜力仍然未被充分探索。文本提供了明确的语义信息，可以增强对异常行为的描述和减少误报率。但是，由于通用语言模型难以捕捉特定于异常的信息以及相关描述稀少的问题，提取有效文本特征颇具挑战性。此外，在多模态融合过程中经常会出现冗余和不平衡的问题。为了解决这些问题，我们提出了一种新颖的基于文本引导框架的方法。 首先，我们引入了一个基于上下文学习的多阶段文本增强机制，用于生成高质量的异常文本样本以对文本特征提取器进行微调。其次，我们设计了一种多尺度瓶颈Transformer融合模块，该模块使用压缩后的瓶颈令牌逐步整合不同模态的信息，从而缓解冗余和不平衡的问题。 在UCF-Crime和XD-Violence数据集上的实验结果表明了我们的方法达到了最先进的性能水平。
URL
https://arxiv.org/abs/2602.10549
