Abstract
Increasing scene-awareness is a key challenge in video anomaly detection (VAD). In this work, we propose a hierarchical semantic contrast (HSC) method to learn a scene-aware VAD model from normal videos. We first incorporate foreground object and background scene features with high-level semantics by taking advantage of pre-trained video parsing models. Then, building upon the autoencoder-based reconstruction framework, we introduce both scene-level and object-level contrastive learning to enforce the encoded latent features to be compact within the same semantic classes while being separable across different classes. This hierarchical semantic contrast strategy helps to deal with the diversity of normal patterns and also increases their discrimination ability. Moreover, for the sake of tackling rare normal activities, we design a skeleton-based motion augmentation to increase samples and refine the model further. Extensive experiments on three public datasets and scene-dependent mixture datasets validate the effectiveness of our proposed method.
Abstract (translated)
增加场景意识是视频异常检测(VAD)中的一个关键挑战。在本研究中，我们提出了一种Hierarchical Semantic Contrast(HSC)方法，以从正常视频中提取场景意识的目标VAD模型。我们首先利用训练好的视频解析模型利用高级别语义信息融合前景对象和背景场景特征。然后，基于自编码器的重建框架，我们引入了场景和对象级别的对比学习，以强制相同的语义类别中的编码隐状态特征紧凑，但可以在不同类别之间分离。这种Hierarchical Semantic Contrast策略有助于处理正常模式的多样性，并增加它们的区分能力。此外，为了应对罕见的正常活动，我们设计了一种基于骨架的运动增强方法，以增加样本并进一步改进模型。对三个公共数据集和场景相关的混合数据集进行了广泛的实验，验证了我们提出的方法的有效性。
URL
https://arxiv.org/abs/2303.13051
