Abstract
In this paper, we address the challenging problem of single-scene, fully unsupervised video anomaly detection (VAD), where raw videos containing both normal and abnormal events are used directly for training and testing without any labels. This differs sharply from prior work that either requires extensive labeling (fully or weakly supervised) or depends on normal-only videos (one-class classification), which are vulnerable to distribution shifts and contamination. We propose an entropy-guided autoencoder that detects anomalies through reconstruction error by reconstructing normal frames well while making anomalies reconstruct poorly. The key idea is to combine the standard reconstruction loss with a novel Minimal Latent Entropy (MLE) loss in the autoencoder. Reconstruction loss alone maps normal and abnormal inputs to distinct latent clusters due to their inherent differences, but also risks reconstructing anomalies too well to detect. Therefore, MLE loss addresses this by minimizing the entropy of latent embeddings, encouraging them to concentrate around high-density regions. Since normal frames dominate the raw video, sparse anomalous embeddings are pulled into the normal cluster, so the decoder emphasizes normal patterns and produces poor reconstructions for anomalies. This dual-loss design produces a clear reconstruction gap that enables effective anomaly detection. Extensive experiments on two widely used benchmarks and a challenging self-collected driving dataset demonstrate that our method achieves robust and superior performance over baselines.
Abstract (translated)
本文针对单场景全无监督视频异常检测这一挑战性问题，即直接使用包含正常与异常事件的原始视频进行训练和测试而无需任何标签。这与以往需要大量标注（全监督或弱监督）或仅依赖正常视频（单类分类）的工作形成鲜明对比，后者易受分布偏移和污染影响。我们提出一种基于熵引导的自编码器，通过重建误差检测异常——使正常帧重建良好而异常帧重建较差。核心思想是在自编码器中结合标准重建损失与新颖的最小潜在熵（MLE）损失。仅靠重建损失虽能因正常与异常输入的固有差异将其映射到不同的潜在簇，但也可能过度优化异常重建而难以检测。因此，MLE损失通过最小化潜在嵌入的熵来解决此问题，促使嵌入向高密度区域集中。由于正常帧在原始视频中占主导，稀疏的异常嵌入会被拉入正常簇，从而使解码器聚焦正常模式并对异常产生较差重建。这种双损失设计形成了清晰的重建差距，可实现有效异常检测。在两个广泛使用的基准数据集及具有挑战性的自采驾驶数据集上的大量实验表明，我们的方法相比基线具有稳健且卓越的性能。
URL
https://arxiv.org/abs/2603.23868
