Abstract
While videos can be falsified in many different ways, most existing forensic networks are specialized to detect only a single manipulation type (e.g. deepfake, inpainting). This poses a significant issue as the manipulation used to falsify a video is not known a priori. To address this problem, we propose MVFNet - a multipurpose video forensics network capable of detecting multiple types of manipulations including inpainting, deepfakes, splicing, and editing. Our network does this by extracting and jointly analyzing a broad set of forensic feature modalities that capture both spatial and temporal anomalies in falsified videos. To reliably detect and localize fake content of all shapes and sizes, our network employs a novel Multi-Scale Hierarchical Transformer module to identify forensic inconsistencies across multiple spatial scales. Experimental results show that our network obtains state-of-the-art performance in general scenarios where multiple different manipulations are possible, and rivals specialized detectors in targeted scenarios.
Abstract (translated)
虽然视频可以通过多种方式被伪造，但现有的大多数取证网络仅专注于检测单一类型的操纵（例如深度伪造、修补）。这在实践中带来了重大问题，因为在验证视频的真实性时通常不知道是哪种方法进行了篡改。为了解决这个问题，我们提出了MVFNet——一种多功能的视频取证网络，能够检测包括修补、深度伪造、拼接和编辑在内的多种类型的操作。我们的网络通过提取并联合分析一套广泛的取证特征模态来实现这一目标，这些特征可以捕捉到被篡改视频中的空间和时间异常。 为了可靠地检测并定位所有形状和大小的虚假内容，我们的网络采用了一种创新的多尺度分层变换器模块，在多个空间尺度上识别取证上的不一致性。实验结果表明，我们的网络在多种不同类型的操纵都可能发生的一般场景中达到了最先进的性能，并且在针对特定类型伪造手段的场景中也能与专用检测器相媲美。
URL
https://arxiv.org/abs/2503.20991
