Abstract
We introduce VIBA, a novel approach for explainable video classification by adapting Information Bottlenecks for Attribution (IBA) to video sequences. While most traditional explainability methods are designed for image models, our IBA framework addresses the need for explainability in temporal models used for video analysis. To demonstrate its effectiveness, we apply VIBA to video deepfake detection, testing it on two architectures: the Xception model for spatial features and a VGG11-based model for capturing motion dynamics through optical flow. Using a custom dataset that reflects recent deepfake generation techniques, we adapt IBA to create relevance and optical flow maps, visually highlighting manipulated regions and motion inconsistencies. Our results show that VIBA generates temporally and spatially consistent explanations, which align closely with human annotations, thus providing interpretability for video classification and particularly for deepfake detection.
Abstract (translated)
我们介绍了一种新的方法VIBA，这是一种通过将信息瓶颈归因法（Information Bottlenecks for Attribution，简称IBA）应用于视频序列来实现可解释性视频分类的方法。大多数传统的可解释性方法是为图像模型设计的，而我们的IBA框架则解决了在用于视频分析的时间模型中需要可解释性的问题。为了证明其有效性，我们将VIBA应用于视频深度伪造检测，并针对两种架构进行了测试：一种是使用Xception模型来提取空间特征，另一种则是基于VGG11模型通过光学流捕捉运动动态的架构。 我们利用了一个定制的数据集来进行实验，该数据集反映了最新的深度伪造生成技术。在这一基础上，我们将IBA方法扩展到创建相关性和光学流图，以视觉方式突出显示被篡改的区域和运动不一致性。我们的结果显示，VIBA能够生成与人类标注高度一致的时间和空间上连贯的解释，从而为视频分类及其特定应用——深度伪造检测提供了可解释性。
URL
https://arxiv.org/abs/2501.16889
