Abstract
Recently, face swapping has been developing rapidly and achieved a surprising reality, raising concerns about fake content. As a countermeasure, various detection approaches have been proposed and achieved promising performance. However, most existing detectors struggle to maintain performance on unseen face swapping methods and low-quality images. Apart from the generalization problem, current detection approaches have been shown vulnerable to evasion attacks crafted by detection-aware manipulators. Lack of robustness under adversary scenarios leaves threats for applying face swapping detection in real world. In this paper, we propose a novel face swapping detection approach based on face identification probability distributions, coined as IdP_FSD, to improve the generalization and robustness. IdP_FSD is specially designed for detecting swapped faces whose identities belong to a finite set, which is meaningful in real-world applications. Compared with previous general detection methods, we make use of the available real faces with concerned identities and require no fake samples for training. IdP_FSD exploits face swapping's common nature that the identity of swapped face combines that of two faces involved in swapping. We reflect this nature with the confusion of a face identification model and measure the confusion with the maximum value of the output probability distribution. What's more, to defend our detector under adversary scenarios, an attention-based finetuning scheme is proposed for the face identification models used in IdP_FSD. Extensive experiments show that the proposed IdP_FSD not only achieves high detection performance on different benchmark datasets and image qualities but also raises the bar for manipulators to evade the detection.
Abstract (translated)
近年来，人脸交换技术正在快速发展并实现了令人惊讶的现状，引起了关于虚假内容的忧虑。作为一种应对措施，已经提出了多种检测方法，并取得了令人瞩目的表现。然而，大多数现有检测方法都在未展示的人脸交换方法和低质量图像上 struggle 维持表现。除了泛化问题，当前检测方法还受到了检测意识操纵者精心制作的规避攻击的脆弱性。在对抗性场景下缺乏鲁棒性，使在现实世界应用人脸交换检测的威胁仍然存在。在本文中，我们提出了一种基于人脸身份识别概率分布的新人脸交换检测方法，称为 idP_FSD，以改善泛化和鲁棒性。 idP_FSD 专门设计用于检测身份属于有限集合的swapped faces，这在现实世界应用中有意义。与以前的通用检测方法相比，我们利用已知的真实 faces 和相关的身份信息，不需要假样本进行训练。 idP_FSD 利用人脸交换的普遍特性，即swapped face 的身份与在swapping过程中涉及的两个 Face 的身份相结合。我们通过人脸身份识别模型的混淆来反映这种特性，并使用输出概率分布的最大值来衡量混淆。此外，为了保护我们在对抗性场景下的检测器，我们提出了一种基于注意力的微调方案，用于 idP_FSD 中使用的人脸身份识别模型。广泛的实验结果表明，提出的 idP_FSD 不仅可以在不同类型的基准数据集和图像质量上实现高水平的检测性能，而且还提高了操纵者规避检测的难度。
URL
https://arxiv.org/abs/2303.13131
