Abstract
With the development of deep learning technology, the facial manipulation system has become powerful and easy to use. Such systems can modify the attributes of the given facial images, such as hair color, gender, and age. Malicious applications of such systems pose a serious threat to individuals' privacy and reputation. Existing studies have proposed various approaches to protect images against facial manipulations. Passive defense methods aim to detect whether the face is real or fake, which works for posterior forensics but can not prevent malicious manipulation. Initiative defense methods protect images upfront by injecting adversarial perturbations into images to disrupt facial manipulation systems but can not identify whether the image is fake. To address the limitation of existing methods, we propose a novel two-tier protection method named Information-containing Adversarial Perturbation (IAP), which provides more comprehensive protection for {facial images}. We use an encoder to map a facial image and its identity message to a cross-model adversarial example which can disrupt multiple facial manipulation systems to achieve initiative protection. Recovering the message in adversarial examples with a decoder serves passive protection, contributing to provenance tracking and fake image detection. We introduce a feature-level correlation measurement that is more suitable to measure the difference between the facial images than the commonly used mean squared error. Moreover, we propose a spectral diffusion method to spread messages to different frequency channels, thereby improving the robustness of the message against facial manipulation. Extensive experimental results demonstrate that our proposed IAP can recover the messages from the adversarial examples with high average accuracy and effectively disrupt the facial manipulation systems.
Abstract (translated)
随着深度学习技术的发展,面部操纵系统变得强大且易于使用。这些系统可以修改给定面部图像的属性,如发色、性别和年龄。恶意使用这些系统对个人隐私和声誉构成了严重的威胁。现有研究已经提出了多种方法来保护图像免受面部操纵。被动防御方法旨在检测面部是否真实或伪造,这种方法适用于后法医学,但不能防止恶意操纵。主动防御方法通过注入对抗性干扰来破坏多个面部操纵系统,但无法识别图像是否伪造。为了应对现有方法的局限性,我们提出了一种名为“包含信息对抗干扰”的新两级保护方法,该方法为面部图像提供了更加全面的保护。我们使用编码器将面部图像及其身份消息映射到跨模型的对抗示例中,该示例可以破坏多个面部操纵系统以实现主动防御。通过解码器恢复对抗示例中的信息提供被动保护,有助于追踪来源和检测假图像。我们提出了一种特征级协方差测量方法,比常用的平方误差测量方法更适合测量面部图像之间的差异。此外,我们提出了一种谱扩散方法将信息传播到不同的频率通道中,从而提高了面部操纵系统对信息的可靠性。广泛的实验结果表明,我们提出的IAP可以以高平均准确性从对抗示例中恢复信息,并有效地破坏了面部操纵系统。
URL
https://arxiv.org/abs/2303.11625
