Abstract
With the popularity of social media platforms such as Instagram and TikTok, and the widespread availability and convenience of retouching tools, an increasing number of individuals are utilizing these tools to beautify their facial photographs. This poses challenges for fields that place high demands on the authenticity of photographs, such as identity verification and social media. By altering facial images, users can easily create deceptive images, leading to the dissemination of false information. This may pose challenges to the reliability of identity verification systems and social media, and even lead to online fraud. To address this issue, some work has proposed makeup removal methods, but they still lack the ability to restore images involving geometric deformations caused by retouching. To tackle the problem of facial retouching restoration, we propose a framework, dubbed Face2Face, which consists of three components: a facial retouching detector, an image restoration model named FaceR, and a color correction module called Hierarchical Adaptive Instance Normalization (H-AdaIN). Firstly, the facial retouching detector predicts a retouching label containing three integers, indicating the retouching methods and their corresponding degrees. Then FaceR restores the retouched image based on the predicted retouching label. Finally, H-AdaIN is applied to address the issue of color shift arising from diffusion models. Extensive experiments demonstrate the effectiveness of our framework and each module.
Abstract (translated)
随着社交媒体平台如Instagram和TikTok的流行，以及修图工具的广泛可用性和便利性，越来越多的人使用这些工具来美化他们的面部照片。这给对照片真实度有很高要求的领域（如身份验证和社交媒体）带来了挑战。通过改变面部图像，用户可以轻松创建误导性的图像，导致传播虚假信息。这可能对身份验证系统的可靠性和社交媒体造成挑战，甚至可能导致网络欺诈。为解决这个问题，一些工作提出了化妆去除方法，但他们仍然缺乏修复由修图引起的几何变形图像的能力。为解决面部修图恢复问题，我们提出了一个名为Face2Face的框架，它由三个组件组成：面部修图检测器、一个名为FaceR的图像修复模型和一个名为Hierarchical Adaptive Instance Normalization（H-AdaIN）的颜色校正模块。首先，面部修图检测器预测包含三个整数的修图标签，表示修图方法和它们的相应程度。然后，FaceR根据预测的修图标签恢复被修复的图像。最后，H-AdaIN应用于解决扩散模型引起的颜色偏移问题。大量实验证明了我们框架和每个模块的有效性。
URL
https://arxiv.org/abs/2404.14177
