Abstract
Given an artistic portrait, recovering the latent photorealistic face that preserves the subject's identity is challenging because the facial details are often distorted or fully lost in artistic portraits. We develop an Identity-preserving Face Recovery from Portraits (IFRP) method that utilizes a Style Removal network (SRN) and a Discriminative Network (DN). Our SRN, composed of an autoencoder with residual block-embedded skip connections, is designed to transfer feature maps of stylized images to the feature maps of the corresponding photorealistic faces. Owing to the Spatial Transformer Network (STN), SRN automatically compensates for misalignments of stylized portraits to output aligned realistic face images. To ensure the identity preservation, we promote the recovered and ground truth faces to share similar visual features via a distance measure which compares features of recovered and ground truth faces extracted from a pre-trained FaceNet network. DN has multiple convolutional and fully-connected layers, and its role is to enforce recovered faces to be similar to authentic faces. Thus, we can recover high-quality photorealistic faces from unaligned portraits while preserving the identity of the face in an image. By conducting extensive evaluations on a large-scale synthesized dataset and a hand-drawn sketch dataset, we demonstrate that our method achieves superior face recovery and attains state-of-the-art results. In addition, our method can recover photorealistic faces from unseen stylized portraits, artistic paintings, and hand-drawn sketches.
Abstract (translated)
对于一幅艺术肖像，恢复隐藏的真实照片的面部，以保持主体的身份是具有挑战性的，因为面部细节往往在艺术肖像中被扭曲或完全丢失。我们开发了一种保留身份的人像人脸恢复（IFRP）方法，该方法利用了样式删除网络（SRN）和识别网络（DN）。我们的SRN由一个带有剩余块嵌入跳跃连接的自动编码器组成，旨在将风格化图像的特征图传输到相应的真实照片面的特征图。由于空间变换网络（STN），SRN自动补偿了风格化人像的错位，从而输出对齐的真实人脸图像。为了确保身份保护，我们通过距离测量，将从预先训练的facenet网络中提取的恢复和地面真面相比较，促进恢复和地面真面相共享类似的视觉特征。DN具有多个卷积和完全连接的层，其作用是强制恢复的面与真实面相似。因此，我们可以从未对齐的人像中恢复高质量的真实感人脸，同时保留图像中人脸的身份。通过对大规模合成数据集和手绘草图数据集进行广泛的评估，我们证明了我们的方法可以实现出色的面部恢复，并获得最先进的结果。此外，我们的方法可以从看不见的风格化的肖像、艺术画和手绘草图中恢复照片真实的面孔。
URL
https://arxiv.org/abs/1904.04241
