Abstract
Recovering a photorealistic face from an artistic portrait is a challenging task since crucial facial details are often distorted or completely lost in artistic compositions. To handle this loss, we propose an Attribute-guided Face Recovery from Portraits (AFRP) that utilizes a Face Recovery Network (FRN) and a Discriminative Network (DN). FRN consists of an autoencoder with residual block-embedded skip-connections and incorporates facial attribute vectors into the feature maps of input portraits at the bottleneck of the autoencoder. DN has multiple convolutional and fully-connected layers, and its role is to enforce FRN to generate authentic face images with corresponding facial attributes dictated by the input attribute vectors. %Leveraging on the spatial transformer networks, FRN automatically compensates for misalignments of portraits. % and generates aligned face images. For the preservation of identities, we impose the recovered and ground-truth faces to share similar visual features. Specifically, DN determines whether the recovered image looks like a real face and checks if the facial attributes extracted from the recovered image are consistent with given attributes. %Our method can recover high-quality photorealistic faces from unaligned portraits while preserving the identity of the face images as well as it can reconstruct a photorealistic face image with a desired set of attributes. Our method can recover photorealistic identity-preserving faces with desired attributes from unseen stylized portraits, artistic paintings, and hand-drawn sketches. On large-scale synthesized and sketch datasets, we demonstrate that our face recovery method achieves state-of-the-art results.
Abstract (translated)
从艺术肖像中恢复照片真实的面部是一项具有挑战性的任务，因为重要的面部细节经常被扭曲或完全丢失在艺术作品中。为了解决这一问题，我们提出了一种基于属性引导的人像人脸恢复方法（AFRP），该方法利用人脸恢复网络（FRN）和识别网络（DN）。FRN由一个带有剩余块嵌入跳过连接的自动编码器组成，并在自动编码器瓶颈处将面部属性向量合并到输入肖像的特征图中。dn具有多个卷积层和全连接层，其作用是强制frn生成真实的人脸图像，其相应的人脸属性由输入属性向量决定。%利用空间变换网络，FRN自动补偿人像的错位。%并生成对齐的面图像。为了保存身份，我们将恢复的和基本的真面相强加给共享相似的视觉特征。具体来说，dn确定恢复的图像是否看起来像真实的脸，并检查从恢复的图像中提取的面部属性是否与给定的属性一致。%我们的方法可以从未对齐的人像中恢复高质量的真实感人脸，同时保留人脸图像的身份，还可以用所需的一组属性重建真实感人脸图像。我们的方法可以从看不见的风格化肖像、艺术绘画和手绘草图中恢复具有所需属性的照片真实身份保护面。在大规模合成和草图数据集上，我们证明了我们的人脸恢复方法达到了最先进的结果。
URL
https://arxiv.org/abs/1904.03612
