Abstract
Near-range portrait photographs often contain perspective distortion artifacts that bias human perception and challenge both facial recognition and reconstruction techniques. We present the first deep learning based approach to remove such artifacts from unconstrained portraits. In contrast to the previous state-of-the-art approach, our method handles even portraits with extreme perspective distortion, as we avoid the inaccurate and error-prone step of first fitting a 3D face model. Instead, we predict a distortion correction flow map that encodes a per-pixel displacement that removes distortion artifacts when applied to the input image. Our method also automatically infers missing facial features, i.e. occluded ears caused by strong perspective distortion, with coherent details. We demonstrate that our approach significantly outperforms the previous state-of-the-art both qualitatively and quantitatively, particularly for portraits with extreme perspective distortion or facial expressions. We further show that our technique benefits a number of fundamental tasks, significantly improving the accuracy of both face recognition and 3D reconstruction and enables a novel camera calibration technique from a single portrait. Moreover, we also build the first perspective portrait database with a large diversity in identities, expression and poses, which will benefit the related research in this area.
Abstract (translated)
近距离人像照片往往含有透视畸变伪影，这些伪影会使人的感知产生偏差，并对人脸识别和重建技术提出挑战。我们提出了第一个基于深度学习的方法，从无约束的肖像中移除这些伪影。与之前最先进的方法相比，我们的方法处理具有极端透视失真的人像，因为我们避免了首先拟合3D人脸模型的不准确和容易出错的步骤。相反，我们预测一个畸变校正流程图，它编码每像素的位移，当应用到输入图像时，消除畸变伪影。我们的方法还可以自动推断缺少的面部特征，即由强烈的透视扭曲造成的耳朵遮挡，细节连贯。我们证明，我们的方法在质量和数量上都明显优于先前的技术水平，特别是对于具有极端透视扭曲或面部表情的肖像。我们进一步表明，我们的技术有利于许多基本任务，显著提高了人脸识别和三维重建的准确性，并使一个新的相机校准技术从一幅肖像。此外，我们还建立了第一个身份、表情和姿势多样的透视人像数据库，这将有助于这一领域的相关研究。
URL
https://arxiv.org/abs/1905.07515
