Abstract
As a significant step for human face modeling, editing, and generation, face landmarking aims at extracting facial keypoints from images. A generalizable face landmarker is required in practice because real-world facial images, e.g., the avatars in animations and games, are often stylized in various ways. However, achieving generalizable face landmarking is challenging due to the diversity of facial styles and the scarcity of labeled stylized faces. In this study, we propose a simple but effective paradigm to learn a generalizable face landmarker based on labeled real human faces and unlabeled stylized faces. Our method learns the face landmarker as the key module of a conditional face warper. Given a pair of real and stylized facial images, the conditional face warper predicts a warping field from the real face to the stylized one, in which the face landmarker predicts the ending points of the warping field and provides us with high-quality pseudo landmarks for the corresponding stylized facial images. Applying an alternating optimization strategy, we learn the face landmarker to minimize $i)$ the discrepancy between the stylized faces and the warped real ones and $ii)$ the prediction errors of both real and pseudo landmarks. Experiments on various datasets show that our method outperforms existing state-of-the-art domain adaptation methods in face landmarking tasks, leading to a face landmarker with better generalizability. Code is available at this https URL}{this https URL.
Abstract (translated)
作为人脸建模、编辑和生成的显著一步，目标是从图像中提取面部关键点。在实践中，需要一个通用的面部关键点检测器，因为现实世界的人脸图像，例如动画和游戏中的人物，通常以各种方式进行扭曲。然而，实现通用的面部关键点检测器具有挑战性，因为人脸风格的多样性以及标注有风格的人脸的稀缺性。在这项研究中，我们提出了一个简单但有效的范例，基于标注的实际人脸和未标注的有风格的人脸学习一个通用的面部关键点检测器。我们的方法将面部关键点检测器作为一个条件式人脸扭曲的模块学习。给定一对真实和有风格的人脸图像，条件式人脸扭曲预测从真实人脸到有风格人脸的扭曲场，面部关键点检测器预测扭曲场的终点，并提供我们高质量的伪关键点，对应的有风格的人脸图像。采用交替优化策略，我们学习面部关键点检测器最小化$i)$建模轮廓与扭曲真实图像之间的差异和$ii)$同时预测真实和伪关键点的误差。在各种数据集上的实验表明，我们的方法在面部关键点任务上优于现有的领域自适应方法，导致具有更好泛化能力的面部关键点。代码可在此处下载：https://this.url
URL
https://arxiv.org/abs/2404.12322
