Abstract
Facial Aesthetics Enhancement (FAE) aims to improve facial attractiveness by adjusting the structure and appearance of a facial image while preserving its identity as much as possible. Most existing methods adopted deep feature-based or score-based guidance for generation models to conduct FAE. Although these methods achieved promising results, they potentially produced excessively beautified results with lower identity consistency or insufficiently improved facial attractiveness. To enhance facial aesthetics with less loss of identity, we propose the Nearest Neighbor Structure Guidance based on Diffusion (NNSG-Diffusion), a diffusion-based FAE method that beautifies a 2D facial image with 3D structure guidance. Specifically, we propose to extract FAE guidance from a nearest neighbor reference face. To allow for less change of facial structures in the FAE process, a 3D face model is recovered by referring to both the matched 2D reference face and the 2D input face, so that the depth and contour guidance can be extracted from the 3D face model. Then the depth and contour clues can provide effective guidance to Stable Diffusion with ControlNet for FAE. Extensive experiments demonstrate that our method is superior to previous relevant methods in enhancing facial aesthetics while preserving facial identity.
Abstract (translated)
面部美学增强（FAE）的目标是通过调整面部图像的结构和外观来提高面部吸引力，同时尽可能地保留其身份特征。现有的大多数方法采用基于深度特征或评分的引导方式，用于生成模型进行FAE处理。尽管这些方法取得了令人满意的结果，但它们可能会产生过度美化的效果，导致身份一致性降低，或者对脸部美感的改善不足。为了在较少损失面部身份的情况下增强面部美学，我们提出了基于扩散的最近邻结构指导（NNSG-Diffusion）方法，这是一种利用3D结构引导来美化2D面部图像的扩散基础FAE方法。 具体而言，我们提出从最近的邻居参考脸中提取FAE指南。为了在FAE过程中减少面部结构的变化，通过参考匹配的2D参考脸和输入的2D人脸恢复出一个3D脸部模型，这样可以从该3D脸部模型中提取深度和轮廓指导信息。然后，这些深度和轮廓线索可以为Stable Diffusion结合ControlNet提供有效的指导，以进行FAE处理。 大量的实验表明，我们的方法在增强面部美学的同时保留面部身份方面优于以前的相关方法。
URL
https://arxiv.org/abs/2503.14402
