Abstract
Face recognition systems store face templates for efficient matching. Once leaked, these templates pose a threat: inverting them can yield photorealistic surrogates that compromise privacy and enable impersonation. Although existing research has achieved relatively realistic face template inversion, the reconstructed facial images exhibit over-smoothed facial-part attributes (eyes, nose, mouth) and limited transferability. To address this problem, we present CLIP-FTI, a CLIP-driven fine-grained attribute conditioning framework for face template inversion. Our core idea is to use the CLIP model to obtain the semantic embeddings of facial features, in order to realize the reconstruction of specific facial feature attributes. Specifically, facial feature attribute embeddings extracted from CLIP are fused with the leaked template via a cross-modal feature interaction network and projected into the intermediate latent space of a pretrained StyleGAN. The StyleGAN generator then synthesizes face images with the same identity as the templates but with more fine-grained facial feature attributes. Experiments across multiple face recognition backbones and datasets show that our reconstructions (i) achieve higher identification accuracy and attribute similarity, (ii) recover sharper component-level attribute semantics, and (iii) improve cross-model attack transferability compared to prior reconstruction attacks. To the best of our knowledge, ours is the first method to use additional information besides the face template attack to realize face template inversion and obtains SOTA results.
Abstract (translated)
面部识别系统存储人脸模板，以实现高效的匹配。一旦这些模板被泄露，它们会构成威胁：通过逆向工程可以生成逼真的面部替身，从而侵犯隐私并可能导致身份冒用。尽管现有的研究已经实现了相对逼真的面部模板逆向过程，但重建的面部图像展示出过度平滑的眼部、鼻子和嘴巴等面部特征属性，并且具有有限的跨模型迁移能力。 为了应对这个问题，我们提出了CLIP-FTI框架，这是一个由CLIP驱动的细粒度属性条件化框架，用于实现面部模板逆向过程。我们的核心理念是利用CLIP模型获取面部特征的语义嵌入，从而能够重构特定的面部特征属性。具体而言，从CLIP中提取出的面部特征属性嵌入通过跨模态特征交互网络与泄露的人脸模板融合，并投影到预先训练过的StyleGAN的中间潜在空间内。随后，StyleGAN生成器会合成具有与原始模板相同身份但拥有更精细的面部特征属性的脸部图像。 在多个面部识别骨干架构和数据集上的实验显示，我们的重建方法实现了以下改进：（i）更高的识别准确性和属性相似度；（ii）恢复了更为清晰、细节层次的语义信息；以及（iii）跨模型攻击迁移性的提升。据我们所知，这是首个使用除人脸模板之外额外信息来实现面部模板逆向过程的方法，并且在现有技术中取得了最佳结果（SOTA）。
URL
https://arxiv.org/abs/2512.15433
