Abstract
Face identity provides a powerful signal for deepfake detection. Prior studies show that even when not explicitly modeled, classifiers often learn identity features implicitly. This has led to conflicting views: some suppress identity cues to reduce bias, while others rely on them as forensic evidence. To reconcile these views, we analyze two hypotheses: (1) whether face identity alone is discriminative for detecting deepfakes, and (2) whether such identity features generalize poorly across manipulation methods. Our experiments confirm that identity is informative but context-dependent. While some manipulations preserve identity-consistent artifacts, others distort identity cues and harm generalization. We argue that identity features should neither be blindly suppressed nor relied upon, but instead be explicitly modeled and adaptively controlled based on per-sample relevance. We propose \textbf{SELFI} (\textbf{SEL}ective \textbf{F}usion of \textbf{I}dentity), a generalizable detection framework that dynamically modulates identity usage. SELFI consists of: (1) a Forgery-Aware Identity Adapter (FAIA) that extracts identity embeddings from a frozen face recognition model and projects them into a forgery-relevant space via auxiliary supervision; and (2) an Identity-Aware Fusion Module (IAFM) that selectively integrates identity and visual features using a relevance-guided fusion mechanism. Experiments on four benchmarks show that SELFI improves cross-manipulation generalization, outperforming prior methods by an average of 3.1\% AUC. On the challenging DFDC dataset, SELFI exceeds the previous best by 6\%. Code will be released upon paper acceptance.
Abstract (translated)
人脸身份为深度伪造检测提供了一个强大的信号。先前的研究表明，即使没有被明确建模，分类器也往往会隐式地学习到身份特征。这导致了相互冲突的观点：一些研究抑制身份线索以减少偏差，而另一些则依赖它们作为法医证据。为了调和这些观点，我们分析了两个假设：(1) 人脸身份本身是否足以用于检测深度伪造；(2) 这类身份特征在不同的操作方法下是否泛化能力较差。我们的实验确认身份信息是有用的但也是上下文相关的。虽然某些操纵保留了与身份一致的痕迹，但也有一些则扭曲了身份线索并损害了泛化性能。我们主张，不应盲目地抑制或依赖身份特征，而应该根据每个样本的相关性对其进行明确建模和自适应控制。 为此，我们提出了**SELFI**(Selective Fusion of Identity)，这是一个可广泛应用于检测任务中的框架，能够动态调整对身份信息的使用方式。SELFI 包含两个主要部分：(1) 伪造感知的身份适配器 (FAIA)，它从冻结的人脸识别模型中提取身份嵌入，并通过辅助监督将其投影到与伪造相关的空间；以及 (2) 身份感知融合模块 (IAFM)，该模块使用基于相关性引导的机制选择性地结合身份和视觉特征。 在四个基准测试上的实验表明，SELFI 在跨操作泛化方面取得了提升，平均 AUC 比先前的方法提高了 3.1%。特别是在具有挑战性的 DFDC 数据集上，SELFI 的表现超过了之前的最佳结果 6%。待论文被接受后将公开代码。
URL
https://arxiv.org/abs/2506.17592
