Abstract
Modern face recognition models achieve high overall accuracy but continue to exhibit systematic biases that disproportionately affect certain subpopulations. Conventional bias evaluation frameworks rely on labeled attributes to form subpopulations, which are expensive to obtain and limited to predefined categories. We introduce Latent Feature Alignment (LFA), an attribute-label-free algorithm that uses latent directions to identify subpopulations. This yields two main benefits over standard clustering: (i) semantically coherent grouping, where faces sharing common attributes are grouped together more reliably than by proximity-based methods, and (ii) discovery of interpretable directions, which correspond to semantic attributes such as age, ethnicity, or attire. Across four state-of-the-art recognition models (ArcFace, CosFace, ElasticFace, PartialFC) and two benchmarks (RFW, CelebA), LFA consistently outperforms k-means and nearest-neighbor search in intra-group semantic coherence, while uncovering interpretable latent directions aligned with demographic and contextual attributes. These results position LFA as a practical method for representation auditing of face recognition models, enabling practitioners to identify and interpret biased subpopulations without predefined attribute annotations.
Abstract (translated)
现代面部识别模型在整体准确性方面表现出色，但仍存在系统性偏见，这些偏见对某些特定亚群体的影响尤为显著。传统的偏差评估框架依赖于带有标签的属性来划分亚群，而获得这样的标签既昂贵又局限于预定义的类别范围之内。我们引入了一种名为隐含特征对齐（Latent Feature Alignment, LFA）的新算法，它无需标注属性即可通过隐含方向识别出亚群体。这种方法相较于标准聚类方法主要具有两大优势： (i) 半语义一致分组：LFA 能够更可靠地将共享相似属性的面孔进行分组，而不仅仅是基于位置接近性的方法。 (ii) 发现可解释的方向：该算法能够揭示与年龄、种族或着装等语义特征相对应的隐含方向。 在四个最先进的面部识别模型（ArcFace, CosFace, ElasticFace, PartialFC）以及两个基准测试数据集（RFW和CelebA）上，LFA 无论是在组内语义一致性还是发现与人口统计学和上下文属性相匹配的可解释性隐含方向方面均优于 k-means 聚类方法及最近邻搜索。 这些结果表明 LFA 可以作为一种实用的方法来审计面部识别模型的表现形式，帮助从业人员在没有预定义属性注释的情况下识别并解读有偏见的亚群体。
URL
https://arxiv.org/abs/2510.15520
