Abstract
Many recent news headlines have labeled face recognition technology as biased or racist. We report on a methodical investigation into differences in face recognition accuracy between African-American and Caucasian image cohorts of the MORPH dataset. We find that, for all four matchers considered, the impostor and the genuine distributions are statistically significantly different between cohorts. For a fixed decision threshold, the African-American image cohort has a higher false match rate and a lower false non-match rate. ROC curves compare verification rates at the same false match rate, but the different cohorts achieve the same false match rate at different thresholds. This means that ROC comparisons are not relevant to operational scenarios that use a fixed decision threshold. We show that, for the ResNet matcher, the two cohorts have approximately equal separation of impostor and genuine distributions. Using ICAO compliance as a standard of image quality, we find that the initial image cohorts have unequal rates of good quality images. The ICAO-compliant subsets of the original image cohorts show improved accuracy, with the main effect being to reducing the low-similarity tail of the genuine distributions.
Abstract (translated)
最近的许多新闻标题都将面部识别技术称为偏见或种族主义。我们报告了对变形数据集的非裔美国人和白种人图像队列中人脸识别准确性差异的系统调查。我们发现，对于所考虑的四个匹配者，冒名顶替者和真实分布在两个队列之间有显著的统计差异。对于一个固定的决策阈值，非裔美国人图像队列具有较高的假匹配率和较低的假不匹配率。ROC曲线在相同的假匹配率下比较验证率，但不同的队列在不同的阈值下获得相同的假匹配率。这意味着ROC比较与使用固定决策阈值的操作场景无关。我们证明，对于resnet匹配器，两个队列对冒名顶替者和真实分配有近似相等的分离。以国际民航组织的合规性作为图像质量标准，我们发现初始图像队列具有不平等的高质量图像率。原始图像队列中符合国际民航组织的子集显示出更高的准确性，其主要作用是减少真实分布的低相似性尾。
URL
https://arxiv.org/abs/1904.07325
