Abstract
Measuring the accuracy of face recognition (FR) systems is essential for improving performance and ensuring responsible use. Accuracy is typically estimated using large annotated datasets, which are costly and difficult to obtain. We propose a novel method for 1:1 face verification that benchmarks FR systems quickly and without manual annotation, starting from approximate labels (e.g., from web search results). Unlike previous methods for training set label cleaning, ours leverages the embedding representation of the models being evaluated, achieving high accuracy in smaller-sized test datasets. Our approach reliably estimates FR accuracy and ranking, significantly reducing the time and cost of manual labeling. We also introduce the first public benchmark of five FR cloud services, revealing demographic biases, particularly lower accuracy for Asian women. Our rapid test method can democratize FR testing, promoting scrutiny and responsible use of the technology. Our method is provided as a publicly accessible tool at this https URL
Abstract (translated)
人脸识别（FR）系统准确性的测量对于提升性能和确保负责任的使用至关重要。准确性通常通过大型标注数据集来估计，而这些数据集的成本高昂且难以获取。我们提出了一种新颖的方法，用于1:1面部验证，该方法可以在不进行人工标注的情况下快速地对FR系统的性能进行基准测试，从近似的标签（例如来自网络搜索结果的标签）开始。与以前用于训练集标签清洗的方法不同，我们的方法利用了被评估模型的嵌入表示，在较小规模的数据集中实现了高准确性。 我们这种方法可靠地估计了FR系统的准确性和排名，显著减少了人工标注的时间和成本。此外，我们还推出了首个五种云服务人脸识别技术的公开基准测试，揭示了这些系统中存在的种族偏见，尤其是对亚洲女性的识别精度较低。 我们的快速测试方法可以促进更多人参与人脸验证技术的测试，并鼓励对该技术进行仔细审查和负责任地使用。这种方法已作为公共可用工具在以下链接提供：[此URL]（请将这里的URL替换为实际提供的链接）。
URL
https://arxiv.org/abs/2502.14996
