Abstract
We address the problem of whole-body person recognition in unconstrained environments. This problem arises in surveillance scenarios such as those in the IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) program, where biometric data is captured at long standoff distances, elevated viewing angles, and under adverse atmospheric conditions (e.g., turbulence and high wind velocity). To this end, we propose FarSight, a unified end-to-end system for person recognition that integrates complementary biometric cues across face, gait, and body shape modalities. FarSight incorporates novel algorithms across four core modules: multi-subject detection and tracking, recognition-aware video restoration, modality-specific biometric feature encoding, and quality-guided multi-modal fusion. These components are designed to work cohesively under degraded image conditions, large pose and scale variations, and cross-domain gaps. Extensive experiments on the BRIAR dataset, one of the most comprehensive benchmarks for long-range, multi-modal biometric recognition, demonstrate the effectiveness of FarSight. Compared to our preliminary system, this system achieves a 34.1% absolute gain in 1:1 verification accuracy (TAR@0.1% FAR), a 17.8% increase in closed-set identification (Rank-20), and a 34.3% reduction in open-set identification errors (FNIR@1% FPIR). Furthermore, FarSight was evaluated in the 2025 NIST RTE Face in Video Evaluation (FIVE), which conducts standardized face recognition testing on the BRIAR dataset. These results establish FarSight as a state-of-the-art solution for operational biometric recognition in challenging real-world conditions.
Abstract (translated)
我们解决了在非受控环境中进行全身人物识别的问题。该问题主要出现在诸如IARPA生物识别和高度范围识别（BRIAR）计划之类的监控场景中，其中生物特征数据是在长距离、高视角以及恶劣的大气条件下（例如湍流和强风速）捕获的。为此，我们提出了FarSight系统，这是一个统一的端到端人物识别系统，该系统整合了面部、步态和体型等多模态生物特征线索。 FarSight包含四个核心模块的新算法：多主体检测与跟踪、以识别为导向的视频恢复、特定于模式的生物特征编码以及质量导向的跨模态融合。这些组件设计为在图像退化条件、大姿态和尺度变化以及跨域差距下协同工作。 通过对BRIAR数据集进行广泛实验，该数据集是用于长距离多模态生物识别基准测试中最全面的数据集之一，证明了FarSight的有效性。与我们的初步系统相比，在一对一验证准确性（TAR@0.1% FAR）方面提高了34.1%，在封闭式身份识别（Rank-20）中提高了17.8%，开放集合身份识别错误减少了34.3%（FNIR@1% FPIR）。此外，FarSight还在NIST 2025年FIVE（BRIAR数据集的标准化视频面部识别测试）中进行了评估。 这些结果使FarSight成为在具有挑战性的现实条件下进行操作生物特征识别的最新解决方案。
URL
https://arxiv.org/abs/2505.04616
