Abstract
Person search aims to localize specific a target person from a gallery set of images with various scenes. As the scene of moving pedestrian changes, the captured person image inevitably bring in lots of background noise and foreground noise on the person feature, which are completely unrelated to the person identity, leading to severe performance degeneration. To address this issue, we present a Scene-Adaptive Person Search (SEAS) model by introducing bilateral modulations to simultaneously eliminate scene noise and maintain a consistent person representation to adapt to various scenes. In SEAS, a Background Modulation Network (BMN) is designed to encode the feature extracted from the detected bounding box into a multi-granularity embedding, which reduces the input of background noise from multiple levels with norm-aware. Additionally, to mitigate the effect of foreground noise on the person feature, SEAS introduces a Foreground Modulation Network (FMN) to compute the clutter reduction offset for the person embedding based on the feature map of the scene image. By bilateral modulations on both background and foreground within an end-to-end manner, SEAS obtains consistent feature representations without scene noise. SEAS can achieve state-of-the-art (SOTA) performance on two benchmark datasets, CUHK-SYSU with 97.1\% mAP and PRW with 60.5\% mAP. The code is available at this https URL.
Abstract (translated)
人员搜索旨在从具有各种场景的图库中定位特定目标人员。当动态行人经过时，捕获的个人信息图像会不可避免地引入大量背景噪音和前景噪音，而这些噪音与人员身份无关，导致性能下降。为解决这个问题，我们提出了一个场景适应人员搜索（SEAS）模型，通过双向调用来同时消除场景噪音并保持一致的人员表示，以适应各种场景。在SEAS中，我们设计了一个背景模块网络（BMN），将其捕获到的边界框特征编码为多粒度嵌入，从而减少了来自多个级别背景噪音的输入。此外，为了减轻前景噪音对人员特征的影响，SEAS引入了一个前景模块网络（FMN），根据场景图像的特征图计算降噪偏移量。通过在背景和前景之间以端到端的方式进行双向调节，SEAS获得了无场景噪音的稳定特征表示。SEAS在两个基准数据集上的性能达到了最先进水平（SOTA），CUHK-SYSU with 97.1\% mAP和PRW with 60.5\% mAP。代码可在此处访问：https://www.aclweb.org/anthology/J/SEAS2023/10265
URL
https://arxiv.org/abs/2405.02834
