Abstract
Current gait recognition research mainly focuses on identifying pedestrians captured by the same type of sensor, neglecting the fact that individuals may be captured by different sensors in order to adapt to various environments. A more practical approach should involve cross-modality matching across different sensors. Hence, this paper focuses on investigating the problem of cross-modality gait recognition, with the objective of accurately identifying pedestrians across diverse vision sensors. We present CrossGait inspired by the feature alignment strategy, capable of cross retrieving diverse data modalities. Specifically, we investigate the cross-modality recognition task by initially extracting features within each modality and subsequently aligning these features across modalities. To further enhance the cross-modality performance, we propose a Prototypical Modality-shared Attention Module that learns modality-shared features from two modality-specific features. Additionally, we design a Cross-modality Feature Adapter that transforms the learned modality-specific features into a unified feature space. Extensive experiments conducted on the SUSTech1K dataset demonstrate the effectiveness of CrossGait: (1) it exhibits promising cross-modality ability in retrieving pedestrians across various modalities from different sensors in diverse scenes, and (2) CrossGait not only learns modality-shared features for cross-modality gait recognition but also maintains modality-specific features for single-modality recognition.
Abstract (translated)
当前的步态识别研究主要集中在识别由相同类型传感器捕捉的行人，忽视了个人可能被不同类型的传感器捕捉的事实，以适应各种环境。更实际的方法应该涉及不同传感器之间的跨模态匹配。因此，本文重点研究了跨模态步行识别问题，以准确识别不同视觉传感器捕捉的行人。我们提出了CrossGait，这是一种基于特征对齐策略的步行识别方法，具有跨检索不同数据模态的能力。具体来说，我们研究了跨模态识别任务，首先在每种模态中提取特征，然后在这些特征之间进行对齐。为了进一步提高跨模态性能，我们提出了一个原型模态共享注意模块，从两个模态特定的特征中学习模态共享特征。此外，我们还设计了一个Cross-模态特征适配器，将学习到的模态特定特征转换为统一特征空间。在SUSTech1K数据集上进行的大量实验证明CrossGait的有效性：（1）它表现出在不同场景的多样传感器中检索行人具有希望的跨模态能力；（2）CrossGait不仅学习跨模态步行识别的模态共享特征，还保留单模态识别的模态特定特征。
URL
https://arxiv.org/abs/2404.04120
