Abstract
Gait recognition is a biometric technology that has received extensive attention. Most existing gait recognition algorithms are unimodal, and a few multimodal gait recognition algorithms perform multimodal fusion only once. None of these algorithms may fully exploit the complementary advantages of the multiple modalities. In this paper, by considering the temporal and spatial characteristics of gait data, we propose a multi-stage feature fusion strategy (MSFFS), which performs multimodal fusions at different stages in the feature extraction process. Also, we propose an adaptive feature fusion module (AFFM) that considers the semantic association between silhouettes and skeletons. The fusion process fuses different silhouette areas with their more related skeleton joints. Since visual appearance changes and time passage co-occur in a gait period, we propose a multiscale spatial-temporal feature extractor (MSSTFE) to learn the spatial-temporal linkage features thoroughly. Specifically, MSSTFE extracts and aggregates spatial-temporal linkages information at different spatial scales. Combining the strategy and modules mentioned above, we propose a multi-stage adaptive feature fusion (MSAFF) neural network, which shows state-of-the-art performance in many experiments on three datasets. Besides, MSAFF is equipped with feature dimensional pooling (FD Pooling), which can significantly reduce the dimension of the gait representations without hindering the accuracy. this https URL
Abstract (translated)
翻译：步态识别是一种生物识别技术，已受到广泛关注。现有的步态识别算法大多是单模态的，而一些多模态步态识别算法仅在特征提取过程中执行一次多模态融合。这些算法可能没有充分利用多个模态的互补优势。在本文中，我们通过考虑步态数据的时序和空间特性，提出了一个多级特征融合策略（MSFFS），在特征提取过程的不同阶段执行多模态融合。此外，我们还提出了一个自适应特征融合模块（AFFM），考虑了轮廓和骨架之间的语义关联。融合过程将不同轮廓区域的特征与它们更相关的骨架关节融合在一起。由于在步行周期中视觉外观变化和时间流逝是同时发生的，我们提出了一个多尺度空间时元特征提取器（MSSTFE），以深入学习空间时元特征。具体来说，MSSTFE在不同的空间尺度上提取和聚合空间时元连接信息。结合上述策略和模块，我们提出了一个多级自适应特征融合（MSAFF）神经网络，在多个数据集上的实验表现均达到了最先进的水平。此外，MSAFF配备了特征维度池化（FD Pooling）模块，这可以在不降低准确性的情况下显著减少步行表示的维度。
URL
https://arxiv.org/abs/2312.14410
