Abstract
Gait recognition has achieved promising advances in controlled settings, yet it significantly struggles in unconstrained environments due to challenges such as view changes, occlusions, and varying walking speeds. Additionally, efforts to fuse multiple modalities often face limited improvements because of cross-modality incompatibility, particularly in outdoor scenarios. To address these issues, we present a multi-modal Hierarchy in Hierarchy network (HiH) that integrates silhouette and pose sequences for robust gait recognition. HiH features a main branch that utilizes Hierarchical Gait Decomposer (HGD) modules for depth-wise and intra-module hierarchical examination of general gait patterns from silhouette data. This approach captures motion hierarchies from overall body dynamics to detailed limb movements, facilitating the representation of gait attributes across multiple spatial resolutions. Complementing this, an auxiliary branch, based on 2D joint sequences, enriches the spatial and temporal aspects of gait analysis. It employs a Deformable Spatial Enhancement (DSE) module for pose-guided spatial attention and a Deformable Temporal Alignment (DTA) module for aligning motion dynamics through learned temporal offsets. Extensive evaluations across diverse indoor and outdoor datasets demonstrate HiH's state-of-the-art performance, affirming a well-balanced trade-off between accuracy and efficiency.
Abstract (translated)
翻译： 平衡设置中，平衡计取得了进展，但在无约束的环境中，由于诸如视野变化、遮挡和不同行走速度等问题的存在，它 significantly 挣扎。此外，由于跨模态不兼容，将多个模式融合的努力通常面临有限的改进，特别是在户外场景中。为解决这些问题，我们提出了一个多模态层次结构层次网络（HiH），该网络整合了轮廓和姿态序列以实现稳健的步态识别。HiH 具有主分支和辅助分支。主分支利用分层步态分解器（HGD）模块对轮廓数据进行深度和内部模块层次检查，以捕捉整体身体动态到详细肢体运动的运动层次结构。这种方法从总体身体动态到详细肢体运动捕捉运动层次结构，从而在多个空间分辨率上表示步态属性。补充的是，辅助分支基于二维关节序列，丰富了步态分析的时空方面。它采用了一个可塑的空间增强（DSE）模块进行姿态引导的空间关注，和一个可塑的时间对齐（DTA）模块通过学习到的时间偏移来对运动动态进行对齐。在多样室内和室外数据集上进行广泛的评估证明HiH 实现了最先进的性能，确实实现了准确性和效率之间的良好平衡。
URL
https://arxiv.org/abs/2311.11210
