Abstract
The choice of the representations is essential for deep gait recognition methods. The binary silhouettes and skeletal coordinates are two dominant representations in recent literature, achieving remarkable advances in many scenarios. However, inherent challenges remain, in which silhouettes are not always guaranteed in unconstrained scenes, and structural cues have not been fully utilized from skeletons. In this paper, we introduce a novel skeletal gait representation named Skeleton Map, together with SkeletonGait, a skeleton-based method to exploit structural information from human skeleton maps. Specifically, the skeleton map represents the coordinates of human joints as a heatmap with Gaussian approximation, exhibiting a silhouette-like image devoid of exact body structure. Beyond achieving state-of-the-art performances over five popular gait datasets, more importantly, SkeletonGait uncovers novel insights about how important structural features are in describing gait and when do they play a role. Furthermore, we propose a multi-branch architecture, named SkeletonGait++, to make use of complementary features from both skeletons and silhouettes. Experiments indicate that SkeletonGait++ outperforms existing state-of-the-art methods by a significant margin in various scenarios. For instance, it achieves an impressive rank-1 accuracy of over $85\%$ on the challenging GREW dataset. All the source code will be available at this https URL.
Abstract (translated)
选择表示方式对深度步态识别方法至关重要。二值轮廓和骨架坐标是最近文献中两种主导表示方式，在许多场景中取得了显著的进步。然而，仍然存在一些固有挑战，其中轮廓并不总是保证在约束场景中，从骨架中也没有完全利用到结构信息。在本文中，我们提出了一个新颖的骨架步态表示名为骨架映射，以及一个基于骨架的人体骨架图挖掘方法SkeletonGait。具体来说，骨架映射用高斯近似的二维图像表示人类关节的坐标，呈现出类似轮廓的图像，缺乏精确的身体结构。除了在五个流行的步态数据集上实现最先进的性能，更重要的是，SkeletonGait揭示了描述步态和何时结构特征发挥作用的新见解。此外，我们提出了一个多分支架构，名为SkeletonGait++，以利用骨架和轮廓的互补特征。实验表明，SkeletonGait++在各种场景中都显著优于现有最先进的方法。例如，它在具有挑战性的GREW数据集上实现了令人印象深刻的排名前1%的准确性。所有源代码都将公开发布在这个https URL上。
URL
https://arxiv.org/abs/2311.13444
