Abstract
Gesture recognition is an important research area in the field of computer vision. Most gesture recognition efforts focus on close-set scenarios, thereby limiting the capacity to effectively handle unseen or novel gestures. We aim to address class-incremental gesture recognition, which entails the ability to accommodate new and previously unseen gestures over time. Specifically, we introduce a Prototype-Guided Pseudo Feature Replay (PGPFR) framework for data-free class-incremental gesture recognition. This framework comprises four components: Pseudo Feature Generation with Batch Prototypes (PFGBP), Variational Prototype Replay (VPR) for old classes, Truncated Cross-Entropy (TCE) for new classes, and Continual Classifier Re-Training (CCRT). To tackle the issue of catastrophic forgetting, the PFGBP dynamically generates a diversity of pseudo features in an online manner, leveraging class prototypes of old classes along with batch class prototypes of new classes. Furthermore, the VPR enforces consistency between the classifier's weights and the prototypes of old classes, leveraging class prototypes and covariance matrices to enhance robustness and generalization capabilities. The TCE mitigates the impact of domain differences of the classifier caused by pseudo features. Finally, the CCRT training strategy is designed to prevent overfitting to new classes and ensure the stability of features extracted from old classes. Extensive experiments conducted on two widely used gesture recognition datasets, namely SHREC 2017 3D and EgoGesture 3D, demonstrate that our approach outperforms existing state-of-the-art methods by 11.8\% and 12.8\% in terms of mean global accuracy, respectively. The code is available on this https URL.
Abstract (translated)
手势识别是计算机视觉领域的一个重要研究方向。大多数的手势识别工作集中在闭合集场景上，这限制了有效处理未见过或新颖手势的能力。我们旨在解决类增量手势识别问题，即随着时间的推移能够适应新的和以前未见过的手势。为此，我们引入了一个名为基于原型指导伪特征回放（Prototype-Guided Pseudo Feature Replay, PGPFR）的数据无需求框架来进行类增量手势识别。该框架包括四个组成部分：使用批次原型进行伪特征生成（Pseudo Feature Generation with Batch Prototypes, PFGBP）、旧类的变分原型回放（Variational Prototype Replay, VPR）、新类的截断交叉熵（Truncated Cross-Entropy, TCE），以及连续分类器重新训练策略（Continual Classifier Re-Training, CCRT）。 为了应对灾难性遗忘的问题，PFGBP动态地以在线方式生成多样化的伪特征，利用旧类别的类别原型和新批次的类别原型。此外，VPR通过类别原型和协方差矩阵来增强模型的稳健性和泛化能力，强制分类器权重与旧类别的原型保持一致。TCE则减轻了由伪特征引起的分类器领域差异的影响。最后，CCRT训练策略旨在防止过度拟合新类，并确保从老类提取的特征稳定性。 在两个广泛使用的手势识别数据集（SHREC 2017 3D和EgoGesture 3D）上进行的大量实验表明，在平均全局准确性方面，我们的方法分别比现有的最先进方法高出11.8% 和12.8%。相关代码可在提供的链接中获取。
URL
https://arxiv.org/abs/2505.20049
