Abstract
In this work, we propose a new Dual Min-Max Games (DMMG) based self-supervised skeleton action recognition method by augmenting unlabeled data in a contrastive learning framework. Our DMMG consists of a viewpoint variation min-max game and an edge perturbation min-max game. These two min-max games adopt an adversarial paradigm to perform data augmentation on the skeleton sequences and graph-structured body joints, respectively. Our viewpoint variation min-max game focuses on constructing various hard contrastive pairs by generating skeleton sequences from various viewpoints. These hard contrastive pairs help our model learn representative action features, thus facilitating model transfer to downstream tasks. Moreover, our edge perturbation min-max game specializes in building diverse hard contrastive samples through perturbing connectivity strength among graph-based body joints. The connectivity-strength varying contrastive pairs enable the model to capture minimal sufficient information of different actions, such as representative gestures for an action while preventing the model from overfitting. By fully exploiting the proposed DMMG, we can generate sufficient challenging contrastive pairs and thus achieve discriminative action feature representations from unlabeled skeleton data in a self-supervised manner. Extensive experiments demonstrate that our method achieves superior results under various evaluation protocols on widely-used NTU-RGB+D and NTU120-RGB+D datasets.
Abstract (translated)
在本文中,我们提出了一种新的双重最小最大游戏(DMMG)基于自监督的骨骼行动识别方法,通过增加未标记的数据在对比学习框架中增加数据。我们的DMMG由一个观点变化最小最大游戏和一个边缘扰动最小最大游戏组成。这两个最小最大游戏采用对抗范式在骨骼序列和基于Graph结构的身体关节上进行数据增强。我们的观点变化最小最大游戏专注于通过从不同观点生成骨骼序列建立各种强对比对。这些强对比对帮助我们的模型学习代表行动特征,从而促进了模型向下级的任务的迁移。此外,我们的边缘扰动最小最大游戏专门致力于通过在基于Graph的身体关节之间的连接强度扰动建立各种强对比样本。这种连接强度的变化对比对使模型能够捕获不同行动的最小足够信息,例如代表一个行动的手势,同时防止模型过拟合。通过充分利用提出的DMMG,我们可以生成足够的挑战性对比对,从而在自监督的情况下从未标记的骨骼数据中实现分化的行动特征表示。广泛的实验表明,在我们广泛使用的NTU-RGB+D和NTU120-RGB+D数据集上,我们的方法在多种评估协议下取得了更好的结果。
URL
https://arxiv.org/abs/2302.12007
