Abstract
The self-supervised pretraining paradigm has achieved great success in learning 3D action representations for skeleton-based action recognition using contrastive learning. However, learning effective representations for skeleton-based temporal action localization remains challenging and underexplored. Unlike video-level {action} recognition, detecting action boundaries requires temporally sensitive features that capture subtle differences between adjacent frames where labels change. To this end, we formulate a snippet discrimination pretext task for self-supervised pretraining, which densely projects skeleton sequences into non-overlapping segments and promotes features that distinguish them across videos via contrastive learning. Additionally, we build on strong backbones of skeleton-based action recognition models by fusing intermediate features with a U-shaped module to enhance feature resolution for frame-level localization. Our approach consistently improves existing skeleton-based contrastive learning methods for action localization on BABEL across diverse subsets and evaluation protocols. We also achieve state-of-the-art transfer learning performance on PKUMMD with pretraining on NTU RGB+D and BABEL.
Abstract (translated)
自监督预训练范式在使用对比学习方法从基于骨架的动作识别中学习3D动作表示方面取得了巨大成功。然而，为基于骨架的时间动作定位学习有效的表示仍然具有挑战性且研究不足。与视频级动作识别不同，检测动作边界需要能够捕捉到标签变化前后相邻帧之间细微差别的时间敏感特征。为此，我们提出了一个片段区分的预训练任务，将骨架序列密集地投影到非重叠段中，并通过对比学习促进跨视频区分这些段落的特性。此外，我们在基于骨架的动作识别模型的强大骨干网络基础上融合了中间特征和U形模块以增强帧级定位的特征分辨率。我们的方法在BABEL数据集的不同子集和评估协议上持续改进现有的基于骨架的对比学习方法用于动作定位的表现。我们还在PKUMMD上实现了最先进的迁移学习性能，使用NTU RGB+D和BABEL进行预训练。
URL
https://arxiv.org/abs/2512.16504
