Abstract
Skeleton-based motion representations are robust for action localization and understanding for their invariance to perspective, lighting, and occlusion, compared with images. Yet, they are often ambiguous and incomplete when taken out of context, even for human annotators. As infants discern gestures before associating them with words, actions can be conceptualized before being grounded with labels. Therefore, we propose the first unsupervised pre-training framework, Boundary-Interior Decoding (BID), that partitions a skeleton-based motion sequence into discovered semantically meaningful pre-action segments. By fine-tuning our pre-training network with a small number of annotated data, we show results out-performing SOTA methods by a large margin.
Abstract (translated)
基于骨架的运动表示对于动作定位和理解具有良好的鲁棒性，因为它们对视角、照明和遮挡具有不变性，而与图像相比。然而，当脱离上下文时，它们往往是不清晰和不完整的。正如婴儿在将动作与词语关联之前就开始感知一样，在将动作与标签关联之前，动作可以先于标签进行概念化。因此，我们提出了第一个无监督的前训练框架，边界内解码（BID），将基于骨架的运动序列分割为发现 semantically 有意义的前动作段。通过用小量标记数据微调我们的预训练网络，我们证明了其性能优于当前最先进的方法。
URL
https://arxiv.org/abs/2403.07354
