Abstract
Micro-gesture recognition is a challenging task in affective computing due to the subtle, involuntary nature of the gestures and their low movement amplitude. In this paper, we introduce a Pose-Guided Semantics-Aware CLIP-based architecture, or CLIP for Micro-Gesture recognition (CLIP-MG), a modified CLIP model tailored for micro-gesture classification on the iMiGUE dataset. CLIP-MG integrates human pose (skeleton) information into the CLIP-based recognition pipeline through pose-guided semantic query generation and a gated multi-modal fusion mechanism. The proposed model achieves a Top-1 accuracy of 61.82%. These results demonstrate both the potential of our approach and the remaining difficulty in fully adapting vision-language models like CLIP for micro-gesture recognition.
Abstract (translated)
微手势识别是情感计算中的一个挑战性任务，由于这些细微、不自觉的手势动作幅度较低。本文介绍了Pose-Guided Semantics-Aware CLIP-based架构（简称CLIP-MG），这是一种为iMiGUE数据集上的微手势分类而修改的CLIP模型。CLIP-MG通过姿势引导的语义查询生成和门控多模态融合机制，将人体姿态（骨骼）信息整合到基于CLIP的识别管道中。所提出的模型达到了61.82%的Top-1准确率。这些结果既展示了我们方法的潜力，也反映了完全适应视觉-语言模型如CLIP进行微手势识别的难度仍然存在。
URL
https://arxiv.org/abs/2506.16385
