Abstract
In this paper, we present MM-Gesture, the solution developed by our team HFUT-VUT, which ranked 1st in the micro-gesture classification track of the 3rd MiGA Challenge at IJCAI 2025, achieving superior performance compared to previous state-of-the-art methods. MM-Gesture is a multimodal fusion framework designed specifically for recognizing subtle and short-duration micro-gestures (MGs), integrating complementary cues from joint, limb, RGB video, Taylor-series video, optical-flow video, and depth video modalities. Utilizing PoseConv3D and Video Swin Transformer architectures with a novel modality-weighted ensemble strategy, our method further enhances RGB modality performance through transfer learning pre-trained on the larger MA-52 dataset. Extensive experiments on the iMiGUE benchmark, including ablation studies across different modalities, validate the effectiveness of our proposed approach, achieving a top-1 accuracy of 73.213%.
Abstract (translated)
在这篇论文中，我们介绍了MM-Gesture，这是由我们的团队HFUT-VUT开发的解决方案。该方案在IJCAI 2025年举行的第三届MiGA挑战赛微手势分类赛道上排名首位，并且其性能优于之前的最先进的方法。MM-Gesture是一个专门为识别细微且短时长的手势（简称MGs）而设计的多模态融合框架，它结合了关节、肢体、RGB视频、泰勒级数视频、光流视频和深度视频等多种模式中的互补线索。 我们的方法利用了PoseConv3D和Video Swin Transformer架构，并采用了一种新颖的模态加权集成策略。通过在更大的MA-52数据集上进行预训练，我们的方法进一步提升了RGB模态的表现。我们在iMiGUE基准测试上的广泛实验，包括不同模态下的消融研究，验证了我们提出的方法的有效性，实现了73.213%的顶级准确率。
URL
https://arxiv.org/abs/2507.08344
