Abstract
This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hand Landmarker, responsible for extracting 21 skeletal keypoints of the hand, and a convolutional neural network (CNN) trained to classify gestures from a spatiotemporal matrix representation of dimensions 90 by 21 of those keypoints. The method is applied to the recognition of LIBRAS (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures. For real-time inference, a sliding window with temporal frame triplication is used, enabling continuous recognition without recurrent networks. Tests achieved 95\% accuracy under low-light conditions and 92\% under normal lighting. The results indicate that the approach is effective, although systematic experiments with greater user diversity are needed for a more thorough evaluation of generalization.
Abstract (translated)
本文提出一种基于两种模型组合的动态手势识别方法：MediaPipe手部关键点检测器负责提取21个手部骨骼关键点，卷积神经网络（CNN）则对这些关键点形成的90×21维时空矩阵进行手势分类。该方法应用于巴西手语（LIBRAS）手势识别，以实现智能家居设备控制，涵盖11类静态与动态手势。为实现实时推理，采用带时间帧复制的滑动窗口机制，无需循环网络即可实现连续识别。测试表明，该方法在低光照条件下准确率达95%，正常光照下为92%。结果表明该方案有效，但需通过更多样化的用户实验来进一步评估其泛化能力。
URL
https://arxiv.org/abs/2603.25863
