Abstract
Hand gesture understanding is essential for several applications in human-computer interaction, including automatic clinical assessment of hand dexterity. While deep learning has advanced static gesture recognition, dynamic gesture understanding remains challenging due to complex spatiotemporal variations. Moreover, existing datasets often lack multimodal and multi-view diversity, precise ground-truth tracking, and an action quality component embedded within gestures. This paper introduces EHWGesture, a multimodal video dataset for gesture understanding featuring five clinically relevant gestures. It includes over 1,100 recordings (6 hours), captured from 25 healthy subjects using two high-resolution RGB-Depth cameras and an event camera. A motion capture system provides precise ground-truth hand landmark tracking, and all devices are spatially calibrated and synchronized to ensure cross-modal alignment. Moreover, to embed an action quality task within gesture understanding, collected recordings are organized in classes of execution speed that mirror clinical evaluations of hand dexterity. Baseline experiments highlight the dataset's potential for gesture classification, gesture trigger detection, and action quality assessment. Thus, EHWGesture can serve as a comprehensive benchmark for advancing multimodal clinical gesture understanding.
Abstract (translated)
手势理解在人机交互的多个应用中至关重要，包括自动临床评估手部灵巧度。虽然深度学习已经显著提升了静态手势识别的能力，但动态手势的理解仍然面临挑战，因为这些手势包含复杂的时空变化。此外，现有的数据集往往缺乏多模态和多视角多样性、精确的真值追踪以及嵌入手势中的动作质量成分。 本文介绍了一种名为EHWGesture的数据集，它是一个用于手势理解的多模态视频数据集，包含了五个具有临床相关性的手势。该数据集中包含超过1,100个记录（总计6小时），这些记录由25名健康受试者使用两台高分辨率RGB-Depth相机和一台事件相机录制而成。一个运动捕捉系统提供了精确的手部地标追踪真值，并且所有设备均经过空间校准并同步以确保跨模式对齐。此外，为了在手势理解中嵌入动作质量任务，收集的记录按照执行速度类别进行分类，这反映了临床对手部灵巧度评估的方式。 基础实验突显了该数据集在手势分类、手势触发检测和动作质量评估方面的潜力。因此，EHWGesture可以作为推进多模态临床手势理解研究的一个全面基准。
URL
https://arxiv.org/abs/2509.07525
