Abstract
The Indian classical dance-drama Kathakali has a set of hand gestures called Mudras, which form the fundamental units of all its dance moves and postures. Recognizing the depicted mudra becomes one of the first steps in its digital processing. The work treats the problem as a 24-class classification task and proposes a vector-similarity-based approach using pose estimation, eliminating the need for further training or fine-tuning. This approach overcomes the challenge of data scarcity that limits the application of AI in similar domains. The method attains 92% accuracy which is a similar or better performance as other model-training-based works existing in the domain, with the added advantage that the method can still work with data sizes as small as 1 or 5 samples with a slightly reduced performance. Working with images, videos, and even real-time streams is possible. The system can work with hand-cropped or full-body images alike. We have developed and made public a dataset for the Kathakali Mudra Recognition as part of this work.
Abstract (translated)
印度古典舞蹈戏剧Kathakali有一组被称为Mudras的手势，它们构成了所有其舞蹈动作和姿势的基本单位。识别所描绘的手势是其在数字处理中的第一步。该工作将问题视为一个24类分类任务，并使用姿态估计基于向量的方法，消除了进一步的训练或微调的需求。这种方法克服了数据稀缺性，这限制了AI在类似领域中的应用。该方法获得了92%的准确度，这是该领域其他基于模型训练的工作中的类似或更好的性能，并且具有可以与少量数据样本一起工作的稍微降低性能的优点。可以与图像、视频和实时流合作工作。系统可以处理手裁剪或全身图像。我们在这个工作中为Kathakali Mudra识别开发并公开了一个数据集。
URL
https://arxiv.org/abs/2404.11205
