Abstract
Air-writing refers to virtually writing linguistic characters through hand gestures in three-dimensional space with six degrees of freedom. This paper proposes a generic video camera-aided convolutional neural network (CNN) based air-writing framework. Gestures are performed using a marker of fixed color in front of a generic video camera, followed by color-based segmentation to identify the marker and track the trajectory of the marker tip. A pre-trained CNN is then used to classify the gesture. The recognition accuracy is further improved using transfer learning with the newly acquired data. The performance of the system varies significantly on the illumination condition due to color-based segmentation. In a less fluctuating illumination condition, the system is able to recognize isolated unistroke numerals of multiple languages. The proposed framework has achieved 97.7%, 95.4% and 93.7% recognition rates in person independent evaluations on English, Bengali and Devanagari numerals, respectively.
Abstract (translated)
空气写作是指通过手势在三维空间中几乎画出语言学字符的方法。本文提出了基于通用视频摄像头辅助的卷积神经网络(CNN)的空气写作框架。手势使用一种固定的颜色标记在通用视频摄像头前进行,然后使用基于颜色的分割来识别标记并跟踪标记的轨迹。然后,使用预先训练的CNN来分类手势。通过使用新获取的数据进行Transfer Learning,识别精度得到了进一步改善。由于基于颜色的分割,系统的照明条件表现差异很大。在较少波动的照明条件下,系统能够识别多种语言的孤立单划数。本文提出的框架在英语、孟加拉语和Devanagari数制的独立评估中分别取得了97.7%、95.4%和93.7%的识别率。
URL
https://arxiv.org/abs/2303.07989
