Abstract
Hand gesture-based human-computer interaction is an important problem that is well explored using color camera data. In this work we proposed a hand gesture detection system using thermal images. Our system is capable of handling multiple hand regions in a frame and process it fast for real-time applications. Our system performs a series of steps including background subtraction-based hand mask generation, k-means based hand region identification, hand segmentation to remove the forearm region, and a Convolutional Neural Network (CNN) based gesture classification. Our work introduces two novel algorithms, bubble growth and bubble search, for faster hand segmentation. We collected a new thermal image data set with 10 gestures and reported an end-to-end hand gesture recognition accuracy of 97%.
Abstract (translated)
基于手势的人机交互是一个重要问题,已经通过彩色相机数据进行了充分的探索。在这个项目中,我们提出了一种利用热成像技术进行手势检测的系统。我们的系统能够在一个帧内处理多个手区域,并快速应用于实时应用程序。我们的系统执行了一系列步骤,包括基于背景减除的手遮挡生成、基于聚类技术的手部区域识别、手部分割以去除前臂区域,以及基于卷积神经网络(CNN)的手势分类。我们的工作介绍了两个新的算法,即 bubble growth 和 bubble search,以加快手部分割。我们收集了包含10个手势的新热成像数据集,并报告了端到端手部识别准确率高达97%。
URL
https://arxiv.org/abs/2303.02321
