Abstract
Gesture recognition presents a promising avenue for interfacing with unmanned aerial vehicles (UAVs) due to its intuitive nature and potential for precise interaction. This research conducts a comprehensive comparative analysis of vision-based hand gesture detection methodologies tailored for UAV Control. The existing gesture recognition approaches involving cropping, zooming, and color-based segmentation, do not work well for this kind of applications in dynamic conditions and suffer in performance with increasing distance and environmental noises. We propose to use a novel approach leveraging hand landmarks drawing and classification for gesture recognition based UAV control. With experimental results we show that our proposed method outperforms the other existing methods in terms of accuracy, noise resilience, and efficacy across varying distances, thus providing robust control decisions. However, implementing the deep learning based compute intensive gesture recognition algorithms on the UAV's onboard computer is significantly challenging in terms of performance. Hence, we propose to use a edge-computing based framework to offload the heavier computing tasks, thus achieving closed-loop real-time performance. With implementation over AirSim simulator as well as over a real-world UAV, we showcase the advantage of our end-to-end gesture recognition based UAV control system.
Abstract (translated)
手势识别为与无人机（UAV）交互提供了一条有前景的途径，因其直观性和潜在的精确互动能力。本研究对基于视觉的手势检测方法进行了全面的比较分析，这些方法专门用于UAV控制。现有的涉及裁剪、缩放和基于颜色分割的手势识别方法，在动态条件下效果不佳，并且随着距离增加和环境噪声加剧，性能会下降。我们提出了一种新的方法，利用手部关键点绘制与分类进行手势识别，从而实现基于UAV的控制。通过实验结果表明，相较于现有的方法，我们的方法在准确性、抗噪性和不同距离下的有效性方面表现更佳，因此能够提供稳健的控制决策。然而，在无人机上部署深度学习驱动的手势识别算法面临计算资源有限的问题。为此，我们提出了一种基于边缘计算框架的方法来卸载繁重的计算任务，从而实现闭环实时性能。通过在AirSim模拟器和实际UAV上的实施，展示了我们的端到端手势识别控制系统的优势。
URL
https://arxiv.org/abs/2505.17303
