Abstract
Gesture-based control for mobile manipulators faces persistent challenges in reliability, efficiency, and intuitiveness. This paper presents a dual-hand gesture interface that integrates TinyML, spectral analysis, and sensor fusion within a ROS framework to address these limitations. The system uses left-hand tilt and finger flexion, captured using accelerometer and flex sensors, for mobile base navigation, while right-hand IMU signals are processed through spectral analysis and classified by a lightweight neural network. This pipeline enables TinyML-based gesture recognition to control a 7-DOF Kinova Gen3 manipulator. By supporting simultaneous navigation and manipulation, the framework improves efficiency and coordination compared to sequential methods. Key contributions include a bimanual control architecture, real-time low-power gesture recognition, robust multimodal sensor fusion, and a scalable ROS-based implementation. The proposed approach advances Human-Robot Interaction (HRI) for industrial automation, assistive robotics, and hazardous environments, offering a cost-effective, open-source solution with strong potential for real-world deployment and further optimization.
Abstract (translated)
基于手势控制的移动操作器面临可靠性、效率和直观性等方面的持续挑战。本文提出了一种双手持界面，结合了TinyML（轻量级机器学习）、频谱分析以及多传感器融合技术，在ROS框架内运行以解决这些问题。该系统使用左手倾斜及手指弯曲动作来导航移动基座，这些动作通过加速度计和柔性传感器捕捉；而右手的IMU信号则经过频谱分析，并由轻量化神经网络进行分类。这一流程使得基于TinyML的手势识别能够控制具有7个自由度的Kinova Gen3操作器。通过支持同时进行导航与操作，该框架在效率和协调性上相比顺序方法有所提升。 关键贡献包括： - 双手控制系统架构 - 实时低功耗手势识别技术 - 稳健多模式传感器融合能力 - 基于ROS的可扩展实现 所提出的这种方法推进了人机交互（HRI）在工业自动化、辅助机器人和危险环境中的应用，提供了一个成本效益高且开源的解决方案，具有显著的实际部署潜力以及进一步优化的空间。
URL
https://arxiv.org/abs/2509.19521
