Abstract
This paper proposes a wearable-controlled mobile manipulator system for intelligent smart home assistance, integrating MEMS capacitive microphones, IMU sensors, vibration motors, and pressure feedback to enhance human-robot interaction. The wearable device captures forearm muscle activity and converts it into real-time control signals for mobile manipulation. The wearable device achieves an offline classification accuracy of 88.33\%\ across six distinct movement-force classes for hand gestures by using a CNN-LSTM model, while real-world experiments involving five participants yield a practical accuracy of 83.33\%\ with an average system response time of 1.2 seconds. In Human-Robot synergy in navigation and grasping tasks, the robot achieved a 98\%\ task success rate with an average trajectory deviation of only 3.6 cm. Finally, the wearable-controlled mobile manipulator system achieved a 93.3\%\ gripping success rate, a transfer success of 95.6\%\, and a full-task success rate of 91.1\%\ during object grasping and transfer tests, in which a total of 9 object-texture combinations were evaluated. These three experiments' results validate the effectiveness of MEMS-based wearable sensing combined with multi-sensor fusion for reliable and intuitive control of assistive robots in smart home scenarios.
Abstract (translated)
这篇论文提出了一种用于智能智能家居辅助的可穿戴设备控制移动机械臂系统，集成了MEMS电容式麦克风、IMU传感器、振动电机和压力反馈来增强人机交互。该可穿戴装置捕捉前臂肌肉活动，并将其转换为实时控制信号以进行移动操作。通过使用CNN-LSTM模型，该可穿戴装置在六种不同的手势力度分类上实现了88.33%的离线分类准确率，而涉及五名参与者的实际实验中获得了83.33%的实际准确性，平均系统响应时间为1.2秒。在导航和抓取任务中的人机协同作用中，机器人达到了98%的任务成功率，并且平均轨迹偏差仅为3.6厘米。最后，在物体抓取和转移测试中，可穿戴控制的移动机械臂系统实现了93.3%的成功夹持率、95.6%的成功传输率以及91.1%的整体任务成功率，共评估了九种不同的物体纹理组合。这三项实验的结果验证了基于MEMS的可穿戴传感技术结合多传感器融合在智能家庭场景中实现辅助机器人可靠且直观控制的有效性。
URL
https://arxiv.org/abs/2504.13370
