Abstract
This paper presents a novel human-robot interaction (HRI) framework that enables intuitive gesture-driven control through a capacitance-based woven tactile skin. Unlike conventional interfaces that rely on panels or handheld devices, the woven tactile skin integrates seamlessly with curved robot surfaces, enabling embodied interaction and narrowing the gap between human intent and robot response. Its woven design combines fabric-like flexibility with structural stability and dense multi-channel sensing through the interlaced conductive threads. Building on this capability, we define a gesture-action mapping of 14 single- and multi-touch gestures that cover representative robot commands, including task-space motion and auxiliary functions. A lightweight convolution-transformer model designed for gesture recognition in real time achieves an accuracy of near-100%, outperforming prior baseline approaches. Experiments on robot arm tasks, including pick-and-place and pouring, demonstrate that our system reduces task completion time by up to 57% compared with keyboard panels and teach pendants. Overall, our proposed framework demonstrates a practical pathway toward more natural and efficient embodied HRI.
Abstract (translated)
本文提出了一种新颖的人机交互（HRI）框架，该框架通过基于电容的编织触觉皮肤实现了直观的手势驱动控制。与依赖于面板或手持设备的传统界面不同，这种编织触觉皮肤可以无缝集成到机器人曲面表面中，从而实现具身化互动，并缩小了人类意图和机器人响应之间的差距。其编织设计结合了类似织物的灵活性、结构稳定性以及通过交织导电线获得的密集多通道感应能力。 在此基础上，我们定义了一个由14种单触点及多触点手势组成的动作映射，涵盖了包括任务空间运动和辅助功能在内的代表性机器人指令。为了实现实时的手势识别，我们设计了一种轻量级卷积-变压器模型，并且该模型达到了接近百分之百的准确率，在实时性能上优于之前的方法。 通过在机器人手臂任务中的实验（如取放物品和倒水），我们的系统与键盘面板以及教学挂件相比，在完成这些任务的时间上最多减少了57%。总的来说，我们提出的框架为实现更加自然且高效的具身化HRI提供了一条实用途径。
URL
https://arxiv.org/abs/2509.25951
