Abstract
Long-range Human-Robot Interaction (HRI) remains underexplored. Within it, Command Source Identification (CSI) - determining who issued a command - is especially challenging due to multi-user and distance-induced sensor ambiguity. We introduce HiSync, an optical-inertial fusion framework that treats hand motion as binding cues by aligning robot-mounted camera optical flow with hand-worn IMU signals. We first elicit a user-defined (N=12) gesture set and collect a multimodal command gesture dataset (N=38) in long-range multi-user HRI scenarios. Next, HiSync extracts frequency-domain hand motion features from both camera and IMU data, and a learned CSINet denoises IMU readings, temporally aligns modalities, and performs distance-aware multi-window fusion to compute cross-modal similarity of subtle, natural gestures, enabling robust CSI. In three-person scenes up to 34m, HiSync achieves 92.32% CSI accuracy, outperforming the prior SOTA by 48.44%. HiSync is also validated on real-robot deployment. By making CSI reliable and natural, HiSync provides a practical primitive and design guidance for public-space HRI.
Abstract (translated)
长距离人机交互（HRI）领域尚待探索。其中，命令来源识别（CSI）——即确定哪位用户发出的指令——由于多用户环境和因距离导致的传感器模糊性而尤为具有挑战性。我们提出了一种光学-惯性融合框架HiSync，该框架通过将机器人安装摄像头捕捉到的手部运动光流与佩戴在手上的IMU信号对齐来处理手势绑定线索。 首先，我们从一组12名用户中定义并征集了一系列自定义手势，并收集了长距离多用户HRI场景下的跨模态命令手势数据集（N=38）。接下来，HiSync框架会分别从摄像头和IMU数据中提取频域下手部运动特征；通过CSINet算法对IMU读数进行去噪处理并完成不同传感器的时间同步工作，并采用具有距离感知能力的多窗口融合技术计算细微自然手势间的跨模态相似度，从而实现可靠的CSI。 在三人场景（最远34米）中，HiSync框架实现了92.32%的CSI准确性，比之前的最佳解决方案高出48.44%。此外，HiSync还在实际机器人部署中进行了验证。通过确保CSI既可靠又自然，HiSync为公共场所的人机交互提供了一种实用的基础技术，并对未来的系统设计提出了指导性建议。
URL
https://arxiv.org/abs/2603.11809
