Abstract
Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation, and a simple geometric model based on shoulder-wrist extension to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating tool accuracy, the tool is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over tool limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: this https URL.
Abstract (translated)
指向手势是人机协作中用于各种任务的一种常见交互方式，从选择目标到指导工业流程都适用。这项研究提出了一种在平面工作空间内定位指向目标的方法。该方法采用姿态估计，并基于肩腕延伸的简单几何模型，从RGB-D流中提取手势数据。本研究提出了评估指向手势和目标选择的严格方法及全面分析，在典型的机器人任务中进行验证。除了评价工具的准确性外，还将该工具集成到一个概念验证型机器人系统中，包括物体检测、语音转录和语音合成功能，以展示在协作应用中多模态数据的融合。最后，讨论了工具的局限性和性能，以便理解它在多模态机器人系统中的角色。所有开发成果都可在以下网址获取：this https URL。
URL
https://arxiv.org/abs/2506.22116
