Abstract
One of the main goals of robotics and intelligent agent research is to enable natural communication with humans in physically situated settings. While recent work has focused on verbal modes such as language and speech, non-verbal communication is crucial for flexible interaction. We present a framework for generating pointing gestures in embodied agents by combining imitation and reinforcement learning. Using a small motion capture dataset, our method learns a motor control policy that produces physically valid, naturalistic gestures with high referential accuracy. We evaluate the approach against supervised learning and retrieval baselines in both objective metrics and a virtual reality referential game with human users. Results show that our system achieves higher naturalness and accuracy than state-of-the-art supervised models, highlighting the promise of imitation-RL for communicative gesture generation and its potential application to robots.
Abstract (translated)
机器人和智能代理研究的主要目标之一是实现与人类在物理环境中的自然沟通。虽然最近的工作主要集中在语言和语音等口头模式上，但非言语交流对于灵活互动至关重要。我们提出了一种结合模仿学习和强化学习的框架，用于生成具身化代理人的指向手势。通过使用一个小规模的动作捕捉数据集，我们的方法能够学习一种运动控制策略，该策略可以产生物理上有效且自然的手势，并具有很高的指代准确性。我们在客观指标以及与人类用户进行虚拟现实指代游戏中的基准测试中评估了这种方法相对于监督学习和检索基线的表现。实验结果显示，我们系统在自然度和准确性方面均优于最新的监督模型，这凸显了模仿-强化学习在生成沟通手势方面的潜力及其在未来机器人应用中的潜在价值。
URL
https://arxiv.org/abs/2509.12507
