Abstract
Nonverbal behaviors, particularly gaze direction, play a crucial role in enhancing effective communication in social interactions. As social robots increasingly participate in these interactions, they must adapt their gaze based on human activities and remain receptive to all cues, whether human-generated or not, to ensure seamless and effective communication. This study aims to increase the similarity between robot and human gaze behavior across various social situations, including both human and non-human stimuli (e.g., conversations, pointing, door openings, and object drops). A key innovation in this study, is the investigation of gaze responses to non-human stimuli, a critical yet underexplored area in prior research. These scenarios, were simulated in the Unity software as a 3D animation and a 360-degree real-world video. Data on gaze directions from 41 participants were collected via virtual reality (VR) glasses. Preprocessed data, trained two neural networks-LSTM and Transformer-to build predictive models based on individuals' gaze patterns. In the animated scenario, the LSTM and Transformer models achieved prediction accuracies of 67.6% and 70.4%, respectively; In the real-world scenario, the LSTM and Transformer models achieved accuracies of 72% and 71.6%, respectively. Despite the gaze pattern differences among individuals, our models outperform existing approaches in accuracy while uniquely considering non-human stimuli, offering a significant advantage over previous literature. Furthermore, deployed on the NAO robot, the system was evaluated by 275 participants via a comprehensive questionnaire, with results demonstrating high satisfaction during interactions. This work advances social robotics by enabling robots to dynamically mimic human gaze behavior in complex social contexts.
Abstract (translated)
非言语行为，尤其是目光方向，在社交互动中有效沟通的提升方面扮演着关键角色。随着社交机器人越来越多地参与这些互动，它们必须根据人类活动调整自己的目光，并且对所有线索保持敏感，无论该线索是人为生成还是其它来源的，以确保无缝和有效的交流。本研究旨在增加在各种社交情境下机器人与人眼动行为之间的相似性，包括人类及非人类刺激（例如对话、指向动作、门开关以及物体掉落）。 这项研究的一个关键创新点在于对非人类刺激的眼动反应进行了调查，这是一个以前的研究中相对较少探讨的重要领域。这些场景通过Unity软件作为3D动画和360度真实世界视频被模拟出来。41名参与者的目光方向数据通过虚拟现实（VR）眼镜收集。预处理后的数据训练了两个神经网络模型——LSTM和Transformer——基于个体的眼动模式构建预测模型。 在动画情景中，LSTM和Transformer模型分别实现了67.6%和70.4%的预测准确率；而在真实世界的情景中，它们分别达到了72%和71.6%。尽管个人之间存在眼动模式差异，我们的模型在精度上超越了现有方法，并且独特地考虑到了非人类刺激的影响，在此方面具有显著优势。 此外，该系统被部署到NAO机器人上，并通过一份详尽的问卷由275名参与者进行评估，结果显示在互动过程中满意度很高。这项工作推动了社交机器人的发展，使机器人能够动态模仿在复杂社会环境中的人类眼动行为。
URL
https://arxiv.org/abs/2602.11648
