Abstract
During multi-party interactions, gaze direction is a key indicator of interest and intent, making it essential for social robots to direct their attention appropriately. Understanding the social context is crucial for robots to engage effectively, predict human intentions, and navigate interactions smoothly. This study aims to develop an empirical motion-time pattern for human gaze behavior in various social situations (e.g., entering, leaving, waving, talking, and pointing) using deep neural networks based on participants' data. We created two video clips-one for a computer screen and another for a virtual reality headset-depicting different social scenarios. Data were collected from 30 participants: 15 using an eye-tracker and 15 using an Oculus Quest 1 headset. Deep learning models, specifically Long Short-Term Memory (LSTM) and Transformers, were used to analyze and predict gaze patterns. Our models achieved 60% accuracy in predicting gaze direction in a 2D animation and 65% accuracy in a 3D animation. Then, the best model was implemented onto the Nao robot; and 36 new participants evaluated its performance. The feedback indicated overall satisfaction, with those experienced in robotics rating the models more favorably.
Abstract (translated)
在多主体互动中，目光的方向是兴趣和意图的关键指标，使社会机器人能够适当关注这一点变得至关重要。理解社交环境对于机器人的有效参与、预测人类意图以及顺畅地进行交互来说是至关重要的。这项研究旨在使用基于参与者数据的深度神经网络开发出一套经验性运动时间模式，以描述人在各种社交情况下的目光行为（例如：进入、离开、挥手、对话和指物）。我们创建了两个视频片段——一个用于计算机屏幕，另一个用于虚拟现实头盔——来展示不同的社交场景。从30名参与者那里收集数据：15人使用眼动仪，另外15人则使用Oculus Quest 1 头显。研究中运用了深度学习模型（特别是长短期记忆网络LSTM和Transformer）来分析并预测目光模式。我们的模型在2D动画中的目光方向预测准确率为60%，而在3D动画中的准确率达到了65%。然后，最佳模型被部署到了Nao机器人上，并由36名新参与者对其表现进行了评估。反馈表明总体满意度较高，有机器人技术经验的人对该模型的评价更为积极。
URL
https://arxiv.org/abs/2602.10946
