Abstract
Understanding user enjoyment is crucial in human-robot interaction (HRI), as it can impact interaction quality and influence user acceptance and long-term engagement with robots, particularly in the context of conversations with social robots. However, current assessment methods rely solely on self-reported questionnaires, failing to capture interaction dynamics. This work introduces the Human-Robot Interaction Conversational User Enjoyment Scale (HRI CUES), a novel scale for assessing user enjoyment from an external perspective during conversations with a robot. Developed through rigorous evaluations and discussions of three annotators with relevant expertise, the scale provides a structured framework for assessing enjoyment in each conversation exchange (turn) alongside overall interaction levels. It aims to complement self-reported enjoyment from users and holds the potential for autonomously identifying user enjoyment in real-time HRI. The scale was validated on 25 older adults' open-domain dialogue with a companion robot that was powered by a large language model for conversations, corresponding to 174 minutes of data, showing moderate to good alignment. Additionally, the study offers insights into understanding the nuances and challenges of assessing user enjoyment in robot interactions, and provides guidelines on applying the scale to other domains.
Abstract (translated)
理解用户的喜爱在人机交互（HRI）中至关重要，因为它可能会影响交互质量和影响用户对机器的接受程度以及与机器的长期参与，特别是在与社交机器人的对话中。然而，目前的评估方法仅依赖自我报告问卷，无法捕捉交互动态。这项工作介绍了一个名为人机交互聊天机器人用户喜爱量表（HRI CUES）的新量表，用于从外部角度评估用户在机器人对话中的喜爱。通过与具有相关专业知识的三位注释者的深入讨论和严格的评估，该量表构建了一个结构化的框架，用于评估每个对话交流（回合）的喜爱程度以及整个交互水平。该量表旨在补充来自用户的自我报告喜爱，并具有在实时HRI中自动识别用户喜好的潜力。 该量表在25名年龄较大的成年人与一台由大型语言模型驱动的伴侣机器人进行开放领域的对话上进行了验证，对话持续了174分钟，显示出中等至良好的相关性。此外，这项研究揭示了评估用户喜爱在机器人交互中的细微问题和挑战，并为其他领域提供了应用该量表的指导。
URL
https://arxiv.org/abs/2405.01354
