Abstract
Robots that interact with humans must adapt to individual users' preferences to operate effectively in human-centered environments. An intuitive and effective technique to learn non-expert users' preferences is through rankings of robot behaviors, e.g., trajectories, gestures, or voices. Existing techniques primarily focus on generating queries that optimize preference learning outcomes, such as sample efficiency or final preference estimation accuracy. However, the focus on outcome overlooks key user expectations in the process of providing these rankings, which can negatively impact users' adoption of robotic systems. This work proposes the Covariance Matrix Adaptation Evolution Strategies with Information Gain (CMA-ES-IG) algorithm. CMA-ES-IG explicitly incorporates user experience considerations into the preference learning process by suggesting perceptually distinct and informative trajectories for users to rank. We demonstrate these benefits through both simulated studies and real-robot experiments. CMA-ES-IG, compared to state-of-the-art alternatives, (1) scales more effectively to higher-dimensional preference spaces, (2) maintains computational tractability for high-dimensional problems, (3) is robust to noisy or inconsistent user feedback, and (4) is preferred by non-expert users in identifying their preferred robot behaviors. This project's code is available at this http URL
Abstract (translated)
与人类互动的机器人必须适应个别用户的偏好，才能在以人为中心的环境中有效运行。学习非专家用户偏好的直观且有效的技术之一是通过机器人的行为排名来进行，例如路径、手势或语音等。现有技术主要关注于生成优化偏好学习结果的查询，比如样本效率或者最终的偏好估计精度。然而，只关注结果忽视了用户提供这些排序过程中的重要期望，这可能会影响用户对机器人系统的采用。 本文提出了一种算法——协方差矩阵适应进化策略与信息增益（Covariance Matrix Adaptation Evolution Strategies with Information Gain, CMA-ES-IG）。CMA-ES-IG通过建议用户可以感知并具有信息量的路径来明确地将用户体验考虑纳入偏好学习过程中，使用户能够对其进行排名。我们通过模拟研究和真实机器人的实验展示了这种方法的好处。 与最先进的替代方案相比，CMA-ES-IG（1）更有效地扩展到更高的维度偏好数量空间；（2）对于高维问题保持计算可行性；（3）对嘈杂或不一致的用户反馈具有鲁棒性；（4）在识别非专家用户的优选机器人行为方面更受用户青睐。该项目的代码可在提供的网址中获取。 原文链接：请将“this http URL”替换为实际的有效URL，以访问项目代码。
URL
https://arxiv.org/abs/2603.09011
