Abstract
Interactive reinforcement learning (IRL) extends traditional reinforcement learning (RL) by allowing an agent to interact with parent-like trainers during a task. In this paper, we present an IRL approach using dynamic audio-visual input in terms of vocal commands and hand gestures as feedback. Our architecture integrates multi-modal information to provide robust commands from multiple sensory cues along with a confidence value indicating the trustworthiness of the feedback. The integration process also considers the case in which the two modalities convey incongruent information. Additionally, we modulate the influence of sensory-driven feedback in the IRL task using goal-oriented knowledge in terms of contextual affordances. We implement a neural network architecture to predict the effect of performed actions with different objects to avoid failed-states, i.e., states from which it is not possible to accomplish the task. In our experimental setup, we explore the interplay of multimodal feedback and task-specific affordances in a robot cleaning scenario. We compare the learning performance of the agent under four different conditions: traditional RL, multi-modal IRL, and each of these two setups with the use of contextual affordances. Our experiments show that the best performance is obtained by using audio-visual feedback with affordancemodulated IRL. The obtained results demonstrate the importance of multi-modal sensory processing integrated with goal-oriented knowledge in IRL tasks.
Abstract (translated)
交互式强化学习（IRL）通过允许代理在任务期间与父母类培训师交互来扩展传统强化学习（RL）。在本文中，我们提出了一种使用动态视听输入的IRL方法，在声乐命令和手势方面作为反馈。我们的架构集成了多模态信息，可提供来自多个感知线索的强大命令，以及表示反馈可信度的置信度值。整合过程还考虑了两种方式传达不一致信息的情况。此外，我们使用面向目标的知识来调节感官驱动的反馈在IRL任务中的影响。我们实现神经网络体系结构来预测用不同对象执行的动作的效果，以避免失败状态，即，不可能完成任务的状态。在我们的实验设置中，我们探索了机器人清洁场景中多模态反馈和任务特定功能的相互作用。我们在四种不同的条件下比较了代理的学习性能：传统的RL，多模式IRL，以及这两种设置中的每一种都使用了上下文可供性。我们的实验表明，通过使用具有可供调节的IRL的视听反馈获得最佳性能。获得的结果证明了多模态感知处理在IRL任务中与目标导向知识相结合的重要性。
URL
https://arxiv.org/abs/1807.09991
