Abstract
Most prior research in deep imitation learning has predominantly utilized fixed cameras for image input, which constrains task performance to the predefined field of view. However, enabling a robot to actively maneuver its neck can significantly expand the scope of imitation learning to encompass a wider variety of tasks and expressive actions such as neck gestures. To facilitate imitation learning in robots capable of neck movement while simultaneously performing object manipulation, we propose a teaching system that systematically collects datasets incorporating neck movements while minimizing discomfort caused by dynamic viewpoints during teleoperation. In addition, we present a novel network model for learning manipulation tasks including active neck motion. Experimental results showed that our model can achieve a high success rate of around 90\%, regardless of the distraction from the viewpoint variations by active neck motion. Moreover, the proposed model proved particularly effective in challenging scenarios, such as when objects were situated at the periphery or beyond the standard field of view, where traditional models struggled. The proposed approach contributes to the efficiency of dataset collection and extends the applicability of imitation learning to more complex and dynamic scenarios.
Abstract (translated)
大多数关于深度模仿学习的早期研究主要采用固定摄像头输入图像，这限制了任务性能仅限于预定义的视野范围内。然而，让机器人能够主动操纵其颈部可以显著扩展模仿学习的应用范围，包括更多样化的任务和表达性动作，如颈部手势。为了促进能够在进行物体操作的同时执行颈部运动的机器人的模仿学习，我们提出了一种教学系统，该系统有系统地收集包含颈部运动的数据集，并在远程操控时尽量减少因动态视角变化带来的不适感。此外，我们还提出了一个新颖的网络模型来学习包括主动颈部动作在内的操纵任务。实验结果显示，无论是在主动颈部运动导致的视点变化中，我们的模型都能实现大约90%的成功率。此外，在一些具有挑战性的场景（如物体位于视野边缘或超出标准视野范围）下，所提出的模型表现出特别的有效性，而传统模型在这些情况下则表现不佳。我们提出的方法不仅提高了数据集收集的效率，还扩展了模仿学习在更复杂和动态场景中的应用范围。
URL
https://arxiv.org/abs/2506.17624
