Abstract
The Embodied AI community has made significant strides in visual navigation tasks, exploring targets from 3D coordinates, objects, language descriptions, and images. However, these navigation models often handle only a single input modality as the target. With the progress achieved so far, it is time to move towards universal navigation models capable of handling various goal types, enabling more effective user interaction with robots. To facilitate this goal, we propose GOAT-Bench, a benchmark for the universal navigation task referred to as GO to AnyThing (GOAT). In this task, the agent is directed to navigate to a sequence of targets specified by the category name, language description, or image in an open-vocabulary fashion. We benchmark monolithic RL and modular methods on the GOAT task, analyzing their performance across modalities, the role of explicit and implicit scene memories, their robustness to noise in goal specifications, and the impact of memory in lifelong scenarios.
Abstract (translated)
实体增强AI社区在视觉导航任务方面取得了显著进展，探索了从3D坐标、物体、语言描述和图像中的目标。然而，这些导航模型通常仅处理单个输入模态作为目标。随着取得的进展，是时候朝通用导航模型迈进，这些模型能够处理各种目标类型，使机器人与用户更有效地互动。为了实现这一目标，我们提出了GOAT-Bench，一个被称为GO to AnyThing（GOAT）的通用导航任务的基准。在这个任务中，代理被指导以按照类别名称、语言描述或图像中的目标导航到一系列指定目标。我们对GOAT任务上的单元化RL和模块化方法进行了基准，分析它们在各个方面的表现，包括模态、显式和隐式场景记忆的作用、对目标规格中噪声的鲁棒性以及记忆在终身场景中的影响。
URL
https://arxiv.org/abs/2404.06609
