Abstract
The ability to accurately locate and navigate to a specific object is a crucial capability for embodied agents that operate in the real world and interact with objects to complete tasks. Such object navigation tasks usually require large-scale training in visual environments with labeled objects, which generalizes poorly to novel objects in unknown environments. In this work, we present a novel zero-shot object navigation method, Exploration with Soft Commonsense constraints (ESC), that transfers commonsense knowledge in pre-trained models to open-world object navigation without any navigation experience nor any other training on the visual environments. First, ESC leverages a pre-trained vision and language model for open-world prompt-based grounding and a pre-trained commonsense language model for room and object reasoning. Then ESC converts commonsense knowledge into navigation actions by modeling it as soft logic predicates for efficient exploration. Extensive experiments on MP3D, HM3D, and RoboTHOR benchmarks show that our ESC method improves significantly over baselines, and achieves new state-of-the-art results for zero-shot object navigation (e.g., 225\% relative Success Rate improvement than CoW on MP3D).
Abstract (translated)
准确定位和导航特定对象对于存在于现实世界中的实体行为主体非常重要。这些对象导航任务通常需要在有标签的对象环境下进行大规模训练,这对于在未知环境中 novel 对象的推广效果很差。在本文中,我们提出了一种零次导航对象方法,称为“软常识约束的探索(ESC)”,该方法将预先训练的常识知识转移到无导航经验和视觉环境训练中的开放世界对象导航。首先,ESC利用开放世界条件先验框架和常识语言模型,以软逻辑条件的形式用于高效的探索。然后,ESC将常识知识转化为导航行动,将其建模为软逻辑谓词,以进行高效的探索。在MP3D、HM3D和RoboTHOR基准模型上进行广泛的实验表明,我们的ESC方法相对于基准方法显著提高了,并取得了零次导航对象方法的最新前沿结果(例如,MP3D中COW的相对成功率提高225%)。
URL
https://arxiv.org/abs/2301.13166
