Abstract
Simultaneous localization and mapping (SLAM) is a critical technology that enables autonomous robots to be aware of their surrounding environment. With the development of deep learning, SLAM systems can achieve a higher level of perception of the environment, including the semantic and text levels. However, current works are limited in their ability to achieve a natural-language level of perception of the world. To address this limitation, we propose LP-SLAM, the first language-perceptive SLAM system that leverages large language models (LLMs). LP-SLAM has two major features: (a) it can detect text in the scene and determine whether it represents a landmark to be stored during the tracking and mapping phase, and (b) it can understand natural language input from humans and provide guidance based on the generated map. We illustrated three usages of the LLM in the system including text cluster, landmark judgment, and natural language navigation. Our proposed system represents an advancement in the field of LLMs based SLAM and opens up new possibilities for autonomous robots to interact with their environment in a more natural and intuitive way.
Abstract (translated)
同步定位与地图编制(SLAM)是一种关键技术,使自主机器人能够感知其周围环境。随着深度学习的发展,SLAM系统能够感知更高级别的环境,包括语义和文本级别。然而,当前的研究能力有限,无法达到自然语言级别的感知。为了解决这个问题,我们提出了LP-SLAM,它是第一个利用大型语言模型(LLM)的Language-perceive SLAM系统。LP-SLAM有两个主要特性:(a)它可以在场景中检测文本,并确定它是否代表需要在跟踪和地图编制阶段存储的地标,(b)它可以从人类输入的自然语言中理解并基于生成的地图提供指导。我们举例说明了三个使用LLM的系统应用,包括文本簇、地标判断和自然语言导航。我们提出的系统代表了基于LLM的SLAM领域的进步,为自主机器人以更自然和直观的方式与环境交互打开了新的可能性。
URL
https://arxiv.org/abs/2303.10089
