Abstract
As intelligent robots become more integrated into human environments, there is a growing need for intuitive and reliable Human-Robot Interaction (HRI) interfaces that are adaptable and more natural to interact with. Traditional robot control methods often require users to adapt to interfaces or memorize predefined commands, limiting usability in dynamic, unstructured environments. This paper presents a novel framework that bridges natural language understanding and robotic execution by combining Large Language Models (LLMs) with Behavior Trees. This integration enables robots to interpret natural language instructions given by users and translate them into executable actions by activating domain-specific plugins. The system supports scalable and modular integration, with a primary focus on perception-based functionalities, such as person tracking and hand gesture recognition. To evaluate the system, a series of real-world experiments was conducted across diverse environments. Experimental results demonstrate that the proposed approach is practical in real-world scenarios, with an average cognition-to-execution accuracy of approximately 94%, making a significant contribution to HRI systems and robots. The complete source code of the framework is publicly available at this https URL.
Abstract (translated)
随着智能机器人在人类环境中的集成越来越紧密，对于直观且可靠的机器人与人交互（HRI）界面的需求也在增长。这些界面需要具备适应性和自然性以更好地进行互动。传统机器人的控制方法通常要求用户去适应特定的接口或记忆预定义的命令，这限制了其在动态、非结构化环境中的可用性。本文提出了一种新颖的框架，该框架结合大型语言模型（LLMs）与行为树，旨在将自然语言理解和机器人执行相结合。这种集成使机器人能够解读用户给出的自然语言指令，并通过激活特定领域的插件将其转化为可执行的动作。系统支持模块化和可扩展的整合，主要聚焦于感知功能，例如人员跟踪和手势识别。 为了评估该系统的性能，在各种环境进行了系列现实世界实验。实验证明了所提出的这种方法在实际场景中的实用性，平均认知到执行准确性达到约94%，这为HRI系统及机器人技术做出了重要贡献。框架的完整源代码可在以下网址公开获取：[此URL]（请将"[此URL]"替换为具体的URL）。
URL
https://arxiv.org/abs/2508.09621
