Abstract
Situationally-aware artificial agents operating with competence in natural environments face several challenges: spatial awareness, object affordance detection, dynamic changes and unpredictability. A critical challenge is the agent's ability to identify and monitor environmental elements pertinent to its objectives. Our research introduces a neurosymbolic modular architecture for reactive robotics. Our system combines a neural component performing object recognition over the environment and image processing techniques such as optical flow, with symbolic representation and reasoning. The reasoning system is grounded in the embodied cognition paradigm, via integrating image schematic knowledge in an ontological structure. The ontology is operatively used to create queries for the perception system, decide on actions, and infer entities' capabilities derived from perceptual data. The combination of reasoning and image processing allows the agent to focus its perception for normal operation as well as discover new concepts for parts of objects involved in particular interactions. The discovered concepts allow the robot to autonomously acquire training data and adjust its subsymbolic perception to recognize the parts, as well as making planning for more complex tasks feasible by focusing search on those relevant object parts. We demonstrate our approach in a simulated world, in which an agent learns to recognize parts of objects involved in support relations. While the agent has no concept of handle initially, by observing examples of supported objects hanging from a hook it learns to recognize the parts involved in establishing support and becomes able to plan the establishment/destruction of the support relation. This underscores the agent's capability to expand its knowledge through observation in a systematic way, and illustrates the potential of combining deep reasoning [...].
Abstract (translated)
情景感知型人工智能代理在自然环境中操作时面临若干挑战：空间意识、物体可利用性检测、动态变化和不可预测性。其中关键的一个挑战是代理能够识别并监控与其目标相关联的环境元素的能力。我们的研究介绍了一种用于反应式机器人的神经符号模块化架构。该系统结合了执行对象识别及图像处理技术（如光流）的神经网络组件，以及基于本体论结构整合图式知识的符号表示和推理机制。 本体论在此被操作性地用来创建感知系统的查询、决定行动，并从感知数据中推断实体的能力。这种推理与图像处理相结合的方法使代理能够聚焦于正常运行所需的感知任务，同时还能发现参与特定交互的部分物体的新概念。所发现的概念允许机器人自主获取训练数据并调整其亚符号感知以识别这些部分；这使得通过集中搜索相关对象部件来进行更复杂任务的规划成为可能。 我们在一个模拟环境中展示了我们的方法：代理学会识别支持关系中涉及的对象部分。尽管一开始没有“把手”的概念，但通过观察挂在钩子上的支撑物体的例子，它学会了识别建立和支持关系的相关部分，并能够计划这种关系的确立或破坏。这强调了代理人系统地扩展其知识的能力，并展示了将深度推理与感知相结合的潜力。
URL
https://arxiv.org/abs/2507.20832
