Abstract
Prediction of road users' behaviors in the context of autonomous driving has gained considerable attention by the scientific community in the last years. Most works focus on predicting behaviors based on kinematic information alone, a simplification of the reality since road users are humans, and as such they are highly influenced by their surrounding context. In addition, a large plethora of research works rely on powerful Deep Learning techniques, which exhibit high performance metrics in prediction tasks but may lack the ability to fully understand and exploit the contextual semantic information contained in the road scene, not to mention their inability to provide explainable predictions that can be understood by humans. In this work, we propose an explainable road users' behavior prediction system that integrates the reasoning abilities of Knowledge Graphs (KG) and the expressiveness capabilities of Large Language Models (LLM) by using Retrieval Augmented Generation (RAG) techniques. For that purpose, Knowledge Graph Embeddings (KGE) and Bayesian inference are combined to allow the deployment of a fully inductive reasoning system that enables the issuing of predictions that rely on legacy information contained in the graph as well as on current evidence gathered in real time by onboard sensors. Two use cases have been implemented following the proposed approach: 1) Prediction of pedestrians' crossing actions; 2) Prediction of lane change maneuvers. In both cases, the performance attained surpasses the current state of the art in terms of anticipation and F1-score, showing a promising avenue for future research in this field.
Abstract (translated)
近年来，自动驾驶背景下预测道路使用者的行为已经引起了科学界的广泛关注。大多数工作都基于运动信息预测行为，简化现实，因为道路使用者是是人，所以他们对周围环境的影响很大。此外，大量研究作品依赖强大的深度学习技术，在预测任务中表现出高的性能指标，但可能无法完全理解并利用道路场景中的上下文语义信息，更不用说无法提供可解释的预测，让人类能够理解。在本文中，我们提出了一个可解释的道路使用者行为预测系统，通过使用检索增强生成（RAG）技术将知识图谱的推理能力和大型语言模型的表现力相结合。为此，知识图谱嵌入（KGE）和贝叶斯推理被结合使用，以便部署一个完全归纳推理系统，该系统能够基于图形中的旧信息以及车载传感器实时收集的证据发出预测。以下是根据所提出的方法实现的两个用例：1）预测行人过马路的行为；2）预测车道变更操作。在这两个用例中，取得的性能已经超越了当前的技术水平，显示了该领域未来研究的希望。
URL
https://arxiv.org/abs/2405.00449
