Abstract
Large Language Models (LLMs) demonstrate impressive natural language capabilities but often struggle with knowledge-intensive reasoning tasks. Knowledge Base Question Answering (KBQA), which leverages structured Knowledge Graphs (KGs) exemplifies this challenge due to the need for accurate multi-hop reasoning. Existing approaches typically perform sequential reasoning steps guided by predefined pipelines, restricting flexibility and causing error cascades due to isolated reasoning at each step. To address these limitations, we propose KG-Hopper, a novel Reinforcement Learning (RL) framework that empowers compact open LLMs with the ability to perform integrated multi-hop KG reasoning within a single inference round. Rather than reasoning step-by-step, we train a Reasoning LLM that embeds the entire KG traversal and decision process into a unified ``thinking'' stage, enabling global reasoning over cross-step dependencies and dynamic path exploration with backtracking. Experimental results on eight KG reasoning benchmarks show that KG-Hopper, based on a 7B-parameter LLM, consistently outperforms larger multi-step systems (up to 70B) and achieves competitive performance with proprietary models such as GPT-3.5-Turbo and GPT-4o-mini, while remaining compact, open, and data-efficient. The code is publicly available at: this https URL.
Abstract (translated)
大型语言模型（LLMs）展现出卓越的自然语言能力，但在知识密集型推理任务中仍面临挑战。知识库问答（KBQA）利用结构化知识图谱（KGs），因需进行精确的多跳推理而成为典型难题。现有方法通常依赖预定义流水线执行顺序推理步骤，限制了灵活性，并因各步骤孤立推理导致误差累积。为应对这些局限，我们提出KG-Hopper——一种新颖的强化学习（RL）框架，使紧凑的开源LLMs能够在单次推理中实现集成的多跳KG推理。我们转而训练一个推理LLM，将整个KG遍历与决策过程嵌入统一的“思考”阶段，从而支持跨步骤依赖的全局推理及带回溯的动态路径探索。在八个KG推理基准测试中，基于70亿参数LLM的KG-Hopper持续超越规模更大的多步系统（最高达700亿参数），并与GPT-3.5-Turbo和GPT-4o-mini等专有模型表现相当，同时保持模型紧凑、开源且数据高效。代码已公开：此网址。
URL
https://arxiv.org/abs/2603.21440
