Abstract
Multi-hop QA benchmarks frequently reward Large Language Models (LLMs) for spurious correctness, masking ungrounded or flawed reasoning steps. To shift toward rigorous reasoning, we propose SAFE, a dynamic benchmarking framework that replaces the ungrounded Chain-of-Thought (CoT) with a strictly verifiable sequence of grounded entities. Our framework operates across two phases: (1) train-time verification, where we establish an atomic error taxonomy and a Knowledge Graph (KG)-grounded verification pipeline to eliminate noisy supervision in standard benchmarks, identifying up to 14% of instances as unanswerable, and (2) inference-time verification, where a feedback model trained on this verified dataset dynamically detects ungrounded steps in real-time. Experimental results demonstrate that SAFE not only exposes the critical flaws of existing benchmarks at train-time, but also significantly outperforms standard baselines, achieving an average accuracy gain of 8.4 pp while guaranteeing verifiable trajectories at inference-time.
Abstract (translated)
多跳问答基准测试经常奖励大语言模型（LLMs）的虚假正确性，掩盖了其缺乏依据或存在缺陷的推理步骤。为转向严谨推理，我们提出了SAFE——一个动态基准测试框架，该框架用严格可验证的可验证实体序列取代了无依据的思维链（CoT）。我们的框架包含两个阶段：（1）训练时验证，在此阶段我们建立了原子级错误分类体系和一个基于知识图谱（KG）的验证流程，以消除标准基准中的噪声监督，识别出高达14%的不可回答实例；（2）推理时验证，在此阶段一个在此已验证数据集上训练的反馈模型能实时动态检测无依据的步骤。实验结果表明，SAFE不仅在训练时暴露了现有基准的关键缺陷，而且在推理时显著超越标准基线，实现了平均8.4个百分点的准确率提升，同时保证了可验证的推理轨迹。
URL
https://arxiv.org/abs/2604.01993
