Abstract
As organizations increasingly integrate AI-powered question-answering systems into financial information systems for compliance, risk assessment, and decision support, ensuring the factual accuracy of AI-generated outputs becomes a critical engineering challenge. Current Knowledge Graph (KG)-augmented QA systems lack systematic mechanisms to detect hallucinations - factually incorrect outputs that undermine reliability and user trust. We introduce FinBench-QA-Hallucination, a benchmark for evaluating hallucination detection methods in KG-augmented financial QA over SEC 10-K filings. The dataset contains 755 annotated examples from 300 pages, each labeled for groundedness using a conservative evidence-linkage protocol requiring support from both textual chunks and extracted relational triplets. We evaluate six detection approaches - LLM judges, fine-tuned classifiers, Natural Language Inference (NLI) models, span detectors, and embedding-based methods under two conditions: with and without KG triplets. Results show that LLM-based judges and embedding approaches achieve the highest performance (F1: 0.82-0.86) under clean conditions. However, most methods degrade significantly when noisy triplets are introduced, with Matthews Correlation Coefficient (MCC) dropping 44-84 percent, while embedding methods remain relatively robust with only 9 percent degradation. Statistical tests (Cochran's Q and McNemar) confirm significant performance differences (p < 0.001). Our findings highlight vulnerabilities in current KG-augmented systems and provide insights for building reliable financial information systems, where hallucinations can lead to regulatory violations and flawed decisions. The benchmark also offers a framework for integrating AI reliability evaluation into information system design across other high-stakes domains such as healthcare, legal, and government.
Abstract (translated)
随着组织越来越多地将人工智能驱动的问答系统集成到金融信息系统中，以用于合规、风险评估和决策支持，确保AI生成输出的事实准确性已成为一项关键工程挑战。当前的知识图谱增强型问答系统缺乏系统性机制来检测幻觉——即那些破坏可靠性和用户信任的事实性错误输出。我们引入了FinBench-QA-Hallucination，这是一个用于评估基于SEC 10-K申报文件的知识图谱增强型金融问答中幻觉检测方法的基准测试。该数据集包含来自300页文档的755个标注样本，每个样本均采用保守的证据链接协议进行依据性标注，要求同时得到文本片段和提取的关系三元组的支持。我们在有无知识图谱三元组的两种条件下，评估了六种检测方法：大语言模型评判器、微调分类器、自然语言推理模型、跨度检测器以及基于嵌入的方法。结果显示，在干净条件下，基于大语言模型的评判器和嵌入方法表现最佳（F1值：0.82-0.86）。然而，当引入噪声三元组时，大多数方法性能显著下降，马修斯相关系数下降44%至84%，而嵌入方法仅下降9%，表现出相对稳健性。统计检验（Cochran's Q检验和McNemar检验）证实了性能差异的显著性（p < 0.001）。我们的研究结果揭示了当前知识图谱增强系统中的漏洞，并为构建可靠的金融信息系统提供了见解——在该系统中，幻觉可能导致违规和错误决策。该基准测试还为将AI可靠性评估整合到其他高风险领域（如医疗保健、法律和政府）的信息系统设计中提供了一个框架。
URL
https://arxiv.org/abs/2603.20252
