Abstract
Corporate financial reports are a valuable source of structured knowledge for Knowledge Graph construction, but the lack of annotated ground truth in this domain makes evaluation difficult. We present a semi-automated pipeline for Subject-Predicate-Object triplet extraction that uses ontology-driven proxy metrics, specifically Ontology Conformance and Faithfulness, instead of ground-truth-based evaluation. We compare a static, manually engineered ontology against a fully automated, document-specific ontology induction approach across different LLMs and two corporate annual reports. The automatically induced ontology achieves 100% schema conformance in all configurations, eliminating the ontology drift observed with the manual approach. We also propose a hybrid verification strategy that combines regex matching with an LLM-as-a-judge check, reducing apparent subject hallucination rates from 65.2% to 1.6% by filtering false positives caused by coreference resolution. Finally, we identify a systematic asymmetry between subject and object hallucinations, which we attribute to passive constructions and omitted agents in financial prose.
Abstract (translated)
公司财务报告是构建知识图谱时有价值的结构化信息来源，但由于该领域缺乏标注的真实数据集，使得评估变得困难。我们提出了一种基于本体驱动的代理指标（特别是符合性和忠实性）而非基于真实数据集评价的半自动主题-谓词-客体三元组提取管道。我们在不同的大型语言模型和两份公司年度报告中比较了静态的手工设计本体与完全自动化的文档特定本体生成方法。自动化生成的本体在所有配置下均实现了100%模式符合性，消除了手动方法中的模式漂移问题。 此外，我们还提出了一种混合验证策略，结合正则表达式匹配和使用大型语言模型作为裁判进行检查的方法，通过过滤掉由共指解析引起的假阳性来降低明显主体幻觉率（从65.2%降至1.6%）。 最后，我们发现主题与客体幻觉之间存在系统性不对称现象，并将其归因于财务文本中的被动语态和省略的施动者。
URL
https://arxiv.org/abs/2602.11886
