Abstract
LLMs are widely used, yet they remain prone to factual errors that erode user trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We first formalize SA through the lenses of selective risk and coverage. We then propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims (short, self-contained statements each expressing a single fact) and replaces uncertain atoms with higher confidence, less specific abstractions. To evaluate this framework, we develop a novel end-to-end pipeline for open-ended generation that instantiates risk as factual correctness and measures coverage using an information-theoretic measure of retained information. Across six open-source models on the FactScore and LongFact-Objects benchmarks, atom-wise SA consistently outperforms existing baselines, improving the area under the risk-coverage curve (AURC) by up to 27.73% over claim removal, demonstrating that reducing specificity can boost accuracy and reliability while preserving most of their original meaning.
Abstract (translated)
大型语言模型（LLMs）被广泛应用，但仍容易出现事实性错误，这会削弱用户信任并限制其在高风险环境中的应用。一种缓解这一问题的方法是为模型配备不确定性估计机制，在信心不足时拒绝生成内容。然而，在长文本场景中这种“要么全有要么全无”的方法过于严格，往往导致有价值的资讯被舍弃。为此，我们引入了选择性抽象（Selective Abstraction, SA）框架，该框架允许LLMs通过选择性减少不确定内容的细节来在具体性和可靠性之间进行权衡。 首先，我们从选择风险和覆盖率的角度对SA进行了形式化定义。然后，提出了原子级选择性抽象（Atom-wise Selective Abstraction），这是一种基于声明级别的实现方式，它将响应分解成原子声明（即每一个表达单一事实的简短、独立的信息单元），并用更高置信度但更不具体的抽象信息替换不确定的原子。 为了评估这一框架的有效性，我们开发了一种新颖的端到端生成管道，在这种管道中，风险被定义为事实准确性，而覆盖率则通过一种基于信息论的方法来衡量保留下来的信息量。在六款开源模型上，针对FactScore和LongFact-Objects基准测试数据集进行了实验，原子级SA始终优于现有基线方法，并将风险-覆盖曲线下面积（AURC）提高了最多27.73%，优于声明删除法的表现。这表明减少具体性可以在保持大部分原始含义的同时提高准确性和可靠性。
URL
https://arxiv.org/abs/2602.11908
