Abstract
Recent large language models (LLMs) achieve impressive performance in source-conditioned text generation but often fail to correctly provide fine-grained attributions for their outputs, undermining verifiability and trust. Moreover, existing attribution methods do not explain how and why models leverage the provided source documents to generate their final responses, limiting interpretability. To overcome these challenges, we introduce a modular generation framework, GenerationPrograms, inspired by recent advancements in executable "code agent" architectures. Unlike conventional generation methods that simultaneously generate outputs and attributions or rely on post-hoc attribution, GenerationPrograms decomposes the process into two distinct stages: first, creating an executable program plan composed of modular text operations (such as paraphrasing, compression, and fusion) explicitly tailored to the query, and second, executing these operations following the program's specified instructions to produce the final response. Empirical evaluations demonstrate that GenerationPrograms significantly improves attribution quality at both the document level and sentence level across two long-form question-answering tasks and a multi-document summarization task. We further demonstrate that GenerationPrograms can effectively function as a post-hoc attribution method, outperforming traditional techniques in recovering accurate attributions. In addition, the interpretable programs generated by GenerationPrograms enable localized refinement through modular-level improvements that further enhance overall attribution quality.
Abstract (translated)
最近的大规模语言模型（LLM）在基于源文档的文本生成方面表现出色，但常常无法正确地为其输出提供细粒度的归属说明，这削弱了验证性和信任。此外，现有的归属方法未能解释模型如何以及为何利用提供的源文件来生成最终响应，从而限制了可解释性。为克服这些挑战，我们引入了一种模块化生成框架——GenerationPrograms，该框架借鉴了最近可执行“代码代理”架构的进展。与传统的生成方法同时生成输出和归属或依赖于事后归属不同，GenerationPrograms将过程分解成两个独立阶段：首先创建一个由模板块文本操作（如改写、压缩和融合）组成的可执行程序计划，这些操作明确针对查询进行了定制；其次根据该程序的指定指令执行这些操作以产生最终响应。实证评估表明，在两项长形式问答任务和一项多文档摘要任务中，GenerationPrograms显著提高了文档级别和句子级别的归属质量。此外，我们还展示了GenerationPrograms可以有效充当事后归属方法，超越传统技术准确恢复归属的能力。生成程序的可解释性使通过模块级改进实现局部精炼成为可能，并进一步提升了整体归属质量。
URL
https://arxiv.org/abs/2506.14580
