Abstract
Standard autoregressive language models generate text token-by-token from a fixed vocabulary, inducing a tree-structured state space when viewing token sampling as an action, which limits flexibility and expressiveness. Recent work introduces dynamic vocabulary by sampling retrieved text spans but overlooks that the same sentence can be composed of spans of varying lengths, lacking explicit modeling of the directed acyclic graph (DAG) state space. This leads to restricted exploration of compositional paths and is biased toward the chosen path. Generative Flow Networks (GFlowNets) are powerful for efficient exploring and generalizing over state spaces, particularly those with a DAG structure. However, prior GFlowNets-based language models operate at the token level and remain confined to tree-structured spaces, limiting their potential. In this work, we propose Flow of SpanS (FOSS), a principled GFlowNets framework for span generation. FoSS constructs a dynamic span vocabulary by segmenting the retrieved text flexibly, ensuring a DAG-structured state space, which allows GFlowNets to explore diverse compositional paths and improve generalization. With specialized reward models, FoSS generates diverse, high-quality text. Empirically, FoSS improves MAUVE scores by up to 12.5% over Transformer on text generation and achieves 3.5% gains on knowledge-intensive tasks, consistently outperforming state-of-the-art methods. Scaling experiments further demonstrate FoSS benefits from larger models, more data, and richer retrieval corpora, retaining its advantage over strong baselines.
Abstract (translated)
标准的自回归语言模型从固定词汇表中逐词生成文本，当把单词采样视为一个动作时，这种做法会产生树状结构的状态空间。这样的方法限制了灵活性和表达能力。最近的研究通过抽取检索到的文本片段来引入动态词汇表，但忽略了同一个句子可以由不同长度的片段组成这一事实，并且没有显式地对有向无环图（DAG）状态空间进行建模。这导致了组合路径探索受限并偏向于所选路径。 生成流网络（GFlowNets）对于有效探索和泛化处理具有DAG结构的状态空间非常有用。然而，基于GFlowNet的语言模型先前仅在词级别上操作，并且仍然局限于树状结构的空间中工作，从而限制了它们的潜力。在这项工作中，我们提出了SpanS流（FOSS），这是一种用于片段生成的原则性GFlowNets框架。FoSS通过灵活地分割检索到的文本构建动态片段词汇表，确保了DAG结构的状态空间，使GFlowNets能够探索多种组合路径并提高泛化能力。 借助专门设计的奖励模型，FoSS可以产生多样且高质量的文本。实证研究表明，在文本生成方面，FoSS相比Transformer提高了高达12.5%的MAUVE分数，并在知识密集型任务上取得了3.5%的改进，持续优于当前最先进的方法。规模实验进一步表明，随着模型更大、数据更多以及检索语料库更丰富时，FoSS能够保持其相对于强基线的优势。
URL
https://arxiv.org/abs/2602.10583
