Abstract
Practitioners have access to an abundance of language models and prompting strategies for solving many language modeling tasks; yet prior work shows that modeling performance is highly sensitive to both choices. Classical machine learning ensembling techniques offer a principled approach: aggregate predictions from multiple sources to achieve better performance than any single one. However, applying ensembling to language models during decoding is challenging: naively aggregating next-token probabilities yields samples from a locally normalized, biased approximation of the generally intractable ensemble distribution over strings. In this work, we introduce a unified framework for composing $K$ language models into $f$-ensemble distributions for a wide range of functions $f\colon\mathbb{R}_{\geq 0}^{K}\to\mathbb{R}_{\geq 0}$. To sample from these distributions, we propose a byte-level sequential Monte Carlo (SMC) algorithm that operates in a shared character space, enabling ensembles of models with mismatching vocabularies and consistent sampling in the limit. We evaluate a family of $f$-ensembles across prompt and model combinations for various structured text generation tasks, highlighting the benefits of alternative aggregation strategies over traditional probability averaging, and showing that better posterior approximations can yield better ensemble performance.
Abstract (translated)
实践者可以访问大量语言模型和提示策略来解决许多语言建模任务；然而，先前的工作表明，模型性能对这两方面的选择非常敏感。经典机器学习的集成技术提供了一种原则性的方法：通过汇集多个来源的预测以实现比单一来源更好的性能。但是，在解码过程中将集成应用到语言模型上是具有挑战性的：简单地聚合下一个标记的概率会产生从局部归一化的、有偏差的近似分布中抽取的样本，这与通常难以处理的整体集成分布不匹配。 在这项工作中，我们介绍了一个统一框架，用于将$K$个语言模型组合成$f$-集成分布（对于一系列函数$f\colon\mathbb{R}_{\geq 0}^{K}\to\mathbb{R}_{\geq 0}$）。为了从这些分布中采样，我们提出了一种字节级别的顺序蒙特卡洛（SMC）算法，在共享字符空间上运行，这使得词汇表不匹配的模型集成和一致的采样成为可能。我们在各种结构化的文本生成任务中评估了一系列$f$-集成，并突出了与传统的概率平均策略相比，其他聚合策略的优势。我们还展示了更好的后验估计可以提高集成性能。 简单来说，这项研究提出了一种新的方法来将多个语言模型合并成一个更强大的系统，通过这种方法可以改善文本生成的质量和多样性。该框架不仅能够处理不同词汇表的模型，还可以灵活地根据不同的目标调整采样策略，从而在多种任务上表现出优越的结果。
URL
https://arxiv.org/abs/2603.05432
