Abstract
As Large Language Models (LLMs) are deployed more widely, customization with respect to vocabulary, style and character becomes more important. In this work we introduce model arithmetic, a novel inference framework for composing and biasing LLMs without the need for model (re)training or highly specific datasets. In addition, the framework allows for more precise control of generated text than direct prompting and prior controlled text generation (CTG) techniques. Using model arithmetic, we can express prior CTG techniques as simple formulas and naturally extend them to new and more effective formulations. Further, we show that speculative sampling, a technique for efficient LLM sampling, extends to our setting. This enables highly efficient text generation with multiple composed models with only marginal overhead over a single model. Our empirical evaluation demonstrates that model arithmetic allows fine-grained control of generated text while outperforming state-of-the-art on the task of toxicity reduction.
Abstract (translated)
随着大型语言模型（LLMs）的部署范围越来越广泛，对于词汇、风格和字符的定制就变得更加重要。在这项工作中，我们引入了模型代数，一种新的用于合成和调整LLM的新推理框架，无需进行模型（重新）训练或高度特定的数据集。此外，该框架比直接提示和受控文本生成（CTG）技术更精确地控制生成的文本。使用模型代数，我们可以将先前的CTG技术表示为简单的公式，并自然地将它们扩展到新的更有效的形式。此外，我们还证明了探索性采样技术（一种高效的LLM抽样技术）扩展到我们设置中。这使得仅使用单个模型即可生成具有微小开销的多组合模型。我们的实证评估表明，模型代数允许在生成文本的同时精细地控制文本，并在毒性降低任务中优于最先进的模型。
URL
https://arxiv.org/abs/2311.14479
