Abstract
A central challenge in large language model inference is the trade-off between generation speed and output quality. Autoregressive models produce high-quality text but generate tokens sequentially. Diffusion models can generate tokens in parallel but often need many iterations to match the same quality. We propose planned diffusion, a hybrid method that combines the strengths of both paradigms. Planned diffusion works in two stages: first, the model creates a short autoregressive plan that breaks the output into smaller, independent spans. Second, the model generates these spans simultaneously using diffusion. This approach expands the speed-quality Pareto frontier and provides a practical path to faster, high-quality text generation. On AlpacaEval, a suite of 805 instruction-following prompts, planned diffusion achieves Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win rate, respectively. Our sensitivity analysis shows that the planning mechanism of planned diffusion is minimal and reliable, and simple runtime knobs exist to provide flexible control of the quality-latency trade-off.
Abstract (translated)
大型语言模型推理中一个核心挑战是在生成速度和输出质量之间的权衡。自回归模型能够产生高质量的文本，但其生成令牌的过程是顺序进行的；而扩散模型可以并行生成令牌，但却常常需要多次迭代才能达到相同的质量水平。我们提出了一种名为“计划式扩散”的混合方法，该方法结合了这两种范式的优点。 计划式扩散的工作流程分为两个阶段：首先，模型创建一个简短的自回归计划，将输出分解为较小且独立的小片段；其次，在这一规划的基础上，模型同时生成这些小片段。这种方法扩展了速度与质量之间的帕累托前沿，并提供了一条实现更快、高质量文本生成的实际路径。 在包含805个指令跟随提示的AlpacaEval测试集上，计划式扩散实现了质量和延迟之间的帕累托最优权衡，在自回归生成的基础上分别取得了1.27倍到1.81倍的速度提升，同时仅以质量胜率减少0.87%至5.4%为代价。我们的敏感性分析表明，计划式扩散的规划机制是微小且可靠的，并存在一些简单的运行时控制参数来灵活地调控质量和延迟之间的权衡。 通过这种方法，大型语言模型能够更高效地生成高质量文本，在确保输出质量的同时显著提高生成速度。
URL
https://arxiv.org/abs/2510.18087
