Abstract
Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV-cache compatibility, prior work commonly adopts block-based diffusion, decoding tokens block by block. However, this paradigm suffers from a structural limitation that we term Boundary-Induced Context Truncation (BICT): undecoded tokens near block boundaries are forced to commit without access to nearby future context, even when such context could substantially reduce uncertainty. This limitation degrades decoding confidence and generation quality, especially for tasks requiring precise reasoning, such as mathematical problem solving and code generation. We propose Deferred Commitment Decoding (DCD), a novel, training-free decoding strategy that mitigates this issue. DCD maintains a confidence-aware sliding window over masked tokens, resolving low-uncertainty tokens early while deferring high-uncertainty tokens until sufficient contextual evidence becomes available. This design enables effective bidirectional information flow within the decoding window without sacrificing efficiency. Extensive experiments across multiple diffusion language models, benchmarks, and caching configurations show that DCD improves generation accuracy by 1.39% with comparable time on average compared to fixed block-based diffusion methods, with the most significant improvement reaching 9.0%. These results demonstrate that deferring token commitment based on uncertainty is a simple yet effective principle for improving both the quality and efficiency of diffusion language model decoding.
Abstract (translated)
最近，扩散语言模型（DLMs）作为自回归模型的有力替代者而兴起，它们通过允许并行文本生成来实现这一点。为了提高推理效率和KV缓存兼容性，先前的工作通常采用基于块的扩散方法，即分块解码令牌。然而，这种范式存在一个我们称之为边界诱导上下文截断（BICT）的结构性限制：接近块边界的未解码令牌被迫在没有访问到附近未来上下文的情况下做出承诺，即使这样的上下文可以显著减少不确定性。这一限制降低了解码信心和生成质量，尤其是在需要精确推理的任务中，例如数学问题求解和代码生成。 我们提出了一种新的、无需训练的解码策略——延迟承诺解码（DCD），以缓解这个问题。DCD在屏蔽令牌上维护一个基于信心的滑动窗口，早期解决低不确定性令牌，而高不确定性的令牌则被推迟直到有足够的上下文证据再做出决定。这种设计使得解码窗口内的双向信息流得以有效进行，而不牺牲效率。 跨多个扩散语言模型、基准和缓存配置的广泛实验表明，在与固定块式扩散方法的时间成本相当的情况下，DCD平均将生成准确性提高了1.39%，最高可达9%的显著改进。这些结果证明了根据不确定性延迟令牌承诺是一种简单而有效的原则，可以提高扩散语言模型解码的质量和效率。
URL
https://arxiv.org/abs/2601.02076
