Abstract
The paradigm of Large Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential ``brick-by-brick'' process. Despite their success, AR models are inherently constrained by a causal bottleneck that limits global structural foresight and iterative refinement. Diffusion Language Models (DLMs) offer a transformative alternative, conceptualizing text generation as a holistic, bidirectional denoising process akin to a sculptor refining a masterpiece. However, the potential of DLMs remains largely untapped as they are frequently confined within AR-legacy infrastructures and optimization frameworks. In this Perspective, we identify ten fundamental challenges ranging from architectural inertia and gradient sparsity to the limitations of linear reasoning that prevent DLMs from reaching their ``GPT-4 moment''. We propose a strategic roadmap organized into four pillars: foundational infrastructure, algorithmic optimization, cognitive reasoning, and unified multimodal intelligence. By shifting toward a diffusion-native ecosystem characterized by multi-scale tokenization, active remasking, and latent thinking, we can move beyond the constraints of the causal horizon. We argue that this transition is essential for developing next-generation AI capable of complex structural reasoning, dynamic self-correction, and seamless multimodal integration.
Abstract (translated)
大型语言模型（LLMs）当前的范式主要由自回归（AR）架构定义，这些架构通过一种“砖块式”的顺序过程生成文本。尽管自回归模型取得了成功，但它们本质上受到因果瓶颈的影响，这限制了其全局结构预见性和迭代优化的能力。扩散语言模型（DLMs）则提供了一种变革性的替代方案，将文本生成视为一种整体的双向去噪过程，类似于雕刻家打磨杰作的过程。然而，由于常被局限在自回归遗留架构和优化框架内，扩散语言模型的巨大潜力尚未得到充分开发。 在这篇视角文章中，我们识别出了阻碍扩散语言模型实现“GPT-4时刻”的十个根本性挑战，从架构惰性和梯度稀疏性到线性推理的限制。为了应对这些挑战，我们提出了一项战略路线图，将其组织为四个支柱：基础架构、算法优化、认知推理和统一多模态智能。通过转向以扩散原生生态系统的特征，如多层次标记化、主动重掩码和隐式思维为中心的方向，我们可以超越因果时间线的限制。 我们认为这一转变对于开发下一代AI系统至关重要，这些系统能够进行复杂的结构推理、动态自我校正，并实现无缝的多模态集成。
URL
https://arxiv.org/abs/2601.14041
