Abstract
We introduce \textbf{Evo}, a duality latent trajectory model that bridges autoregressive (AR) and diffusion-based language generation within a continuous evolutionary generative framework. Rather than treating AR decoding and diffusion generation as separate paradigms, Evo reconceptualizes text generation as a latent flow: each token is associated with a vector-valued embedding that evolves over a progression variable $t_i \in [0, 1]$, indicating its semantic maturity. Low $t_i$ values correspond to confident AR-like refinement, while high values invoke diffusion-style planning, allowing the model to adaptively balance AR and diffusion based on uncertainty. Theoretically, we show that both AR and diffusion models emerge as discretizations of a shared probability flow, and we derive Evo's training objective from a unified variational ELBO. The model is implemented as a time-conditioned Transformer governed by a shared vector field, trained end-to-end to jointly infer latent codes and their progression times. During decoding, Evo performs efficient, semantics-aware refinement, achieving high-quality outputs without sacrificing speed. Empirically, Evo 8B achieves state-of-the-art or highly competitive results on 15 diverse benchmarks, including reasoning (GSM8K, ARC-C), code generation (HumanEval, MBPP), and general language understanding, while maintaining fast inference speed. Our results demonstrate that Evo delivers a new paradigm for LLM design with strong generation quality, robust symbolic reasoning, and decoding efficiency.
Abstract (translated)
我们介绍了一种新的模型——\textbf{Evo}，它是一种连续进化生成框架中的二元潜在轨迹模型，用于连接自回归（AR）和基于扩散的语言生成。Evo 不是将 AR 解码和扩散生成视为独立的范式，而是重新构思文本生成为一个潜在流：每个词与一个矢量值嵌入相关联，该嵌入随时间变量 $t_i \in [0, 1]$ 进化，表示其语义成熟度。低 $t_i$ 值对应于自信的 AR 类似精炼过程，而高值则激发扩散风格规划，使模型能够根据不确定性灵活地在自回归和基于扩散的方法之间进行切换。 从理论上讲，我们展示了自回归和扩散模型都是共享概率流的一种离散化，并且我们从统一的变分ELBO中导出了Evo的训练目标。该模型通过一个受时间条件限制并由共同矢量场控制的Transformer实现，在端到端训练过程中同时推理潜在代码及其进度时间。 在解码阶段，Evo 实现了高效的、语义感知的精炼过程，可以在不牺牲速度的情况下生成高质量输出。实验表明，Evo 8B 在包括逻辑推理（GSM8K, ARC-C）、代码生成（HumanEval, MBPP）和通用语言理解在内的15个多样化基准测试中取得了当前最佳或极具竞争力的结果，并且保持了快速的推断速度。 我们的结果证明，Evo 提供了一种全新的大型语言模型设计范式，具有强大的生成质量、稳健的符号推理能力和高效的解码效率。
URL
https://arxiv.org/abs/2603.06617
