Abstract
We introduce Spectral Generative Flow Models (SGFMs), a physics-inspired alternative to transformer-based large language models. Instead of representing text or video as sequences of discrete tokens processed by attention, SGFMs treat generation as the evolution of a continuous field governed by constrained stochastic dynamics in a multiscale wavelet basis. This formulation replaces global attention with local operators, spectral projections, and Navier--Stokes-like transport, yielding a generative mechanism grounded in continuity, geometry, and physical structure. Our framework provides three key innovations: (i) a field-theoretic ontology in which text and video are unified as trajectories of a stochastic partial differential equation; (ii) a wavelet-domain representation that induces sparsity, scale separation, and computational efficiency; and (iii) a constrained stochastic flow that enforces stability, coherence, and uncertainty propagation. Together, these components define a generative architecture that departs fundamentally from autoregressive modeling and diffusion-based approaches. SGFMs offer a principled path toward long-range coherence, multimodal generality, and physically structured inductive bias in next-generation generative models.
Abstract (translated)
我们介绍了Spectral Generative Flow Models（SGFM），这是一种受物理启发的大语言模型的替代方案，不同于基于变压器的语言模型。SGFMs 不是将文本或视频表示为通过注意力处理的一系列离散标记，而是将其视为由多尺度小波基中约束随机动力学驱动的连续场的演变过程。这一形式化方法用局部操作符、谱投影和类似于Navier-Stokes传输的过程取代了全局注意力机制，从而构建了一种基于连续性、几何和物理结构的基础生成机制。 我们的框架提供了三个关键创新： (i) 一种将文本和视频统一为随机偏微分方程轨迹的场论本体； (ii) 引入稀疏性和计算效率的小波域表示方法，并实现了尺度分离； (iii) 施加了稳定性、连贯性和不确定性传播约束的受限随机流。 这些组件共同定义了一种根本不同于自回归模型和基于扩散的方法的生成架构。SGFMs 为下一代生成模型提供了一个原则性的路径，以实现长程一致性、多模态泛化以及物理结构化的归纳偏差。
URL
https://arxiv.org/abs/2601.08893
