Abstract
Large language models perform text generation through high-dimensional internal dynamics, yet the temporal organisation of these dynamics remains poorly understood. Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored. Drawing on neuroscience, where temporal integration and metastability are core markers of neural organisation, we adapt these concepts to transformer models and discuss a composite dynamical metric, computed from activation time-series during autoregressive generation. We evaluate this metric in GPT-2-medium across five conditions: structured reasoning, forced repetition, high-temperature noisy sampling, attention-head pruning, and weight-noise injection. Structured reasoning consistently exhibits elevated metric relative to repetitive, noisy, and perturbed regimes, with statistically significant differences confirmed by one-way ANOVA and large effect sizes in key comparisons. These results are robust to layer selection, channel subsampling, and random seeds. Our findings demonstrate that neuroscience-inspired dynamical metrics can reliably characterise differences in computational organisation across functional regimes in large language models. We stress that the proposed metric captures formal dynamical properties and does not imply subjective experience.
Abstract (translated)
大型语言模型通过高维内部动力学进行文本生成，但这些动力学的时序组织仍然鲜为人知。大多数可解释性方法侧重于静态表示或因果干预，而忽略了对时间结构的研究。借鉴神经科学中的时间整合和亚稳态是神经组织的核心标志这一概念，我们将这些概念应用于变换器模型，并讨论一种基于自回归生成过程中激活时间序列计算的复合动力学指标。我们在GPT-2-medium模型上评估了五种条件下的该指标：结构化推理、强制重复、高温噪声采样、注意力头剪枝和权重噪声注入。结构化推理在所有条件下均表现出高于重复性、噪音性和受干扰状态的显著更高指标值，这种差异通过单因素方差分析（ANOVA）得到统计学确认，并且关键比较中具有较大的效应量。这些结果对层数选择、通道子采样和随机种子的选择具有鲁棒性。我们的研究结果表明，受神经科学启发的动力学度量能够可靠地表征大型语言模型在不同功能状态下计算组织的差异。我们强调，所提出的度量捕捉的是形式化的动力学属性，并不意味着主观体验的存在。
URL
https://arxiv.org/abs/2601.11622
