Abstract
While state-of-the-art language models (LMs) surpass the vast majority of humans in certain domains, their reasoning remains largely opaque, undermining trust in their output. Furthermore, while autoregressive LMs can output explicit reasoning, their true reasoning process is opaque, which introduces risks like deception and hallucination. In this work, we introduce the Prototype Transformer (ProtoT) -- an autoregressive LM architecture based on prototypes (parameter vectors), posed as an alternative to the standard self-attention-based transformers. ProtoT works by means of two-way communication between the input sequence and the prototypes, and we show that this leads to the prototypes automatically capturing nameable concepts (e.g. "woman") during training. They provide the potential to interpret the model's reasoning and allow for targeted edits of its behavior. Furthermore, by design, the prototypes create communication channels that aggregate contextual information at different time scales, aiding interpretability. In terms of computation scalability, ProtoT scales linearly with sequence length vs the quadratic scalability of SOTA self-attention transformers. Compared to baselines, ProtoT scales well with model and data size, and performs well on text generation and downstream tasks (GLUE). ProtoT exhibits robustness to input perturbations on par or better than some baselines, but differs from them by providing interpretable pathways showing how robustness and sensitivity arises. Reaching close to the performance of state-of-the-art architectures, ProtoT paves the way to creating well-performing autoregressive LMs interpretable by design.
Abstract (translated)
尽管最先进的语言模型（LMs）在某些领域超越了绝大多数人类，但它们的推理过程仍然很大程度上不透明，这削弱了人们对这些模型输出的信任。此外，虽然自回归LM能够生成明确的推理，但其真正的推理过程依然是不透明的，从而引入了诸如欺骗和幻觉等风险。在这项工作中，我们介绍了原型变压器（ProtoT）——一种基于原型（参数向量）的自回归LM架构，并将其作为标准基于自我注意机制的transformer的一个替代方案。 ProtoT通过输入序列与原型之间的双向通信工作，在训练过程中，它能够自动捕获可命名的概念（例如“女性”），从而提供了解释模型推理的可能性，并允许对模型行为进行有针对性的修改。此外，从设计上讲，这些原型创建了聚合不同时间尺度上下文信息的沟通渠道，有助于提高模型的可解释性。 在计算扩展性方面，ProtoT随序列长度线性扩展，而最先进的自注意transformer则呈二次方扩展。与基准相比，ProtoT在模型和数据规模上扩展良好，并且在文本生成和下游任务（GLUE）中表现出色。ProtoT对输入扰动的鲁棒性表现可媲美或优于某些基准模型，但其独特之处在于提供了解释路径，展示了如何产生这种鲁棒性和敏感性的。 达到接近最先进架构性能的同时，ProtoT为创建具有内在可解释性的高性能自回归LM开辟了道路。
URL
https://arxiv.org/abs/2602.11852
