Abstract
Reliance on spurious correlations (shortcuts) has been shown to underlie many of the successes of language models. Previous work focused on identifying the input elements that impact prediction. We investigate how shortcuts are actually processed within the model's decision-making mechanism. We use actor names in movie reviews as controllable shortcuts with known impact on the outcome. We use mechanistic interpretability methods and identify specific attention heads that focus on shortcuts. These heads gear the model towards a label before processing the complete input, effectively making premature decisions that bypass contextual analysis. Based on these findings, we introduce Head-based Token Attribution (HTA), which traces intermediate decisions back to input tokens. We show that HTA is effective in detecting shortcuts in LLMs and enables targeted mitigation by selectively deactivating shortcut-related attention heads.
Abstract (translated)
依赖于虚假相关性（捷径）被认为是许多语言模型成功背后的关键因素。以往的研究主要集中在识别影响预测的输入元素上，而我们则探讨了这些捷径如何在模型决策机制中实际被处理。 为了研究这一问题，我们使用电影评论中的演员名字作为已知能对结果产生影响的可控捷径。通过机械解释性方法（mechanistic interpretability），我们确定了一些特别关注捷径的注意力头（attention heads）。这些注意力头会使模型在处理完全部输入之前就倾向于偏向某个标签，实际上导致了过早决策，从而绕过了上下文分析。 基于以上发现，我们提出了基于注意力头的标记归因法（Head-based Token Attribution, HTA），这种方法能够追踪中间决策回溯到原始输入标记。结果显示HTA在检测LLM中的捷径方面非常有效，并且可以通过选择性地关闭与捷径相关的注意力头来实现有针对性的缓解措施。
URL
https://arxiv.org/abs/2505.06032
