Abstract
Transformers have profoundly influenced AI research, but explaining their decisions remains challenging -- even for relatively simpler tasks such as classification -- which hinders trust and safe deployment in real-world applications. Although activation-based attribution methods effectively explain transformer-based text classification models, our findings reveal that these methods can be undermined by class-irrelevant features within activations, leading to less reliable interpretations. To address this limitation, we propose Contrast-CAT, a novel activation contrast-based attribution method that refines token-level attributions by filtering out class-irrelevant features. By contrasting the activations of an input sequence with reference activations, Contrast-CAT generates clearer and more faithful attribution maps. Experimental results across various datasets and models confirm that Contrast-CAT consistently outperforms state-of-the-art methods. Notably, under the MoRF setting, it achieves average improvements of x1.30 in AOPC and x2.25 in LOdds over the most competing methods, demonstrating its effectiveness in enhancing interpretability for transformer-based text classification.
Abstract (translated)
Transformer模型对AI研究产生了深远的影响，但解释它们的决策仍然具有挑战性——即使对于相对简单的任务（如分类）也是如此。这阻碍了实际应用中的信任和安全部署。虽然基于激活的方法能够有效解释基于Transformer的文本分类模型，我们的研究表明这些方法可能会受到激活中无关类别的特征干扰，从而导致不那么可靠的解释。为了解决这一限制，我们提出了Contrast-CAT，这是一种新的基于激活对比的归因方法，通过过滤掉无关类别的特征来优化令牌级别的归因。通过将输入序列的激活与参考激活进行比较，Contrast-CAT可以生成更清晰且更准确的归因图谱。 在各种数据集和模型上的实验结果证实了Contrast-CAT始终优于现有的最先进方法。特别地，在MoRF设置下，它分别比最佳竞争方法平均提高了AOPC值1.30倍和LOdds值2.25倍，展示了其增强基于Transformer的文本分类解释性的有效性。
URL
https://arxiv.org/abs/2507.21186
