Abstract
Attention mechanism is contributing to the majority of recent advances in machine learning for natural language processing. Additionally, it results in an attention map that shows the proportional influence of each input in its decision. Empirical studies postulate that attention maps can be provided as an explanation for model output. However, it is still questionable to ask whether this explanation helps regular people to understand and accept the model output (the plausibility of the explanation). Recent studies show that attention weights in the RNN encoders are hardly plausible because they spread on input tokens. We thus propose 3 additional constraints to the learning objective function to improve the plausibility of the attention map: regularization to increase the attention weight sparsity, semi-supervision to supervise the map by a heuristic and supervision by human annotation. Results show that all techniques can improve the attention map plausibility at some level. We also observe that specific instructions for human annotation might have a negative effect on classification performance. Beyond the attention map, the result of experiments on text classification tasks also shows that no matter how the constraint brings the gain, the contextualization layer plays a crucial role in finding the right space for finding plausible tokens.
Abstract (translated)
注意力机制在自然语言处理领域的机器学习最近进展中发挥了重要作用。此外，它生成了一个显示每个输入在其决策中的相对影响力的关注图（attention map）。实证研究表明，可以将这些关注图作为模型输出的解释提供给人们。然而，仍值得探讨的是，这种解释是否有助于普通人理解和接受模型的输出结果（即解释的有效性）。 最近的研究表明，在循环神经网络（RNN）编码器中的注意力权重难以具有有效性，因为它们分散在输入令牌上。因此，我们提出了三种附加约束来改善关注图的有效性：增加注意权重稀疏性的正则化、通过启发式方法监督地图的半监督学习以及由人工注释进行监督。 实验结果显示，所有这些技术都在一定程度上提高了注意力图的有效性。同时观察到，特定的人类标注指导可能对分类性能产生负面影响。除了关注图之外，在文本分类任务上的实验证明，无论哪种约束带来了多少收益，上下文化层（contextualization layer）在找到合适的令牌空间以提高有效性方面起着至关重要的作用。
URL
https://arxiv.org/abs/2501.12775
