Abstract
Extracting a small subset of crucial rationales from the full input is a key problem in explainability research. The most widely used fundamental criterion for rationale extraction is the maximum mutual information (MMI) criterion. In this paper, we first demonstrate that MMI suffers from diminishing marginal returns. Once part of the rationale has been identified, finding the remaining portions contributes only marginally to increasing the mutual information, making it difficult to use MMI to locate the rest. In contrast to MMI that aims to reproduce the prediction, we seek to identify the parts of the input that the network can actually utilize. This is achieved by comparing how different rationale candidates match the capability space of the weight matrix. The weight matrix of a neural network is typically low-rank, meaning that the linear combinations of its column vectors can only cover part of the directions in a high-dimensional space (high-dimension: the dimensions of an input vector). If an input is fully utilized by the network, {it generally matches these directions (e.g., a portion of a hypersphere), resulting in a representation with a high norm. Conversely, if an input primarily falls outside (orthogonal to) these directions}, its representation norm will approach zero, behaving like noise that the network cannot effectively utilize. Building on this, we propose using the norms of rationale candidates as an alternative objective to MMI. Through experiments on four text classification datasets and one graph classification dataset using three network architectures (GRUs, BERT, and GCN), we show that our method outperforms MMI and its improved variants in identifying better rationales. We also compare our method with a representative LLM (llama-3.1-8b-instruct) and find that our simple method gets comparable results to it and can sometimes even outperform it.
Abstract (translated)
从全文输入中提取关键理由的子集是可解释性研究中的一个核心问题。目前最广泛使用的基本标准是最大互信息（MMI）准则。在本文中，我们首先展示了MMI存在边际收益递减的问题：一旦部分原因被识别出来，找到剩余的部分只能微弱地增加互信息值，这使得利用MMI来定位其余部分变得困难。 与MMI旨在重现预测不同，我们的目标是识别网络实际可以使用的输入部分。通过比较不同的理由候选者如何匹配权重矩阵的能力空间实现这一目标。神经网络的权重矩阵通常是低秩的，这意味着其列向量的线性组合只能覆盖高维空间中的一部分方向（例如：输入向量维度）。如果一个输入被网络完全利用，则它通常与这些方向相匹配（比如超球体的一小部分），从而产生具有高范数的表示。相反地，如果一个输入主要位于（垂直于）这些方向之外，其表示范数将接近零，表现出像噪音一样无法有效利用的状态。 基于此观察，我们提出使用理由候选者的范数作为MMI的一个替代目标函数。通过在四个文本分类数据集和一个图分类数据集上进行实验，并采用三种网络架构（GRUs、BERT 和 GCN），我们展示了我们的方法在识别更好的理由方面优于MMI及其改进变体。 此外，我们将我们的方法与一个代表性的大语言模型（llama-3.1-8b-instruct）进行了比较，发现我们的简单方法能够获得与其相当的结果，并且有时甚至可以超越它。
URL
https://arxiv.org/abs/2503.06202
