Abstract
Despite the increasing relevance of explainable AI, assessing the quality of explanations remains a challenging issue. Due to the high costs associated with human-subject experiments, various proxy metrics are often used to approximately quantify explanation quality. Generally, one possible interpretation of the quality of an explanation is its inherent value for teaching a related concept to a student. In this work, we extend artificial simulatability studies to the domain of graph neural networks. Instead of costly human trials, we use explanation-supervisable graph neural networks to perform simulatability studies to quantify the inherent usefulness of attributional graph explanations. We perform an extensive ablation study to investigate the conditions under which the proposed analyses are most meaningful. We additionally validate our methods applicability on real-world graph classification and regression datasets. We find that relevant explanations can significantly boost the sample efficiency of graph neural networks and analyze the robustness towards noise and bias in the explanations. We believe that the notion of usefulness obtained from our proposed simulatability analysis provides a dimension of explanation quality that is largely orthogonal to the common practice of faithfulness and has great potential to expand the toolbox of explanation quality assessments, specifically for graph explanations.
Abstract (translated)
尽管解释性人工智能越来越相关，但评估解释质量仍然是一个挑战性的问题。由于人类受试者实验的高成本，通常使用各种指标来大约量化解释质量。一般而言，一个可能的解释质量概念是，解释对于向学生传授相关概念本身的有用性。在这个工作中，我们将人工可模拟研究扩展到图形神经网络领域。我们不要用昂贵的人类试验来执行可解释性图形神经网络的模拟研究，而是使用解释监督的图形神经网络执行模拟研究，以量化 attributed 图形解释的固有有用性。我们进行了广泛的去噪研究，以研究提出的分析中最有意义的条件。我们还验证了我们方法适用于现实世界图形分类和回归数据集的适用性。我们发现，相关的解释可以显著增强图形神经网络样本效率，并分析解释中的噪声和偏差的鲁棒性。我们认为，我们提出的可模拟研究的有用性概念提供了解释质量的一个维度，这在很大程度上与一致性的实践相排斥，并且有很大的潜力扩展解释质量评估的工具箱，特别是对于图形解释。
URL
https://arxiv.org/abs/2305.15961
