Abstract
Deep neural networks (DNNs) have demonstrated remarkable success, yet their wide adoption is often hindered by their opaque decision-making. To address this, attribution methods have been proposed to assign relevance values to each part of the input. However, different methods often produce entirely different relevance maps, necessitating the development of standardized metrics to evaluate them. Typically, such evaluation is performed through perturbation, wherein high- or low-relevance regions of the input image are manipulated to examine the change in prediction. In this work, we introduce a novel approach, which harnesses image generation models to perform targeted perturbation. Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity. This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results. Through extensive experiments, we demonstrate the effectiveness of our approach in generating meaningful rankings across a wide range of models and attribution methods. Crucially, we establish that the ranking produced by our metric exhibits significantly higher correlation with human preferences compared to existing approaches, underscoring its potential for enhancing interpretability in DNNs.
Abstract (translated)
深度神经网络（DNN）展现了显著的成功，但其广泛采用往往受到其不透明决策过程的阻碍。为了解决这一问题，已经提出了归因方法来给输入的每一部分分配相关性值。然而，不同的方法常常产生完全不同的相关性图谱，这需要开发标准化的评估指标来进行评价。通常，这种评估是通过扰动进行的，即操纵输入图像中的高或低相关区域以观察预测的变化。 在本文中，我们引入了一种新颖的方法，利用图像生成模型执行定向扰动。具体来说，我们将仅对输入图像中的高相关像素进行修复（inpainting），同时修改模型的预测并保持图像的真实性。这与现有方法形成对比，后者通常会产生出分布的数据修改，导致结果不可靠。 通过广泛的实验，我们展示了该方法在广泛的不同模型和归因方法中生成有意义排名的有效性。尤其重要的是，我们证明了我们的度量所生成的排序与人类偏好之间的相关性显著高于现有的方法，强调了其增强DNN可解释性的潜力。
URL
https://arxiv.org/abs/2504.06800
