Abstract
Deep learning dominates image classification tasks, yet understanding how models arrive at predictions remains a challenge. Much research focuses on local explanations of individual predictions, such as saliency maps, which visualise the influence of specific pixels on a model's prediction. However, reviewing many of these explanations to identify recurring patterns is infeasible, while global methods often oversimplify and miss important local behaviours. To address this, we propose Segment Attribution Tables (SATs), a method for summarising local saliency explanations into (semi-)global insights. SATs take image segments (such as "eyes" in Chihuahuas) and leverage saliency maps to quantify their influence. These segments highlight concepts the model relies on across instances and reveal spurious correlations, such as reliance on backgrounds or watermarks, even when out-of-distribution test performance sees little change. SATs can explain any classifier for which a form of saliency map can be produced, using segmentation maps that provide named segments. SATs bridge the gap between oversimplified global summaries and overly detailed local explanations, offering a practical tool for analysing and debugging image classifiers.
Abstract (translated)
深度学习在图像分类任务中占据主导地位，但理解模型如何得出预测结果仍然是一项挑战。许多研究集中在个别预测的局部解释上，例如利用敏感性地图（saliency maps）来可视化特定像素对模型预测的影响。然而，通过审查大量这些解释以识别重复模式是不可行的，而全局方法往往过于简化，并且会错过重要的局部行为。为了解决这一问题，我们提出了片段归属表（Segment Attribution Tables, SATs），这是一种将局部敏感性解释汇总成（半）全局见解的方法。 SATs 使用图像片段（例如“眼睛”在吉娃娃犬中的角色）并利用敏感性地图来量化其对模型预测的影响。这些片段突出了模型跨实例依赖的概念，并揭示了虚假相关性，如背景或水印的依赖，即使是在分布外测试性能几乎没有变化的情况下也是如此。 SATs 可以解释任何可以生成某种形式敏感性图的分类器，使用提供命名片段的分割图。SATs 在过于简化的全局摘要和过于详细的局部解释之间架起了桥梁，为分析和调试图像分类器提供了实用工具。
URL
https://arxiv.org/abs/2506.23247
