Abstract
The completeness axiom renders the explanation of a post-hoc XAI method only locally faithful to the model, i.e. for a single decision. For the trustworthy application of XAI, in particular for high-stake decisions, a more global model understanding is required. Recently, concept-based methods have been proposed, which are however not guaranteed to be bound to the actual model reasoning. To circumvent this problem, we propose Multi-dimensional Concept Discovery (MCD) as an extension of previous approaches that fulfills a completeness relation on the level of concepts. Our method starts from general linear subspaces as concepts and does neither require reinforcing concept interpretability nor re-training of model parts. We propose sparse subspace clustering to discover improved concepts and fully leverage the potential of multi-dimensional subspaces. MCD offers two complementary analysis tools for concepts in input space: (1) concept activation maps, that show where a concept is expressed within a sample, allowing for concept characterization through prototypical samples, and (2) concept relevance heatmaps, that decompose the model decision into concept contributions. Both tools together enable a detailed understanding of the model reasoning, which is guaranteed to relate to the model via a completeness relation. This paves the way towards more trustworthy concept-based XAI. We empirically demonstrate the superiority of MCD against more constrained concept definitions.
Abstract (translated)
完备性的公理使得 post-hoc XAI 方法只能对模型做出局部的忠实解释,即做出一个单一决策。对于可信任的 XAI 应用,特别是高度重要性的决策,需要更多的全球模型理解。最近,基于概念的方法被提出,但这些方法不能保证与实际模型推理的绑定。为了避免这个问题,我们提出 Multi-dimensional Concept Discovery(MCD)作为先前方法的扩展,满足概念层面的完备性关系。我们的方法和先前的方法一样,从通用线性子空间作为概念开始,不需要强化概念解释性或重新训练模型部分。我们提议稀疏子空间聚类来发现改进的概念,并充分利用多个维度子空间的潜力。 MCD 提供了两个互补的分析工具,用于输入空间中的概念:(1)概念激活图,显示在样本中某个概念的表达位置,通过典型样本进行概念特征化,(2)概念相关热图,将模型决策分解为概念贡献。这两个工具一起可以实现对模型推理的详细理解,并保证通过完备性关系与模型建立联系。这为更可靠的基于概念的 XAI 铺平了道路。我们经验证地证明了 MCD 相对于更受限的概念定义的优越性。
URL
https://arxiv.org/abs/2301.11911