Abstract
There exist many methods to explain how an image classification model generates its decision, but very little work has explored methods to explain why a classifier might lack confidence in its prediction. As there are various reasons the classifier might lose confidence, it would be valuable for this model to not only indicate its level of uncertainty but also explain why it is uncertain. Counterfactual images have been used to visualize changes that could be made to an image to generate a different classification decision. In this work, we explore the use of counterfactuals to offer an explanation for low model competency--a generalized form of predictive uncertainty that measures confidence. Toward this end, we develop five novel methods to generate high-competency counterfactual images, namely Image Gradient Descent (IGD), Feature Gradient Descent (FGD), Autoencoder Reconstruction (Reco), Latent Gradient Descent (LGD), and Latent Nearest Neighbors (LNN). We evaluate these methods across two unique datasets containing images with six known causes for low model competency and find Reco, LGD, and LNN to be the most promising methods for counterfactual generation. We further evaluate how these three methods can be utilized by pre-trained Multimodal Large Language Models (MLLMs) to generate language explanations for low model competency. We find that the inclusion of a counterfactual image in the language model query greatly increases the ability of the model to generate an accurate explanation for the cause of low model competency, thus demonstrating the utility of counterfactual images in explaining low perception model competency.
Abstract (translated)
有许多方法可以解释图像分类模型如何生成其决策，但很少有研究探索了分类器为何对其预测缺乏信心的方法。由于造成分类器自信度下降的原因多种多样，该模型不仅需要表明它的不确定性水平，还需要解释为什么它会感到不确定。 反事实图像被用来展示对图像进行哪些改变可以产生不同的分类结果。在这项工作中，我们探讨使用反事实来提供低模型能力（一种衡量信心的预测不确定性的泛化形式）的解释的方法。为此，我们开发了五种新的生成高能力反事实图像的方法：图像梯度下降（IGD）、特征梯度下降（FGD）、自动编码器重建（Reco）、潜在梯度下降（LGD）和潜在最近邻（LNN）。我们在包含六种已知导致低模型能力的图像的两个独特数据集上评估了这些方法，并发现Reco、LGD和LNN是生成反事实最有前景的方法。我们进一步研究了这三种方法如何被预训练的多模态大型语言模型（MLLM）用来为低模型能力产生文本解释。我们发现，在语言模型查询中包含一张反事实图像大大提高了其生成准确解释的能力，即解释造成低模型能力的原因，从而证明了反事实图像在解释低感知模型能力方面的实用性。
URL
https://arxiv.org/abs/2504.05254
