Abstract
Explainable Artificial Intelligence (XAI) poses a significant challenge in providing transparent and understandable insights into complex AI models. Traditional post-hoc algorithms, while useful, often struggle to deliver interpretable explanations. Concept-based models offer a promising avenue by incorporating explicit representations of concepts to enhance interpretability. However, existing research on automatic concept discovery methods is often limited by lower-level concepts, costly human annotation requirements, and a restricted domain of background knowledge. In this study, we explore the potential of a Large Language Model (LLM), specifically GPT-4, by leveraging its domain knowledge and common-sense capability to generate high-level concepts that are meaningful as explanations for humans, for a specific setting of image classification. We use minimal textual object information available in the data via prompting to facilitate this process. To evaluate the output, we compare the concepts generated by the LLM with two other methods: concepts generated by humans and the ECII heuristic concept induction system. Since there is no established metric to determine the human understandability of concepts, we conducted a human study to assess the effectiveness of the LLM-generated concepts. Our findings indicate that while human-generated explanations remain superior, concepts derived from GPT-4 are more comprehensible to humans compared to those generated by ECII.
Abstract (translated)
可解释人工智能（XAI）在为复杂AI模型提供透明和可理解的洞察方面提出了重大挑战。虽然传统的后验算法在某种程度上很有用，但往往很难提供可解释的解释。基于概念的模型通过将概念的显式表示来提高可解释性，为这一途径提供了有前景的方法。然而，现有的自动概念发现方法的 research 通常受到较低级别的概念、昂贵的人类标注要求和受限的知识域的限制。在这项研究中，我们探讨了大型语言模型（LLM）特别是 GPT-4 的潜力，通过利用其领域知识和常识能力生成高质量的概念，作为解释为人类对图像分类的特定场景的高层次概念。我们通过提示来最小化数据中可用到的文本对象信息，促进这一过程。为了评估输出，我们比较了 LLM 生成的概念与其他两种方法：由人类生成的概念和 ECII 隐式概念诱导系统生成的概念。由于没有明确的方法来确定人类对概念的理解，我们进行了一项人类研究来评估 LLM 生成的概念的有效性。我们的研究结果表明，尽管人类生成的解释仍然具有优势，但 GPT-4 生成的概念对人类来说更有可理解性，与 ECII 生成的概念相比更具可理解性。
URL
https://arxiv.org/abs/2404.11875
