Abstract
Aligning human-interpretable concepts with the internal representations learned by modern machine learning systems remains a central challenge for interpretable AI. We introduce a geometric framework for comparing supervised human concepts with unsupervised intermediate representations extracted from foundation model embeddings. Motivated by the role of conceptual leaps in scientific discovery, we formalise the notion of concept frustration: a contradiction that arises when an unobserved concept induces relationships between known concepts that cannot be made consistent within an existing ontology. We develop task-aligned similarity measures that detect concept frustration between supervised concept-based models and unsupervised representations derived from foundation models, and show that the phenomenon is detectable in task-aligned geometry while conventional Euclidean comparisons fail. Under a linear-Gaussian generative model we derive a closed-form expression for Bayes-optimal concept-based classifier accuracy, decomposing predictive signal into known-known, known-unknown and unknown-unknown contributions and identifying analytically where frustration affects performance. Experiments on synthetic data and real language and vision tasks demonstrate that frustration can be detected in foundation model representations and that incorporating a frustrating concept into an interpretable model reorganises the geometry of learned concept representations, to better align human and machine reasoning. These results suggest a principled framework for diagnosing incomplete concept ontologies and aligning human and machine conceptual reasoning, with implications for the development and validation of safe interpretable AI for high-risk applications.
Abstract (translated)
将人类可解释的概念与现代机器学习系统学习的内部表征相对齐，始终是可解释人工智能的核心挑战。我们引入了一个几何框架，用于比较基于监督学习的人类概念与从基础模型嵌入中提取的无监督中间表征。受科学发现中概念飞跃作用的启发，我们形式化了“概念挫折”的概念：当未观测概念在已知概念间诱导出无法在现有本体论中保持一致的关系时，由此产生的矛盾。我们开发了任务对齐的相似性度量方法，能够检测基于监督概念模型与源自基础模型的无监督表征之间的概念挫折，并证明该现象在任务对齐几何中可被检测，而传统的欧几里得比较则失效。在线性高斯生成模型下，我们推导了贝叶斯最优基于概念分类器准确率的闭式解，将预测信号分解为已知-已知、已知-未知和未知-未知的贡献，并分析性地识别出挫折影响性能的具体环节。合成数据及真实语言与视觉任务的实验表明，基础模型表征中可检测到挫折现象，且将挫折概念纳入可解释模型会重组已学概念表征的几何结构，从而更好地对齐人类与机器的推理。这些结果提出了一种诊断不完整概念本体论、对齐人类与机器概念推理的原则性框架，对高风险应用中安全可解释人工智能的开发与验证具有重要意义。
URL
https://arxiv.org/abs/2603.29654
