Abstract
Sparse Autoencoders (SAEs) have been successfully used to probe Large Language Models (LLMs) and extract interpretable concepts from their internal representations. These concepts are linear combinations of neuron activations that correspond to human-interpretable features. In this paper, we investigate the effectiveness of SAE-based explainability approaches for sentence classification, a domain where such methods have not been extensively explored. We present a novel SAE-based architecture tailored for text classification, leveraging a specialized classifier head and incorporating an activation rate sparsity loss. We benchmark this architecture against established methods such as ConceptShap, Independent Component Analysis, and other SAE-based concept extraction techniques. Our evaluation covers two classification benchmarks and four fine-tuned LLMs from the Pythia family. We further enrich our analysis with two novel metrics for measuring the precision of concept-based explanations, using an external sentence encoder. Our empirical results show that our architecture improves both the causality and interpretability of the extracted features.
Abstract (translated)
稀疏自编码器（SAEs）已被成功用于探测大型语言模型（LLMs），并从其内部表示中提取可解释的概念。这些概念是与人类可理解特征相对应的神经元激活线性组合。在本文中，我们研究了基于SAE的可解释方法在句子分类领域的有效性，在这一领域此类方法尚未得到广泛探索。我们提出了一种新颖的专为文本分类设计的基于SAE的架构，并利用了一个专门的分类头和激活率稀疏损失。我们将此架构与已有的方法（如ConceptShap、独立成分分析以及其他基于SAE的概念提取技术）进行了基准测试。我们的评估涵盖了两个分类基准以及来自Pythia家族的四个微调后的大型语言模型。我们进一步通过两种新的度量标准丰富了我们的分析，这些度量使用外部句子编码器来衡量基于概念的解释的精确性。实证结果表明，我们的架构提高了提取特征的原因性和可解释性。
URL
https://arxiv.org/abs/2506.23951
