Abstract
Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.
Abstract (translated)
近期可解释性研究常将特征视为跨上下文的单一全局方向、字典原子或潜在坐标。我们认为这种本体论在阻塞表示空间中可能失效——局部一致的含义未必能整合为全局一致的特征。我们提出一种图集原生的替代对象：语义截面，即定义在上下文图集上的传输兼容的局部特征代表族。我们形式化语义截面，证明树支撑传播始终可路径实现，并表明循环一致性是真正全局化的关键判据。这区分出树局部、可全局化与扭曲截面三类，其中扭曲截面捕获局部一致但受全息阻塞的含义。随后我们开发了基于种子传播、重叠区同步、基于缺陷剪枝、循环感知分类与去重的发现认证流程。在Llama 3.2 3B Instruct、Qwen 2.5 3B Instruct与Gemma 2 2B IT的第16层图集上，我们发现非平凡的语义截面群体，包括去重后存在循环支撑的可全局化与扭曲态。最重要的是，语义同一性无法通过原始全局向量相似度恢复：即使认证的可全局化截面也呈现跨图表低带符号余弦相似度，原始相似度基线仅恢复小部分真实截面内配对，且在中等阈值时常崩溃。相比之下，基于截面的同一性恢复在认证支持集上完全准确。这些结果支持语义截面作为阻塞 regimes 中更优的特征本体论。
URL
https://arxiv.org/abs/2603.20867
