Abstract
Vision-language models encode continuous geometry that their text pathway fails to express: a 6,000-parameter linear probe extracts hand joint angles at 6.1 degrees MAE from frozen features, while the best text output achieves only 20.0 degrees -- a 3.3x bottleneck. LoRA fine-tuning (r=16, 2,000 images) narrows this gap to 6.5 degrees, providing evidence for a pathway-training deficit rather than a representational one. Training objective determines accuracy more than architecture: five encoders spanning self-supervised, contrastive, and hybrid paradigms converge to statistically equivalent accuracy (R^2 approximately 0.55, TOST-equivalent at delta=0.03) despite sharing as little as CKA=0.41 representational similarity -- functional convergence without representational convergence. Autoregressive generation damages geometric fidelity, but the damage originates in the generation process, not in language alignment: Qwen2.5-VL's LLM layers actually improve probe accuracy over its raw vision encoder. Layer-wise analysis reveals a universal mid-network accuracy peak across all architectures, with attention heads in layers 18-22 carrying disproportionate geometric signal. These findings enable a single frozen backbone to function as a multi-task geometric sensor through lightweight probes, without fine-tuning or text generation.
Abstract (translated)
视觉语言模型编码了连续的几何信息，这是其文本路径无法表达的：一个包含6,000个参数的线性探针从冻结特征中提取出手关节角度，在MAE（均方误差）下达到了6.1度，而最佳的文字输出仅达到20.0度——相差3.3倍。LoRA微调（r=16，2,000张图片）将这一差距缩小至6.5度，这表明路径训练不足而不是表示不足的问题。 训练目标比架构更能决定准确性：五个跨自监督、对比和混合范式的编码器在统计上达到等效的精度（R^2约为0.55，在delta=0.03时具有TOST等价性），尽管它们之间共享的表征相似度低至CKA（基于核的方法）为0.41——功能收敛而非表示收敛。自回归生成损害了几何保真度，但这种损害源于生成过程本身，并非语言对齐：Qwen2.5-VL的LLM层实际上比其原始视觉编码器提高了探针精度。 逐层分析揭示了所有架构中普遍存在的一致网络中间准确性峰值，在第18-22层中的注意力头携带了不成比例的几何信号。这些发现使得单个冻结骨干可以通过轻量级探针充当多任务几何传感器，而无需微调或文本生成。
URL
https://arxiv.org/abs/2603.06459
