Abstract
Recent self-supervised Vision Transformers (ViTs), such as DINOv3, provide rich feature representations for dense vision tasks. This study investigates the intrinsic few-shot semantic segmentation (FSS) capabilities of frozen DINOv3 features through a training-free baseline, FSSDINO, utilizing class-specific prototypes and Gram-matrix refinement. Our results across binary, multi-class, and cross-domain (CDFSS) benchmarks demonstrate that this minimal approach, applied to the final backbone layer, is highly competitive with specialized methods involving complex decoders or test-time adaptation. Crucially, we conduct an Oracle-guided layer analysis, identifying a significant performance gap between the standard last-layer features and globally optimal intermediate representations. We reveal a "Safest vs. Optimal" dilemma: while the Oracle proves higher performance is attainable, matching the results of compute-intensive adaptation methods, current unsupervised and support-guided selection metrics consistently yield lower performance than the last-layer baseline. This characterizes a "Semantic Selection Gap" in Foundation Models, a disconnect where traditional heuristics fail to reliably identify high-fidelity features. Our work establishes the "Last-Layer" as a deceptively strong baseline and provides a rigorous diagnostic of the latent semantic potentials in this http URL code is publicly available at this https URL.
Abstract (translated)
近期的自监督视觉变换器（ViT），如DINOv3，为密集视觉任务提供了丰富的特征表示。本研究通过一个无训练基线FSSDINO来探究冻结后的DINOv3特征在少量样本语义分割(FSS)方面的内在能力，该方法利用了特定类别的原型和Gram矩阵改进。 我们在二元、多类别以及跨域（CDFSS）基准测试上的结果表明，即使采用简单的最后一层特征处理方式，FSSDINO也能与复杂的解码器或测试时间适应等专门化方法匹敌。尤为关键的是，我们进行了Oracle引导的层次分析，发现标准的最后一层特征和全局最优中间表示之间存在显著性能差距。 我们揭示了一个"最安全 vs. 最优"的困境：尽管Oracle证明了更高的性能是可实现的，并且可以媲美计算密集型适应方法的结果，但目前无监督和支持指导的选择指标始终无法超越最后一层基准的表现。这标志着在基础模型中存在着一个“语义选择差距”，即传统启发式方法未能可靠地识别出高质量特征的现象。 我们的研究确立了"最后一层"作为强大的基线，并为探索该架构的潜在语义潜力提供了严格的诊断。相关代码已公开发布，可在提供的链接地址找到。
URL
https://arxiv.org/abs/2602.07550
