Abstract
Open-vocabulary semantic segmentation (OVSS) is fundamentally hampered by the coarse, image-level representations of CLIP, which lack precise pixel-level details. Existing training-free methods attempt to resolve this by either importing priors from costly external foundation models (e.g., SAM, DINO) or by applying static, hand-crafted heuristics to CLIP's internal features. These approaches are either computationally expensive or sub-optimal. We propose the Attention Refinement Module (ARM), a lightweight, learnable module that effectively unlocks and refines CLIP's internal potential. Unlike static-fusion methods, ARM learns to adaptively fuse hierarchical features. It employs a semantically-guided cross-attention block, using robust deep features (K, V) to select and refine detail-rich shallow features (Q), followed by a self-attention block. The key innovation lies in a ``train once, use anywhere" paradigm. Trained once on a general-purpose dataset (e.g., COCO-Stuff), ARM acts as a universal plug-and-play post-processor for diverse training-free frameworks. Extensive experiments show that ARM consistently boosts baseline performance on multiple benchmarks with negligible inference overhead, establishing an efficient and effective paradigm for training-free OVSS.
Abstract (translated)
开放词汇语义分割（OVSS）的基本问题是，CLIP的粗略、图像级表示缺乏精确的像素级细节。现有的无训练方法试图通过从昂贵的外部基础模型（如SAM、DINO）导入先验知识或应用静态的手工设计启发式来解决这一问题，以利用CLIP内部特征。这些方法要么计算成本高昂，要么次优。我们提出了注意力精炼模块（ARM），这是一个轻量级的学习模块，能够有效解锁和细化CLIP的内在潜力。 与静态融合方法不同，ARM学习自适应地融合分层特征。它采用语义引导的交叉注意块，利用强大的深度特征（K, V）来选择和丰富细节丰富的浅层特征（Q），随后是自我注意块。关键创新在于“训练一次，随处使用”的理念：在通用数据集（如COCO-Stuff）上训练一次后，ARM作为无训练框架的通用即插即用后期处理模块。 广泛的实验表明，在多个基准测试中，ARM能显著提升基线性能，并且几乎没有推理开销。这为无训练OVSS建立了一个高效有效的范式。
URL
https://arxiv.org/abs/2512.24224
