Abstract
Open-vocabulary semantic segmentation (OVSS) extends traditional closed-set segmentation by enabling pixel-wise annotation for both seen and unseen categories using arbitrary textual descriptions. While existing methods leverage vision-language models (VLMs) like CLIP, their reliance on image-level pretraining often results in imprecise spatial alignment, leading to mismatched segmentations in ambiguous or cluttered scenes. However, most existing approaches lack strong object priors and region-level constraints, which can lead to object hallucination or missed detections, further degrading performance. To address these challenges, we propose LoGoSeg, an efficient single-stage framework that integrates three key innovations: (i) an object existence prior that dynamically weights relevant categories through global image-text similarity, effectively reducing hallucinations; (ii) a region-aware alignment module that establishes precise region-level visual-textual correspondences; and (iii) a dual-stream fusion mechanism that optimally combines local structural information with global semantic context. Unlike prior works, LoGoSeg eliminates the need for external mask proposals, additional backbones, or extra datasets, ensuring efficiency. Extensive experiments on six benchmarks (A-847, PC-459, A-150, PC-59, PAS-20, and PAS-20b) demonstrate its competitive performance and strong generalization in open-vocabulary settings.
Abstract (translated)
开放词汇语义分割（OVSS）扩展了传统的封闭集分割，通过允许使用任意文本描述对已见和未见过的类别进行像素级标注。尽管现有方法利用了视觉-语言模型（VLMs），如CLIP，但这些方法依赖于图像级别的预训练往往会导致空间定位不准确，在模糊或杂乱的场景中产生不匹配的分割结果。然而，大多数现有的方法缺乏强大的对象先验和区域级别约束，这可能导致对象幻觉或漏检，进一步降低性能。为了解决这些问题，我们提出了LoGoSeg，这是一个高效的单阶段框架，整合了三个关键创新：(i) 一个基于全局图像-文本相似性的对象存在先验，通过动态加权相关类别有效减少幻觉；(ii) 一个区域感知对齐模块，建立精确的区域级视觉-文本对应关系；以及(iii) 一种双流融合机制，将局部结构信息与全局语义背景最优结合。不同于先前的工作，LoGoSeg消除了对外部掩码提案、额外骨干网络或附加数据集的需求，确保了效率。在六个基准测试（A-847, PC-459, A-150, PC-59, PAS-20和PAS-20b）上的广泛实验表明其在开放词汇设置中的竞争力和强大的泛化能力。
URL
https://arxiv.org/abs/2602.05578
