Abstract
In recent years, Contrastive Language-Image Pretraining (CLIP) has been widely applied to Weakly Supervised Semantic Segmentation (WSSS) tasks due to its powerful cross-modal semantic understanding capabilities. This paper proposes a novel Semantic and Spatial Rectification (SSR) method to address the limitations of existing CLIP-based weakly supervised semantic segmentation approaches: over-activation in non-target foreground regions and background areas. Specifically, at the semantic level, the Cross-Modal Prototype Alignment (CMPA) establishes a contrastive learning mechanism to enforce feature space alignment across modalities, reducing inter-class overlap while enhancing semantic correlations, to rectify over-activation in non-target foreground regions effectively; at the spatial level, the Superpixel-Guided Correction (SGC) leverages superpixel-based spatial priors to precisely filter out interference from non-target regions during affinity propagation, significantly rectifying background over-activation. Extensive experiments on the PASCAL VOC and MS COCO datasets demonstrate that our method outperforms all single-stage approaches, as well as more complex multi-stage approaches, achieving mIoU scores of 79.5% and 50.6%, respectively.
Abstract (translated)
近年来，对比语言-图像预训练（CLIP）由于其强大的跨模态语义理解能力，在弱监督语义分割（WSSS）任务中得到了广泛应用。本文提出了一种新颖的语义和空间矫正（SSR）方法，以解决现有基于 CLIP 的弱监督语义分割方法中存在的问题：非目标前景区域和背景区域中的过度激活现象。 具体来说，在语义层面，跨模态原型对齐（CMPA）建立了一个对比学习机制，强制不同模式之间的特征空间对齐，减少类间重叠并增强语义相关性，从而有效矫正非目标前景区域的过度激活；在空间层面上，超像素引导校正（SGC）利用基于超像素的空间先验知识，在亲和力传播过程中精确过滤掉非目标区域的影响，显著减少了背景的过度激活。 通过 PASCAL VOC 和 MS COCO 数据集上的大量实验表明，我们的方法超越了所有单阶段方法，并且在性能上优于更为复杂的多阶段方法，分别取得了 79.5% 和 50.6% 的 mIoU 分数。
URL
https://arxiv.org/abs/2512.01701
