Abstract
In this paper, we propose ReSeg-CLIP, a new training-free Open-Vocabulary Semantic Segmentation method for remote sensing data. To compensate for the problems of vision language models, such as CLIP in semantic segmentation caused by inappropriate interactions within the self-attention layers, we introduce a hierarchical scheme utilizing masks generated by SAM to constrain the interactions at multiple scales. We also present a model composition approach that averages the parameters of multiple RS-specific CLIP variants, taking advantage of a new weighting scheme that evaluates representational quality using varying text prompts. Our method achieves state-of-the-art results across three RS benchmarks without additional training.
Abstract (translated)
在这篇论文中，我们提出了一种新的无需训练的开放词汇语义分割方法ReSeg-CLIP，专门用于遥感数据。为了弥补如CLIP等视觉语言模型在语义分割过程中由于自注意力层内部不适当交互而造成的问题，我们引入了一个分层次方案，利用SAM生成的掩码来约束多尺度下的交互过程。此外，我们还提出了一种模型组合方法，该方法通过平均多个针对遥感特定设计的CLIP变体的参数，并采用一种新的加权方案，使用不同文本提示来评估表示质量。我们的方法在无需额外训练的情况下，在三个遥感基准数据集上取得了最先进的结果。
URL
https://arxiv.org/abs/2602.23869
