Abstract
Recent advances in Vision Language Models (VLMs) and Vision Foundation Models (VFMs) have opened new opportunities for zero-shot text-guided segmentation of remote sensing imagery. However, most existing approaches still rely on additional trainable components, limiting their generalisation and practical applicability. In this work, we investigate to what extent text-based remote sensing segmentation can be achieved without additional training, by relying solely on existing foundation models. We propose a simple yet effective approach that integrates contrastive and generative VLMs with the Segment Anything Model (SAM), enabling a fully training-free or lightweight LoRA-tuned pipeline. Our contrastive approach employs CLIP as mask selector for SAM's grid-based proposals, achieving state-of-the-art open-vocabulary semantic segmentation (OVSS) in a completely zero-shot setting. In parallel, our generative approach enables reasoning and referring segmentation by generating click prompts for SAM using GPT-5 in a zero-shot setting and a LoRA-tuned Qwen-VL model, with the latter yielding the best results. Extensive experiments across 19 remote sensing benchmarks, including open-vocabulary, referring, and reasoning-based tasks, demonstrate the strong capabilities of our approach. Code will be released at this https URL.
Abstract (translated)
最近在视觉语言模型（VLM）和视觉基础模型（VFMs）方面的进展为零样本文本引导的遥感图像分割带来了新的机遇。然而，大多数现有的方法仍然依赖于额外可训练的组件，这限制了它们的泛化能力和实际应用性。在这项工作中，我们探讨了在仅依靠现有基础模型的情况下实现基于文本的遥感分割到什么程度是可能的。我们提出了一种简单而有效的方法，该方法将对比和生成型VLM与Segment Anything Model（SAM）相结合，从而实现了完全无需训练或轻量级LoRA微调的流水线。 我们的对比方法利用CLIP作为SAM网格化提议的掩码选择器，在完全零样本设置下实现了最先进的开放式词汇语义分割(OVSS)。同时，我们生成的方法通过使用GPT-5在零样本环境下为SAM生成点击提示，并用经过LoRA微调的Qwen-VL模型来实现推理和参考分割任务，后者取得了最佳效果。 我们在包括开放词汇、引用及基于推理的任务在内的19个遥感基准数据集上进行了广泛的实验，展示了我们方法的强大能力。代码将在[提供的URL]发布。
URL
https://arxiv.org/abs/2602.17799
