Abstract
Open-vocabulary semantic segmentation has emerged as a promising research direction in remote sensing, enabling the recognition of diverse land-cover types beyond pre-defined category sets. However, existing methods predominantly rely on the passive mapping of visual features and textual embeddings. This ``appearance-based" paradigm lacks geospatial contextual awareness, leading to severe semantic ambiguity and misclassification when encountering land-cover classes with similar spectral features but distinct semantic attributes. To address this, we propose a Geospatial Reasoning Chain-of-Thought (GR-CoT) framework designed to enhance the scene understanding capabilities of Multimodal Large Language Models (MLLMs), thereby guiding open-vocabulary segmentation models toward precise mapping. The framework comprises two collaborative components: an offline knowledge distillation stream and an online instance reasoning stream. The offline stream establishes fine-grained category interpretation standards to resolve semantic conflicts between similar land-cover types. During online inference, the framework executes a sequential reasoning process involving macro-scenario anchoring, visual feature decoupling, and knowledge-driven decision synthesis. This process generates an image-adaptive vocabulary that guides downstream models to achieve pixel-level alignment with correct geographical semantics. Extensive experiments on the LoveDA and GID5 benchmarks demonstrate the superiority of our approach.
Abstract (translated)
开放词汇语义分割在遥感领域中作为一种有前景的研究方向，它能够识别超出预定义类别集的多样化的地表覆盖类型。然而，现有的方法主要依赖于视觉特征和文本嵌入的被动映射。“基于外观”的这一范式缺乏地理空间上下文感知能力，在遇到具有相似光谱特性但语义属性不同的地表覆盖类时会导致严重的语义模糊和误分类问题。 为了解决这些问题，我们提出了一种名为“地理空间推理链思维”（GR-CoT）的框架，旨在提升多模态大型语言模型（MLLMs）对场景的理解能力，并引导开放词汇分割模型走向精确映射。该框架包含两个协作组件：离线知识提炼流和在线实例推理流。 - 离线流建立了细粒度类别解释标准，以解决相似地表覆盖类型之间的语义冲突。 - 在线推断过程中，框架执行包括宏观场景锚定、视觉特征解耦以及基于知识的决策综合在内的序列化推理过程。这一过程生成了一个适应图像的词汇表，引导下游模型在像素级别与正确的地理语义实现对齐。 在LoveDA和GID5基准测试上的广泛实验验证了我们方法的优势。
URL
https://arxiv.org/abs/2602.08206
