Abstract
Remote sensing imagery presents vast, inherently unstructured spatial data, demanding sophisticated reasoning to interpret complex user intents and contextual relationships beyond simple recognition tasks. In this paper, we aim to construct an Earth observation workflow to handle complex queries by reasoning about spatial context and user intent. As a reasoning workflow, it should be somewhat autonomous, where predefined ground-truth reasoning paths do not constrain the learning process. Furthermore, its architecture ought to be unified yet flexible, enabling the model to perform diverse reasoning tasks with distinct output formats through a single forward pass. Existing remote sensing approaches fail to address these requirements, as they rely on supervised fine-tuning paradigms that constrain the autonomy of reasoning. To this end, we propose RemoteReasoner, a flexible and robust workflow for remote sensing reasoning tasks. The design of RemoteReasoner integrates a multi-modal large language model (MLLM) for interpreting user instructions and localizing targets, together with task adaptation strategies that enable multi-granularity output generation. In contrast to existing methods, our framework is trained with reinforcement learning (RL) to endow the MLLM sufficient autonomy for precise reasoning. At the inference stage, our adaptation strategies enable diverse output formats at inference time without requiring task-specific decoders or further fine-tuning. Preliminary experiments demonstrated that RemoteReasoner achieves remarkable performance across multi-granularity reasoning tasks, including region-level and pixel-level. Additionally, our framework enables novel capabilities such as the contour extraction task beyond the reach of existing reasoning pipelines.
Abstract (translated)
遥感影像提供了大量内在无结构的空间数据，这需要复杂的推理来解读用户意图和上下文关系，而这些超出了简单的识别任务范围。本文旨在构建一个地球观测工作流程，用于处理复杂查询并通过推理空间背景和用户意图来解决这些问题。作为一个推理工作流，它应该是某种程度上自主的，即预定义的真实路径不应限制学习过程。此外，其架构应该统一且灵活，能够通过一次前向传播完成多样化的推理任务，并产生不同格式的结果。现有的遥感方法未能满足这些需求，因为它们依赖于监督微调范式，这会限制推理的自主性。为此，我们提出了RemoteReasoner，一种用于遥感推理任务的灵活而稳健的工作流程。 RemoteReasoner的设计集成了一个多模态大型语言模型（MLLM），用于解释用户指令和定位目标，并结合了多粒度输出生成的任务适应策略。与现有方法不同的是，我们的框架通过强化学习（RL）进行训练，赋予MLLM足够的自主性以实现精确推理。在推断阶段，我们的适应策略能够在无需特定任务解码器或进一步微调的情况下，在推断时支持多样化的输出格式。 初步实验表明，RemoteReasoner在多粒度推理任务中实现了显著的性能提升，包括区域级和像素级任务。此外，我们框架还具备现有的推理流水线无法实现的新能力，例如轮廓提取任务。
URL
https://arxiv.org/abs/2507.19280
