Abstract
Vision-Language Models (VLMs) adapted to remote sensing rely heavily on domain-specific image-text supervision, yet high-quality annotations for satellite and aerial imagery remain scarce and expensive to produce. Prevailing pseudo-labeling pipelines address this gap by distilling knowledge from large frontier models, but this dependence on large teachers is costly, limits scalability, and caps achievable performance at the ceiling of the teacher. We propose OSMDA: a self-contained domain adaptation framework that eliminates this dependency. Our key insight is that a capable base VLM can serve as its own annotation engine: by pairing aerial images with rendered OpenStreetMap (OSM) tiles, we leverage optical character recognition and chart comprehension capabilities of the model to generate captions enriched by OSM's vast auxiliary metadata. The model is then fine-tuned on the resulting corpus with satellite imagery alone, yielding OSMDA-VLM, a domain-adapted VLM that requires no manual labeling and no stronger external model. We conduct exhaustive evaluations spanning 10 benchmarks across image-text-to-text tasks and comparing against 9 competitive baselines. When equally mixed with real data, our method achieves state-of-the-art results, while being substantially cheaper to train than teacher-dependent alternatives. These results suggest that, given a strong foundation model, alignment with crowd-sourced geographic data is a practical and scalable path towards remote sensing domain adaptation. Dataset and model weights will be made publicly available.
Abstract (translated)
视觉-语言模型（VLMs）在遥感领域中的应用高度依赖于特定领域的图像文本监督，然而，高质量的卫星和航空影像标注仍然稀缺且昂贵。现有的伪标签流水线通过从大型前沿模型中提取知识来弥补这一缺口，但这种对大型教师模型的依赖既成本高昂又限制了可扩展性，并将性能上限锁定在教师模型的能力水平上。为此，我们提出了OSMDA：一种自我封闭领域的适应框架，以消除这种依赖性。 我们的关键见解是，有能力的基础VLM可以充当自己的标注引擎：通过将航空影像与渲染的OpenStreetMap（OSM）瓦片配对使用，我们可以利用模型的光学字符识别和图表理解能力生成包含OSM丰富辅助元数据的说明。然后，该模型仅基于卫星图像对其进行微调，从而得到OSMDA-VLM，这是一种无需人工标注且不需要更强大外部模型的领域适应VLM。 我们进行了详尽的评估，涵盖10个跨图-文到文本任务的基准测试，并与9种竞争基线进行比较。当混合使用真实数据时，我们的方法实现了最先进的结果，同时训练成本远低于依赖于教师模型的方法。这些结果表明，在一个强大的基础模型的基础上，与众包地理信息对齐是一种实用且可扩展地走向遥感领域适应的路径。 我们将公开发布该方法的数据集和模型权重。
URL
https://arxiv.org/abs/2603.11804
