Abstract
Vision Language Models (VLMs) have achieved remarkable breakthroughs in the field of remote sensing in recent years. Synthetic Aperture Radar (SAR) imagery, with its all-weather capability, is essential in remote sensing, yet the lack of large-scale, high-quality SAR image-text datasets hinders its semantic understanding. In this paper, we construct SAR-Text, a large-scale and high-quality dataset consisting of over 130,000 SAR image-text pairs. To construct the SAR-Text dataset, we design the SAR-Narrator framework, which generates textual descriptions for SAR images through a multi-stage progressive transfer learning strategy. To verify the effectiveness of the SAR-TEXT dataset, we conduct experiments on three typical vision-language tasks: image-text retrieval, image captioning, and visual question answering (VQA). Specifically, we construct three representative models on SAR-TEXT: SAR-RS-CLIP, SAR-RS-CoCa, and SAR-GPT. SAR-RS-CLIP achieves notable improvements in retrieval performance, boosting average recall by 16.43% and 10.54% on the OSdataset-512 and HRSID test sets, respectively. In the captioning task, SAR-RS-CoCa achieves BLEU-4, SPICE, and CIDEr scores exceeding those of the original CoCa model by more than 8x, 4x, and 10x, respectively. In the VQA task, SAR-GPT outperforms baseline and single-stage models on multiple SAR-VQA datasets, demonstrating stronger semantic understanding and reasoning ability, as further confirmed by qualitative results. It is worth noting that, as a flexible captioning tool, SAR-Narrator can be readily adopted by the community to construct larger-scale SAR image-text datasets.
Abstract (translated)
近年来，视觉语言模型（VLMs）在遥感领域取得了显著的突破。合成孔径雷达（SAR）图像因其全天候能力，在遥感中至关重要，但缺乏大规模、高质量的SAR图像-文本数据集阻碍了其语义理解的进步。在这篇论文中，我们构建了一个名为SAR-Text的大规模高质量数据集，包含超过130,000对SAR图像和描述文本。为了构建SAR-Text数据集，我们设计了一种名为SAR-Narrator的框架，通过多阶段渐进式迁移学习策略为SAR图像生成文字描述。 为了验证SAR-TEXT数据集的有效性，我们在三个典型的视觉语言任务上进行了实验：图像-文本检索、图像字幕生成和视觉问答（VQA）。具体来说，在SAR-TEXT基础上构建了三种代表性模型：SAR-RS-CLIP、SAR-RS-CoCa和SAR-GPT。在图像-文本检索任务中，SAR-RS-CLIP实现了显著的性能提升，在OSdataset-512和HRSID测试集上分别提高了平均召回率16.43%和10.54%。在字幕生成任务中，SAR-RS-CoCa模型在BLEU-4、SPICE和CIDEr评分上均超过了原始CoCa模型8倍、4倍和10倍以上。而在视觉问答（VQA）任务中，SAR-GPT在多个SAR-VQA数据集上优于基准线模型和单阶段模型，展示了更强的语义理解和推理能力，这进一步通过定性结果得到了证实。 值得注意的是，作为一种灵活的文字生成工具，SAR-Narrator可以被社区轻松采用以构建更大规模的SAR图像-文本数据集。
URL
https://arxiv.org/abs/2507.18743
