Abstract
Pretraining and fine-tuning have emerged as a new paradigm in remote sensing image interpretation. Among them, Masked Autoencoder (MAE)-based pretraining stands out for its strong capability to learn general feature representations via reconstructing masked image regions. However, applying MAE to multispectral remote sensing images remains challenging due to complex backgrounds, indistinct targets, and the lack of semantic guidance during masking, which hinders the learning of underlying structures and meaningful spatial-spectral features. To address this, we propose a simple yet effective approach, Spectral Index-Guided MAE (SIGMAE), for multispectral image pretraining. The core idea is to incorporate domain-specific spectral indices as prior knowledge to guide dynamic token masking toward informative regions. SIGMAE introduces Semantic Saliency-Guided Dynamic Token Masking (SSDTM), a curriculum-style strategy that quantifies each patch's semantic richness and internal heterogeneity to adaptively select the most informative tokens during training. By prioritizing semantically salient regions and progressively increasing sample difficulty, SSDTM enhances spectrally rich and structurally aware representation learning, mitigates overfitting, and reduces redundant computation compared with random masking. Extensive experiments on five widely used datasets covering various downstream tasks, including scene classification, semantic segmentation, object extraction and change detection, demonstrate that SIGMAE outperforms other pretrained geospatial foundation models. Moreover, it exhibits strong spatial-spectral reconstruction capability, even with a 90% mask ratio, and improves complex target recognition under limited labeled data. The source codes and model weights will be released at this https URL.
Abstract (translated)
预训练和微调已成为遥感图像解释的新范式。其中，基于遮罩自动编码器（MAE）的预训练因其通过重构被遮罩的图像区域来学习通用特征表示的能力而脱颖而出。然而，由于复杂的背景、不清晰的目标以及在遮罩过程中缺乏语义指导，将MAE应用于多光谱遥感图像仍然具有挑战性，这阻碍了底层结构和有意义的空间-光谱特征的学习。为了解决这个问题，我们提出了一种简单而有效的方法——基于光谱指数的MAE（SIGMAE），用于多光谱图像预训练。核心思想是将领域特定的光谱指数作为先验知识，以指导动态标记遮罩向信息丰富的区域进行导向。 SIGMAE引入了语义显著性引导的动态令牌遮罩（SSDTM）策略，这是一种类似课程的学习方式，它量化每个补丁的语义丰富性和内部异质性，并在训练过程中自适应地选择最富有信息性的令牌。通过优先考虑语义显著区域并逐步增加样本难度，SSDTM增强了光谱丰富的和结构感知表示学习，减轻了过拟合问题，并与随机遮罩相比减少了冗余计算。 在五个广泛使用的数据集上进行了大量实验，这些数据集涵盖了包括场景分类、语义分割、目标提取和变化检测在内的各种下游任务。结果表明，SIGMAE优于其他预训练的地理空间基础模型。此外，即使在90%的遮罩比例下，它也展示了强大的光谱-空间重建能力，并且在标签有限的情况下提高了复杂目标识别的能力。 源代码和模型权重将在以下链接发布：[提供URL]
URL
https://arxiv.org/abs/2603.07463
