Abstract
Satellite image time series (SITS) segmentation is crucial for many applications like environmental monitoring, land cover mapping and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine grained annotation. We propose S4 a new self-supervised pre-training approach that significantly reduces the requirement for labeled training data by utilizing two new insights: (a) Satellites capture images in different parts of the spectrum such as radio frequencies, and visible frequencies. (b) Satellite imagery is geo-registered allowing for fine-grained spatial alignment. We use these insights to formulate pre-training tasks in S4. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially-aligned, multi-modal and geographic specific SITS that serves as representative pre-training data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data.
Abstract (translated)
卫星图像时间序列（SITS）分割对于许多应用，如环境监测、土地覆盖图和农业作物类型分类，至关重要。然而，为SITS分割训练模型仍然具有挑战性，因为缺乏丰富的训练数据，这需要精细标注。我们提出S4一种新的自监督预训练方法，它通过利用两个新的见解显著减少了标注训练数据的需求：（一）卫星捕捉不同频段（如无线电频段和可见频段）的图像；（二）卫星影像实现空间对齐，实现细粒度空间对齐。我们利用这些见解为S4中的预训练任务制定形式。我们还整理了m2s2-SITS，一个大量的、无标签、空间对齐、多模态和地理特定的SITS数据集，作为S4的代表性预训练数据。最后，我们在多个SITS分割数据集上评估S4，并使用有限的标记数据证明了其对抗性基线的有效性。
URL
https://arxiv.org/abs/2405.01656
