Abstract
Diffusion-based remote sensing (RS) generative foundation models are cruial for downstream tasks. However, these models rely on large amounts of globally representative data, which often contain redundancy, noise, and class imbalance, reducing training efficiency and preventing convergence. Existing RS diffusion foundation models typically aggregate multiple classification datasets or apply simplistic deduplication, overlooking the distributional requirements of generation modeling and the heterogeneity of RS imagery. To address these limitations, we propose a training-free, two-stage data pruning approach that quickly select a high-quality subset under high pruning ratios, enabling a preliminary foundation model to converge rapidly and serve as a versatile backbone for generation, downstream fine-tuning, and other applications. Our method jointly considers local information content with global scene-level diversity and representativeness. First, an entropy-based criterion efficiently removes low-information samples. Next, leveraging RS scene classification datasets as reference benchmarks, we perform scene-aware clustering with stratified sampling to improve clustering effectiveness while reducing computational costs on large-scale unlabeled data. Finally, by balancing cluster-level uniformity and sample representativeness, the method enables fine-grained selection under high pruning ratios while preserving overall diversity and representativeness. Experiments show that, even after pruning 85\% of the training data, our method significantly improves convergence and generation quality. Furthermore, diffusion foundation models trained with our method consistently achieve state-of-the-art performance across downstream tasks, including super-resolution and semantic image synthesis. This data pruning paradigm offers practical guidance for developing RS generative foundation models.
Abstract (translated)
基于扩散的遥感（RS）生成基础模型对于下游任务至关重要。然而，这些模型依赖于大量具有全球代表性的数据集，这些数据集中通常包含冗余、噪声和类别不平衡的问题，这会降低训练效率并妨碍收敛性。现有的RS扩散基础模型通常通过聚合多个分类数据集或应用简单的去重技术来处理这些问题，但它们往往忽视了生成建模的分布需求以及遥感图像的异质性。 为了克服这些限制，我们提出了一种无训练、两阶段的数据修剪方法，可以在高修剪比例下快速选择高质量子集，从而使初步基础模型能够迅速收敛，并作为多种应用（如生成、下游微调等）的强大骨干。我们的方法同时考虑了局部信息内容与全局场景级别的多样性和代表性。 首先，使用基于熵的准则高效去除低信息量样本。 其次，利用遥感场景分类数据集作为参考基准，我们执行面向场景的聚类，并采用分层抽样以提高大规模未标记数据上的聚类效果并减少计算成本。 最后，通过平衡集群级别的均匀性和样本代表性，在高修剪比例下实现精细选择的同时保持整体多样性和代表性。 实验表明，即使在剪枝85%训练数据后，我们的方法显著提高了收敛速度和生成质量。此外，使用我们方法训练的扩散基础模型在包括超分辨率和语义图像合成在内的下游任务中始终取得最先进的性能水平。这种数据修剪范式为开发RS生成基础模型提供了实用指导。
URL
https://arxiv.org/abs/2512.23239
