Abstract
Deep learning models benefit from increasing data diversity and volume, motivating synthetic data augmentation to improve existing datasets. However, existing evaluation metrics for synthetic data typically calculate latent feature similarity, which is difficult to interpret and does not always correlate with the contribution to downstream tasks. We propose a vision-language grounded framework for interpretable synthetic data augmentation and evaluation in remote sensing. Our approach combines generative models, semantic segmentation and image captioning with vision and language models. Based on this framework, we introduce ARAS400k: A large-scale Remote sensing dataset Augmented with Synthetic data for segmentation and captioning, containing 100k real images and 300k synthetic images, each paired with segmentation maps and descriptions. ARAS400k enables the automated evaluation of synthetic data by analyzing semantic composition, minimizing caption redundancy, and verifying cross-modal consistency between visual structures and language descriptions. Experimental results indicate that while models trained exclusively on synthetic data reach competitive performance levels, those trained with augmented data (a combination of real and synthetic images) consistently outperform real-data baselines. Consequently, this work establishes a scalable benchmark for remote sensing tasks, specifically in semantic segmentation and image captioning. The dataset is available at this http URL and the code base at this http URL.
Abstract (translated)
深度学习模型从数据多样性和数量的增加中受益，这促使了合成数据增强以改善现有数据集。然而，现有的用于评估合成数据的方法通常计算潜在特征相似性，这种方法难以解释，并且并不总是与对下游任务的影响相关联。我们提出了一种基于视觉和语言的理解框架，旨在促进遥感领域的可解释的合成数据增强和评估。我们的方法结合了生成模型、语义分割以及图像描述（或称为图像配词）技术与视觉及语言模型。 基于这一框架，我们引入了ARAS400k：这是一个大型远程传感数据集，通过合成数据进行了扩充，用于语义分割和图像描述任务，该数据集中包含10万张真实图片和30万张合成图片，每一张图片都配有对应的语义分割图和描述文本。ARAS400k使得可以通过分析语义组成、减少描述的冗余性以及验证视觉结构与语言描述之间的跨模态一致性来进行自动化的合成数据评估。 实验结果显示，在仅使用合成数据训练模型可以达到竞争性的性能水平，但在结合真实图像和合成图像（即增强数据）进行训练时，这些模型始终优于基于真实数据的基线模型。因此，这项工作为遥感任务提供了可扩展的基准测试，特别是在语义分割和图像描述领域。 该数据集可在提供的链接获取，代码库也可在相应的网址上找到。
URL
https://arxiv.org/abs/2603.09625
