Abstract
With the rapid progress of controllable generation, training data synthesis has become a promising way to expand labeled datasets and alleviate manual annotation in remote sensing (RS). However, the complexity of semantic mask control and the uncertainty of sampling quality often limit the utility of synthetic data in downstream semantic segmentation tasks. To address these challenges, we propose a task-oriented data synthesis framework (TODSynth), including a Multimodal Diffusion Transformer (MM-DiT) with unified triple attention and a plug-and-play sampling strategy guided by task feedback. Built upon the powerful DiT-based generative foundation model, we systematically evaluate different control schemes, showing that a text-image-mask joint attention scheme combined with full fine-tuning of the image and mask branches significantly enhances the effectiveness of RS semantic segmentation data synthesis, particularly in few-shot and complex-scene scenarios. Furthermore, we propose a control-rectify flow matching (CRFM) method, which dynamically adjusts sampling directions guided by semantic loss during the early high-plasticity stage, mitigating the instability of generated images and bridging the gap between synthetic data and downstream segmentation tasks. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art controllable generation methods, producing more stable and task-oriented synthetic data for RS semantic segmentation.
Abstract (translated)
随着可控生成技术的迅速发展，训练数据合成已成为扩大标注数据集和减轻远程 sensing（RS）中人工注释负担的一种有前景的方法。然而，语义掩码控制的复杂性和采样质量的不确定性往往限制了合成数据在下游语义分割任务中的实用性。为了解决这些挑战，我们提出了一种任务导向的数据合成框架（TODSynth），该框架包括一个具有统一三元注意力机制的多模态扩散变换器（MM-DiT）和一种由任务反馈引导的即插即用采样策略。 基于强大的DiT基础生成模型，我们系统地评估了不同的控制方案，发现结合文本-图像-掩码联合注意机制以及对图像和掩码分支进行完全微调的方案显著提高了RS语义分割数据合成的有效性，特别是在少量样本和复杂场景的情况下尤为明显。此外，我们提出了一种基于语义损失引导动态调整采样方向的方法——控制校正流匹配（CRFM），这种方法在生成图像早期高可塑性的阶段缓解了生成图像的不稳定性，并且缩小了合成数据与下游分割任务之间的差距。 广泛的实验表明，我们的方法持续优于现有的可控生成技术，在产生更为稳定和面向任务的RS语义分割合成数据方面表现出色。
URL
https://arxiv.org/abs/2512.16740
