Abstract
Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.
Abstract (translated)
合成孔径雷达（SAR）图像提供全天候、全时段和高分辨率的成像能力，但其独特的成像机制使得解释工作高度依赖专家知识，限制了可解释性，特别是在复杂的任务目标中。将SAR图像转换为光学图像是增强解释和支持下游任务的一种有前景的方法。现有的大多数研究集中在场景级别的翻译上，由于配对数据稀缺以及准确保留轮廓和纹理细节的挑战，物体级别翻译的研究相对较少。 为了应对这些问题，本研究提出了一种关键点引导扩散模型（KeypointDiff），用于未配对飞机目标的SAR到光学图像转换。该框架通过关键点在目标类别和方位角方面引入监督，并针对未配对数据设计了训练策略。基于无分类器指导的扩散架构，设计了一个类-角度引导模块（CAGM），将类别和角度信息整合到了扩散生成过程中。此外，对抗损失和一致性损失被用来提高图像的真实感和细节质量，特别适用于飞机目标。 在采样期间，在预训练的关键点检测器的帮助下，该模型消除了手动标注的类别和方位角信息需求，实现了自动化的SAR到光学转换。实验结果表明，所提出的方法在多个指标上优于现有方法，为物体级别的SAR到光学图像转换以及下游任务提供了高效且有效的解决方案。此外，在关键点检测器的支持下，该方法展示了对未训练飞机类型强大的零样本泛化能力。
URL
https://arxiv.org/abs/2503.19798
