Abstract
Open-source pre-trained models hold great potential for diverse applications, but their utility declines when their training data is unavailable. Data-Free Image Synthesis (DFIS) aims to generate images that approximate the learned data distribution of a pre-trained model without accessing the original data. However, existing DFIS meth ods produce samples that deviate from the training data distribution due to the lack of prior knowl edge about natural images. To overcome this limitation, we propose DDIS, the first Diffusion-assisted Data-free Image Synthesis method that leverages a text-to-image diffusion model as a powerful image prior, improving synthetic image quality. DDIS extracts knowledge about the learned distribution from the given model and uses it to guide the diffusion model, enabling the generation of images that accurately align with the training data distribution. To achieve this, we introduce Domain Alignment Guidance (DAG) that aligns the synthetic data domain with the training data domain during the diffusion sampling process. Furthermore, we optimize a single Class Alignment Token (CAT) embedding to effectively capture class-specific attributes in the training dataset. Experiments on PACS and Ima geNet demonstrate that DDIS outperforms prior DFIS methods by generating samples that better reflect the training data distribution, achieving SOTA performance in data-free applications.
Abstract (translated)
开源的预训练模型在各种应用中具有巨大潜力，但当这些模型的训练数据不可用时，它们的实用性会下降。无数据图像合成（DFIS）旨在生成与预训练模型所学的数据分布相近的图像，而无需访问原始数据。然而，现有的DFIS方法由于缺乏关于自然图像的先验知识，会产生偏离训练数据分布的样本。为了解决这一局限性，我们提出了DDIS——首个利用文本到图像扩散模型作为强大图像先验的知识辅助无数据图像合成方法，从而提高了合成图像的质量。DDIS从给定的模型中提取有关所学分布的知识，并将其用于指导扩散模型，在生成与训练数据分布准确对齐的图像方面发挥了作用。 为了实现这一目标，我们引入了领域对齐引导（DAG），在扩散采样过程中使合成数据域与训练数据域对齐。此外，我们优化了一个单一的类别对齐标记（CAT）嵌入，以有效捕捉训练数据集中的特定类别属性。在PACS和ImageNet上的实验表明，DDIS优于以前的DFIS方法，生成的样本更准确地反映了训练数据分布，在无数据应用中实现了最先进的性能。
URL
https://arxiv.org/abs/2506.15381
