Abstract
Few-shot image synthesis entails generating diverse and realistic images of novel categories using only a few example images. While multiple recent efforts in this direction have achieved impressive results, the existing approaches are dependent only upon the few novel samples available at test time in order to generate new images, which restricts the diversity of the generated images. To overcome this limitation, we propose Conditional Distribution Modelling (CDM) -- a framework which effectively utilizes Diffusion models for few-shot image generation. By modelling the distribution of the latent space used to condition a Diffusion process, CDM leverages the learnt statistics of the training data to get a better approximation of the unseen class distribution, thereby removing the bias arising due to limited number of few shot samples. Simultaneously, we devise a novel inversion based optimization strategy that further improves the approximated unseen class distribution, and ensures the fidelity of the generated samples to the unseen class. The experimental results on four benchmark datasets demonstrate the effectiveness of our proposed CDM for few-shot generation.
Abstract (translated)
少量样本图像生成意味着使用仅几张示例图像生成具有新颖类别的新颖且真实的图像。虽然在这个方向上已经有很多最近的尝试取得了令人印象深刻的成果，但现有的方法仅依赖于测试时间有限的几个新颖样本生成新图像，这限制了生成的图像的多样性。为了克服这个限制，我们提出了条件分布建模（CDM）框架——一个有效利用扩散模型进行少量样本图像生成的框架。通过建模用于条件扩散过程的潜在空间分布，CDM利用训练数据的已学习统计量来获得更好的类分布近似，从而消除由于样本数量有限而产生的偏差。同时，我们还设计了一种新的基于优化的策略，进一步改善了类分布的近似程度，并确保生成的样本与原始类别的一致性。在四个基准数据集上的实验结果表明，我们提出的CDM对于少量样本生成非常有效。
URL
https://arxiv.org/abs/2404.16556
