Abstract
Recently, conditional diffusion models have gained popularity in numerous applications due to their exceptional generation ability. However, many existing methods are training-required. They need to train a time-dependent classifier or a condition-dependent score estimator, which increases the cost of constructing conditional diffusion models and is inconvenient to transfer across different conditions. Some current works aim to overcome this limitation by proposing training-free solutions, but most can only be applied to a specific category of tasks and not to more general conditions. In this work, we propose a training-Free conditional Diffusion Model (FreeDoM) used for various conditions. Specifically, we leverage off-the-shelf pre-trained networks, such as a face detection model, to construct time-independent energy functions, which guide the generation process without requiring training. Furthermore, because the construction of the energy function is very flexible and adaptable to various conditions, our proposed FreeDoM has a broader range of applications than existing training-free methods. FreeDoM is advantageous in its simplicity, effectiveness, and low cost. Experiments demonstrate that FreeDoM is effective for various conditions and suitable for diffusion models of diverse data domains, including image and latent code domains.
Abstract (translated)
近年来,条件扩散模型因其出色的生成能力而在多个应用中获得了 popular 。然而,许多现有方法需要训练。它们需要训练一个时间依赖性分类器或条件依赖得分估计器,这增加了构建条件扩散模型的成本,并不利于在不同条件下的转移。一些当前工作旨在提出不需要训练的解决方案,但大多数只能应用于特定类别的任务,而不是更一般的条件。在本工作中,我们提出了一种不需要训练的训练-free条件扩散模型(FreeDoM),适用于各种条件。具体来说,我们利用现有预训练网络,如人脸识别模型,构建时间 independent 的能量函数,无需训练指导生成过程。此外,由于能量函数的构建非常灵活和适应各种条件,我们提出的 FreeDoM 比现有的不需要训练的方法应用范围更广。FreeDoM 的优点在于它的简单性、有效性和低成本。实验表明,FreeDoM 适用于各种条件,适用于包括图像和隐含代码 domains 等多种数据 domains 的条件扩散模型。
URL
https://arxiv.org/abs/2303.09833
