Abstract
Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining. In this work, we propose a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain any use-specific components. We show that our algorithm successfully generates quality images with guidance functions including segmentation, face recognition, object detection, and classifier signals. Code is available at this https URL.
Abstract (translated)
典型的扩散模型被训练接受特定的 conditioning 形式,最常见的是文字,并且无法在没有重新训练的情况下接受其他形式的训练。在本文中,我们提出了一种通用的指导算法,它使扩散模型能够以任意的指导模式来控制,而无需重新训练任何特定的组件。我们展示了我们的算法成功地生成包括 segmentation、人脸识别、物体检测和分类信号的指导函数高质量的图像。代码在此 https URL 可用。
URL
https://arxiv.org/abs/2302.07121
