Abstract
While there has been significant progress in customizing text-to-image generation models, generating images that combine multiple personalized concepts remains challenging. In this work, we introduce Concept Weaver, a method for composing customized text-to-image diffusion models at inference time. Specifically, the method breaks the process into two steps: creating a template image aligned with the semantics of input prompts, and then personalizing the template using a concept fusion strategy. The fusion strategy incorporates the appearance of the target concepts into the template image while retaining its structural details. The results indicate that our method can generate multiple custom concepts with higher identity fidelity compared to alternative approaches. Furthermore, the method is shown to seamlessly handle more than two concepts and closely follow the semantic meaning of the input prompt without blending appearances across different subjects.
Abstract (translated)
尽管在自定义文本到图像生成模型的定制方面已经取得了显著的进展，但生成结合多个个性化概念的图像仍然具有挑战性。在这项工作中，我们引入了Concept Weaver方法，一种在推理时组合自定义文本到图像扩散模型的方法。具体来说，方法分为两个步骤：创建与输入提示的语义对齐的模板图像，然后使用概念融合策略个性化模板。融合策略将目标概念的 appearance 融入模板图像中，同时保留其结构细节。结果显示，与 alternative 方法相比，我们的方法可以生成具有更高identity fidelity 的多个自定义概念。此外，该方法被证明可以轻松处理超过两个概念，并且可以紧密跟踪输入提示的语义意义，而不会在不同的主题之间混淆外观。
URL
https://arxiv.org/abs/2404.03913
