Abstract
Recent advancement in text-to-image models (e.g., Stable Diffusion) and corresponding personalized technologies (e.g., DreamBooth and LoRA) enables individuals to generate high-quality and imaginative images. However, they often suffer from limitations when generating images with resolutions outside of their trained domain. To overcome this limitation, we present the Resolution Adapter (ResAdapter), a domain-consistent adapter designed for diffusion models to generate images with unrestricted resolutions and aspect ratios. Unlike other multi-resolution generation methods that process images of static resolution with complex post-process operations, ResAdapter directly generates images with the dynamical resolution. Especially, after learning a deep understanding of pure resolution priors, ResAdapter trained on the general dataset, generates resolution-free images with personalized diffusion models while preserving their original style domain. Comprehensive experiments demonstrate that ResAdapter with only 0.5M can process images with flexible resolutions for arbitrary diffusion models. More extended experiments demonstrate that ResAdapter is compatible with other modules (e.g., ControlNet, IP-Adapter and LCM-LoRA) for image generation across a broad range of resolutions, and can be integrated into other multi-resolution model (e.g., ElasticDiffusion) for efficiently generating higher-resolution images. Project link is this https URL
Abstract (translated)
近年来，文本到图像模型（例如，Stable Diffusion）及其相应个性化技术（例如，DreamBooth 和 LoRA）的进步使得个人能够生成高质量和富有想象力的图像。然而，在生成训练领域之外的高分辨率图像时，它们往往存在局限性。为了克服这一局限，我们提出了分辨率适配器（ResAdapter），一种针对扩散模型的领域一致的适配器，用于生成无限制分辨率和平衡比的图像。与其他多分辨率生成方法不同，ResAdapter直接生成具有动态分辨率的图像。特别是在学习了对纯分辨率 prior 的深刻理解后，ResAdapter在训练通用数据集的同时，使用个性化的扩散模型生成具有个人风格域的分辨率无限制的图像。全面的实验证明，ResAdapter仅需0.5M即可处理任意扩散模型的灵活分辨率图像。更广泛的实验证明，ResAdapter与其他模块（例如，控制网、IP-适配器和LCM-LoRA）在各种分辨率范围内生成图像兼容，并可以集成到其他多分辨率模型（例如，ElasticDiffusion）中，以高效生成高分辨率图像。项目链接是：<https://www.projectlink.io/project/resadapter>
URL
https://arxiv.org/abs/2403.02084
