Abstract
Despite the advancements in diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region segmentation (adaptive clustering on latent diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise alignment); and style transfer optimization (energy function-guided diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.
Abstract (translated)
尽管基于扩散的图像风格迁移技术取得了进步，现有的方法通常受限于以下几点：1)语义差距：风格参考可能缺少适当的内容语义，导致无法控制的风格化；2)对额外约束（如语义蒙版）的依赖，限制了适用性；3)刚性的特征关联缺乏自适应全局-局部对齐能力，在精细风格化和全局内容保留之间难以取得平衡。这些局限性，特别是不能灵活利用风格输入的问题，从根本上限制了风格迁移在个性化、准确性和适应性方面的表现。 为了解决这些问题，我们提出了StyleGallery，这是一个无需训练且具有语义感知的框架，支持任意参考图像作为输入，并能够实现有效的个性化定制。它包含三个核心阶段：语义区域分割（通过隐式扩散特征自适应聚类来划分区域，不依赖额外输入）；集群区域匹配（在提取的特征上执行块过滤以实现精确对齐）；以及风格迁移优化（使用能量函数指导下的扩散采样和区域样式损失来优化风格化）。我们在引入的基准测试上的实验表明，StyleGallery 在内容结构保留、区域风格化、可解释性和个性化定制方面优于现有最先进的方法，尤其是在利用多个风格参考时表现尤为突出。
URL
https://arxiv.org/abs/2603.10354
