Abstract
Diffusion models have significantly advanced image manipulation techniques, and their ability to generate photorealistic images is beginning to transform retail workflows, particularly in presale visualization. Beyond artistic style transfer, the capability to perform fine-grained visual feature transfer is becoming increasingly important. Embroidery is a textile art form characterized by intricate interplay of diverse stitch patterns and material properties, which poses unique challenges for existing style transfer methods. To explore the customization for such fine-grained features, we propose a novel contrastive learning framework that disentangles fine-grained style and content features with a single reference image, building on the classic concept of image analogy. We first construct an image pair to define the target style, and then adopt a similarity metric based on the decoupled representations of pretrained diffusion models for style-content separation. Subsequently, we propose a two-stage contrastive LoRA modulation technique to capture fine-grained style features. In the first stage, we iteratively update the whole LoRA and the selected style blocks to initially separate style from content. In the second stage, we design a contrastive learning strategy to further decouple style and content through self-knowledge distillation. Finally, we build an inference pipeline to handle image or text inputs with only the style blocks. To evaluate our method on fine-grained style transfer, we build a benchmark for embroidery customization. Our approach surpasses prior methods on this task and further demonstrates strong generalization to three additional domains: artistic style transfer, sketch colorization, and appearance transfer.
Abstract (translated)
扩散模型在图像处理技术方面取得了显著进展，它们生成逼真照片的能力已经开始改变零售工作流程，特别是在预售可视化领域。除了艺术风格转换外，执行细粒度视觉特征转移的能力变得越来越重要。刺绣是一种纺织艺术品形式，以其多样的针法图案和材料特性复杂交织为特点，这对现有的风格迁移方法提出了独特挑战。为了探索对这种细粒度特性的定制，我们提出了一种新颖的对比学习框架，在单个参考图像的基础上分离出细粒度的风格与内容特征，基于经典的概念——图像类比。首先，我们构建一对图象来定义目标风格，然后采用基于预训练扩散模型解耦表示的相似性指标来进行风格和内容的分离。随后，我们提出了一种两阶段对比LoRA调制技术，以捕捉细粒度的风格特征。在第一阶段，我们迭代更新整个LoRA及选定的风格模块来初步分离风格与内容。在第二阶段，我们设计了一个基于自知识蒸馏的对比学习策略，进一步解耦风格和内容。最后，我们建立了一条推理管线，仅使用风格块就可以处理图像或文本输入。 为了评估我们的方法在细粒度风格转换上的性能，我们为刺绣定制建立了基准测试。我们的方法在此任务上超越了先前的方法，并进一步展示了其在三个额外领域中的强大泛化能力：艺术风格转移、草图着色和外观变换。
URL
https://arxiv.org/abs/2509.18948
