Abstract
Training-free diffusion-based methods have achieved remarkable success in style transfer, eliminating the need for extensive training or fine-tuning. However, due to the lack of targeted training for style information extraction and constraints on the content image layout, training-free methods often suffer from layout changes of original content and content leakage from style images. Through a series of experiments, we discovered that an effective startpoint in the sampling stage significantly enhances the style transfer process. Based on this discovery, we propose StyleSSP, which focuses on obtaining a better startpoint to address layout changes of original content and content leakage from style image. StyleSSP comprises two key components: (1) Frequency Manipulation: To improve content preservation, we reduce the low-frequency components of the DDIM latent, allowing the sampling stage to pay more attention to the layout of content images; and (2) Negative Guidance via Inversion: To mitigate the content leakage from style image, we employ negative guidance in the inversion stage to ensure that the startpoint of the sampling stage is distanced from the content of style image. Experiments show that StyleSSP surpasses previous training-free style transfer baselines, particularly in preserving original content and minimizing the content leakage from style image.
Abstract (translated)
无需训练的基于扩散的方法在风格转换方面取得了显著的成功，消除了广泛训练或微调的需求。然而，由于缺乏针对风格信息提取的目标训练以及对内容图像布局的限制，无训练方法通常会遭受原始内容布局变化和从风格图像中渗入内容的问题。通过一系列实验，我们发现采样阶段的有效起点可以极大地提升风格转换过程的效果。基于这一发现，我们提出了StyleSSP，它专注于获得更好的起点来解决原始内容布局的变化以及从风格图像中渗入的内容问题。StyleSSP包含两个关键组成部分：（1）频率操控：为了提高内容保留效果，我们将DDIM潜在空间中的低频成分减少，使采样阶段可以更多地关注内容图像的布局；（2）通过反演操作实现负向指导：为了解决从风格图像中渗入的内容问题，我们在反演阶段采用负向指导来确保采样阶段的起点远离了风格图像的内容。实验表明，StyleSSP在保留原始内容和减少来自风格图像的内容渗漏方面超越了之前的无训练风格转换基线方法。
URL
https://arxiv.org/abs/2501.11319
