Abstract
Diffusion models have emerged as the dominant paradigm for style transfer, but their text-driven mechanism is hindered by a core limitation: it treats textual descriptions as uniform, monolithic guidance. This limitation overlooks the semantic gap between the non-spatial nature of textual descriptions and the spatially-aware attributes of visual style, often leading to the loss of semantic structure and fine-grained details during stylization. In this paper, we propose StyDeco, an unsupervised framework that resolves this limitation by learning text representations specifically tailored for the style transfer task. Our framework first employs Prior-Guided Data Distillation (PGD), a strategy designed to distill stylistic knowledge without human supervision. It leverages a powerful frozen generative model to automatically synthesize pseudo-paired data. Subsequently, we introduce Contrastive Semantic Decoupling (CSD), a task-specific objective that adapts a text encoder using domain-specific weights. CSD performs a two-class clustering in the semantic space, encouraging source and target representations to form distinct clusters. Extensive experiments on three classic benchmarks demonstrate that our framework outperforms several existing approaches in both stylistic fidelity and structural preservation, highlighting its effectiveness in style transfer with semantic preservation. In addition, our framework supports a unique de-stylization process, further demonstrating its extensibility. Our code is vailable at this https URL.
Abstract (translated)
扩散模型已经成为风格迁移的主导范式，但其基于文本驱动的机制受到一个核心限制：它将文字描述视为统一且单一的指导信息。这一限制忽视了文本描述非空间性质与视觉风格的空间感知属性之间的语义差距，常常导致在风格化过程中丧失语义结构和细粒度细节。 本文提出了一种名为StyDeco的新框架，该框架通过学习专门针对风格迁移任务设计的文本表示来解决上述问题。我们的框架首先采用Prior-Guided Data Distillation (PGD)策略，这是一种无需人类监督即可提取风格知识的方法。它利用一个强大的冻结生成模型自动合成伪配对数据。随后，我们引入了Contrastive Semantic Decoupling (CSD)，这是一个任务特定的目标函数，使用领域特有权重来调整文本编码器。CSD在语义空间中执行两类聚类操作，鼓励源表示和目标表示形成不同的集群。 通过三个经典基准测试的广泛实验表明，我们的框架在风格保真度和结构保留方面均优于多种现有方法，在具有语义保存能力的风格迁移方面表现出显著效果。此外，我们的框架还支持独特的去风格化过程，进一步证明了其可扩展性。代码可在以下链接获取：[此 URL]（请将 [此 URL] 替换为实际的代码库链接）。
URL
https://arxiv.org/abs/2508.01215
