Abstract
We present Style Matching Score (SMS), a novel optimization method for image stylization with diffusion models. Balancing effective style transfer with content preservation is a long-standing challenge. Unlike existing efforts, our method reframes image stylization as a style distribution matching problem. The target style distribution is estimated from off-the-shelf style-dependent LoRAs via carefully designed score functions. To preserve content information adaptively, we propose Progressive Spectrum Regularization, which operates in the frequency domain to guide stylization progressively from low-frequency layouts to high-frequency details. In addition, we devise a Semantic-Aware Gradient Refinement technique that leverages relevance maps derived from diffusion semantic priors to selectively stylize semantically important regions. The proposed optimization formulation extends stylization from pixel space to parameter space, readily applicable to lightweight feedforward generators for efficient one-step stylization. SMS effectively balances style alignment and content preservation, outperforming state-of-the-art approaches, verified by extensive experiments.
Abstract (translated)
我们提出了一种新颖的优化方法——风格匹配得分（SMS），用于基于扩散模型的图像风格化。在有效的风格转换与内容保留之间取得平衡是一个长期存在的挑战。不同于现有的努力，我们的方法将图像风格化重新定义为一个风格分布匹配的问题。目标风格分布是通过从现成的依赖于风格的LoRAs中经精心设计的评分函数来估计得出的。为了自适应地保持内容信息，我们提出了渐进频谱正则化技术，在频率域操作，逐步引导风格化过程，先从低频布局到高频细节进行处理。此外，我们还开发了一种语义感知梯度精炼技术，利用由扩散模型语义先验导出的相关图来选择性地对具有重要语义的区域进行风格化处理。所提出的优化方法将图像风格化的范围从像素空间扩展到了参数空间，并且能够轻松应用于轻量级前馈生成器以实现高效的一步式风格化过程。在广泛的实验验证下，SMS有效地平衡了风格一致性和内容保留，优于现有的最先进的方法。
URL
https://arxiv.org/abs/2503.07601
