Abstract
Attention-based arbitrary style transfer methods have gained significant attention recently due to their impressive ability to synthesize style details. However, the point-wise matching within the attention mechanism may overly focus on local patterns such that neglect the remarkable global features of style images. Additionally, when processing large images, the quadratic complexity of the attention mechanism will bring high computational load. To alleviate above problems, we propose Holistic Style Injector (HSI), a novel attention-style transformation module to deliver artistic expression of target style. Specifically, HSI performs stylization only based on global style representation that is more in line with the characteristics of style transfer, to avoid generating local disharmonious patterns in stylized images. Moreover, we propose a dual relation learning mechanism inside the HSI to dynamically render images by leveraging semantic similarity in content and style, ensuring the stylized images preserve the original content and improve style fidelity. Note that the proposed HSI achieves linear computational complexity because it establishes feature mapping through element-wise multiplication rather than matrix multiplication. Qualitative and quantitative results demonstrate that our method outperforms state-of-the-art approaches in both effectiveness and efficiency.
Abstract (translated)
基于注意力机制的任意风格迁移方法由于其合成样式细节的能力而最近备受关注。然而，注意力机制中的逐点匹配可能会过分注重局部模式，从而忽视了样图中显著的整体特征。此外，在处理大图像时，注意力机制的二次复杂性会带来很高的计算负载。为了解决上述问题，我们提出了一个新颖的关注式风格转换模块——整体风格注入器（Holistic Style Injector, HSI），用于传递目标样式的表现力。具体而言，HSI仅基于全局样式的表示进行风格化，这更符合风格迁移的特性，并且可以避免生成图像中不和谐的地方模式。此外，在HSI内部我们提出了一个双关系学习机制，通过利用内容和样图之间的语义相似性来动态渲染图像，确保样式化的图像保留原始的内容并提高样式的一致性。 值得注意的是，由于HSI通过逐元素相乘而非矩阵相乘建立特征映射，其计算复杂度为线性。定性和定量的结果均表明我们提出的方法在有效性和效率上都优于现有的方法。
URL
https://arxiv.org/abs/2502.04369
