Abstract
Global effective receptive field plays a crucial role for image style transfer (ST) to obtain high-quality stylized results. However, existing ST backbones (e.g., CNNs and Transformers) suffer huge computational complexity to achieve global receptive fields. Recently, the State Space Model (SSM), especially the improved variant Mamba, has shown great potential for long-range dependency modeling with linear complexity, which offers a approach to resolve the above dilemma. In this paper, we develop a Mamba-based style transfer framework, termed SaMam. Specifically, a mamba encoder is designed to efficiently extract content and style information. In addition, a style-aware mamba decoder is developed to flexibly adapt to various styles. Moreover, to address the problems of local pixel forgetting, channel redundancy and spatial discontinuity of existing SSMs, we introduce both local enhancement and zigzag scan. Qualitative and quantitative results demonstrate that our SaMam outperforms state-of-the-art methods in terms of both accuracy and efficiency.
Abstract (translated)
全局有效的感受野在图像风格转换（ST）中扮演着关键角色，以获得高质量的风格化结果。然而，现有的风格转移骨干网络（如CNN和Transformer），为了实现全局感受野，面临着巨大的计算复杂度问题。最近，状态空间模型（SSM），特别是改进版的Mamba，在长距离依赖建模方面表现出线性复杂度的巨大潜力，为解决上述难题提供了可能的方法。本文中，我们开发了一个基于Mamba的风格转换框架，称为SaMam。具体而言，设计了一种高效的mamba编码器来提取内容和样式信息，并且开发了一种感知样式的mamba解码器以灵活适应各种样式。此外，为了应对现有SSMs中存在的局部像素遗忘、通道冗余以及空间不连续性问题，我们引入了局部增强技术和之字形扫描技术。定性和定量的结果都表明，我们的SaMam框架在准确性和效率方面均优于当前最先进的方法。
URL
https://arxiv.org/abs/2503.15934
