Abstract
Recent Super-Resolution~(SR) methods mainly adopt Transformers for their strong long-range modeling capability and exceptional representational capacity. However, most SR Transformers rely heavily on relative positional bias~(RPB), which prevents them from leveraging hardware-efficient attention kernels such as FlashAttention. This limitation imposes a prohibitive computational burden during both training and inference, severely restricting attempts to scale SR Transformers by enlarging the training patch size or the self-attention window. Consequently, unlike other domains that actively exploit the inherent scalability of Transformers, SR Transformers remain heavily focused on effectively utilizing limited receptive fields. In this paper, we propose Rank-factorized Implicit Neural Bias~(RIB), an alternative to RPB that enables FlashAttention in SR Transformers. Specifically, RIB approximates positional bias using low-rank implicit neural representations and concatenates them with pixel content tokens in a channel-wise manner, turning the element-wise bias addition in attention score computation into a dot-product operation. Further, we introduce a convolutional local attention and a cyclic window strategy to fully leverage the advantages of long-range interactions enabled by RIB and FlashAttention. We enlarge the window size up to \textbf{96$\times$96} while jointly scaling the training patch size and the dataset size, maximizing the benefits of Transformers in the SR task. As a result, our network achieves \textbf{35.63\,dB PSNR} on Urban100$\times$2, while reducing training and inference time by \textbf{2.1$\times$} and \textbf{2.9$\times$}, respectively, compared to the RPB-based SR Transformer~(PFT).
Abstract (translated)
近期的超分辨率(SR)方法主要采用Transformer架构，因其强大的长程建模能力和卓越的表现力。然而，大多数SR Transformer严重依赖相对位置偏差(RPB)，这阻碍了它们利用高效的硬件注意力内核（如FlashAttention）。这种限制在训练和推理过程中带来了巨大的计算负担，并且极大地限制了通过增加训练补丁尺寸或自注意力窗口来扩展SR Transformers的尝试。因此，与积极利用Transformer固有可扩展性的其他领域不同，SR Transformer仍然集中于有效地使用有限的感受野。 在这篇论文中，我们提出了一个名为秩因子化隐式神经偏差(RIB)的方法，这是一种替代RPB的方法，它使SR Transformer能够采用FlashAttention。具体来说，RIB通过低秩的隐式神经表示来近似位置偏差，并以通道方式将其与像素内容令牌拼接在一起，从而将注意力得分计算中的逐元素偏差相加转换为点积操作。 此外，我们还引入了卷积局部注意机制和循环窗口策略，以便充分利用由RIB和FlashAttention提供的长程交互的优势。我们将窗口大小扩大到**96×96**，同时扩展训练补丁尺寸和数据集的规模，最大化Transformer在SR任务中的优势。结果表明，我们的网络在Urban100x2数据集上实现了**35.63 dB PSNR** 的性能，相比基于RPB的SR Transformer（PFT）减少了**2.1倍**的训练时间以及**2.9倍**的推理时间。
URL
https://arxiv.org/abs/2603.06738
