Abstract
Video super-resolution (VSR) faces critical challenges in effectively modeling non-local dependencies across misaligned frames while preserving computational efficiency. Existing VSR methods typically rely on optical flow strategies or transformer architectures, which struggle with large motion displacements and long video sequences. To address this, we propose MambaVSR, the first state-space model framework for VSR that incorporates an innovative content-aware scanning mechanism. Unlike rigid 1D sequential processing in conventional vision Mamba methods, our MambaVSR enables dynamic spatiotemporal interactions through the Shared Compass Construction (SCC) and the Content-Aware Sequentialization (CAS). Specifically, the SCC module constructs intra-frame semantic connectivity graphs via efficient sparse attention and generates adaptive spatial scanning sequences through spectral clustering. Building upon SCC, the CAS module effectively aligns and aggregates non-local similar content across multiple frames by interleaving temporal features along the learned spatial order. To bridge global dependencies with local details, the Global-Local State Space Block (GLSSB) synergistically integrates window self-attention operations with SSM-based feature propagation, enabling high-frequency detail recovery under global dependency guidance. Extensive experiments validate MambaVSR's superiority, outperforming the Transformer-based method by 0.58 dB PSNR on the REDS dataset with 55% fewer parameters.
Abstract (translated)
视频超分辨率（VSR）面临着一个关键挑战，即如何有效地建模不同步帧之间的非局部依赖关系，同时保持计算效率。现有的VSR方法通常依赖于光学流策略或变压器架构，这些方法在处理大运动位移和长视频序列时存在困难。为了解决这些问题，我们提出了一种名为MambaVSR的框架，这是首个基于状态空间模型（SSM）的VSR框架，它集成了创新的内容感知扫描机制。 与传统的视觉Mamba方法中刚性的1D顺序处理不同，我们的MambaVSR通过共享罗盘构造（SCC）和内容感知序列化（CAS）实现了动态时空交互。具体来说，SCC模块通过高效的稀疏注意力构建了帧内语义连接图，并利用谱聚类生成自适应空间扫描序列。在此基础上，CAS模块通过在学习到的空间顺序中交错时间特征来有效地对多个帧中的非局部相似内容进行对齐和聚合。 为了将全局依赖关系与局部细节相结合，我们提出了全局-局部状态空间块（GLSSB），该模块巧妙地结合了窗口自我注意力操作和基于SSM的特性传播，使得在全局依赖性的指导下能够恢复高频细节。 广泛的实验验证了MambaVSR的优势，在RED数据集上超越了基于Transformer的方法0.58 dB PSNR，同时参数减少了55%。
URL
https://arxiv.org/abs/2506.11768
