Abstract
State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at this https URL.
Abstract (translated)
状态空间模型（SSMs），尤其是近期提出的Mamba架构，已在序列建模任务中取得显著成功。然而，由于视觉数据的非序列结构及其复杂的二维空间依赖关系，将SSMs扩展至计算机视觉领域仍面临挑战。尽管多项早期研究已探索针对视觉应用的自适应SSMs，但大多数方法主要依赖对同一输入采用多种遍历策略，这引入了冗余并扭曲了图像内部精细的空间关系。为应对这些挑战，我们提出了MFil-Mamba——一种基于多滤波扫描主干的新型视觉状态空间架构。与固定的多方向遍历方法不同，我们的设计使每次扫描能够捕获独特且与上下文相关的空间信息，同时最大限度减少冗余。此外，除架构改进外，我们还融入了自适应加权机制以有效融合多扫描输出。MFil-Mamba在图像分类、目标检测、实例分割和语义分割等多个基准测试中，均实现了优于现有最先进模型的卓越性能。例如，我们的轻量级变体在ImageNet-1K上达到83.2%的Top-1准确率，在MS COCO上取得47.3%的框AP和42.7%的掩码AP，并在ADE20K数据集上实现48.5%的mIoU。代码与模型已公开于此https链接。
URL
https://arxiv.org/abs/2603.20074
