Abstract
Mamba, a recent selective structured state space model, performs excellently on long sequence modeling tasks. Mamba mitigates the modeling constraints of convolutional neural networks and offers advanced modeling capabilities similar to those of Transformers, through global receptive fields and dynamic weighting. Crucially, it achieves this without incurring the quadratic computational complexity typically associated with Transformers. Due to its advantages over the former two mainstream foundation models, Mamba exhibits great potential to be a visual foundation model. Researchers are actively applying Mamba to various computer vision tasks, leading to numerous emerging works. To help keep pace with the rapid advancements in computer vision, this paper aims to provide a comprehensive review of visual Mamba approaches. This paper begins by delineating the formulation of the original Mamba model. Subsequently, our review of visual Mamba delves into several representative backbone networks to elucidate the core insights of the visual Mamba. We then categorize related works using different modalities, including image, video, point cloud, multi-modal, and others. Specifically, for image applications, we further organize them into distinct tasks to facilitate a more structured discussion. Finally, we discuss the challenges and future research directions for visual Mamba, providing insights for future research in this quickly evolving area. A comprehensive list of visual Mamba models reviewed in this work is available at this https URL.
Abstract (translated)
Mamba是一个最近的选择性结构化状态空间模型，在长序列建模任务中表现出色。Mamba通过全局感受野和动态权重减轻了卷积神经网络的建模约束，并提供了与Transformer相似的建模能力。关键是，它在不产生通常与Transformer相关的二次计算复杂性的情况下实现了这一点。由于其在前两种主流基础模型上的优势，Mamba在视觉领域具有巨大的潜在成为视觉基础模型的潜力。研究人员正积极将Mamba应用于各种计算机视觉任务，导致了许多新兴的工作。为了跟上计算机视觉领域快速发展的步伐，本文旨在对视觉Mamba方法进行全面回顾。本文首先概述了原始Mamba模型的表述。接着，我们对视觉Mamba进行了深入研究，以阐明视觉Mamba的核心见解。然后，我们根据不同的模式对相关研究进行分类，包括图像、视频、点云、多模态等。特别地，对于图像应用，我们进一步将它们划分为不同的任务，以促进更结构化的讨论。最后，我们讨论了视觉Mamba的挑战和未来研究方向，为未来研究在快速发展的领域提供了见解。本文全面回顾的视觉Mamba模型列表可以在该https URL找到。
URL
https://arxiv.org/abs/2404.18861
