Abstract
Remote sensing image classification forms the foundation of various understanding tasks, serving a crucial function in remote sensing image interpretation. The recent advancements of Convolutional Neural Networks (CNNs) and Transformers have markedly enhanced classification accuracy. Nonetheless, remote sensing scene classification remains a significant challenge, especially given the complexity and diversity of remote sensing scenarios and the variability of spatiotemporal resolutions. The capacity for whole-image understanding can provide more precise semantic cues for scene discrimination. In this paper, we introduce RSMamba, a novel architecture for remote sensing image classification. RSMamba is based on the State Space Model (SSM) and incorporates an efficient, hardware-aware design known as the Mamba. It integrates the advantages of both a global receptive field and linear modeling complexity. To overcome the limitation of the vanilla Mamba, which can only model causal sequences and is not adaptable to two-dimensional image data, we propose a dynamic multi-path activation mechanism to augment Mamba's capacity to model non-causal data. Notably, RSMamba maintains the inherent modeling mechanism of the vanilla Mamba, yet exhibits superior performance across multiple remote sensing image classification datasets. This indicates that RSMamba holds significant potential to function as the backbone of future visual foundation models. The code will be available at \url{this https URL}.
Abstract (translated)
遥感图像分类是各种理解任务的基石，在遥感图像解释中具有关键作用。最近卷积神经网络（CNN）和Transformer的进步显著提高了分类准确性。然而，遥感场景分类仍然是一个重要的挑战，尤其是在遥感场景的复杂性和多样性以及时空分辨率的不确定性方面。全图理解能力可以提供场景区分的更精确的语义线索。在本文中，我们引入了RSMamba，一种新型的遥感图像分类架构。RSMamba基于状态空间模型（SSM），并采用了一种高效且硬件意识的设计，称为Mamba。它整合了全局感受野和线性建模复杂度的优势。为了克服普通Mamba的局限性（只能建模因果序列，不适用于二维图像数据），我们提出了动态多路径激活机制来增强Mamba的建模非因果数据的能力。值得注意的是，RSMamba保留了普通Mamba的固有建模机制，同时在多个遥感图像分类数据集上表现出卓越的性能。这表明，RSMamba具有成为未来视觉基础模型 backbone的重要潜力。代码将公开在 \url{这个链接} 上。
URL
https://arxiv.org/abs/2403.19654
