Abstract
State Space Model (SSM)-based machine learning architectures have recently gained significant attention for processing sequential data. Mamba, a recent sequence-to-sequence SSM, offers competitive accuracy with superior computational efficiency compared to state-of-the-art transformer models. While this advantage makes Mamba particularly promising for resource-constrained edge devices, no hardware acceleration frameworks are currently optimized for deploying it in such environments. This paper presents eMamba, a comprehensive end-to-end hardware acceleration framework explicitly designed for deploying Mamba models on edge platforms. eMamba maximizes computational efficiency by replacing complex normalization layers with lightweight hardware-aware alternatives and approximating expensive operations, such as SiLU activation and exponentiation, considering the target applications. Then, it performs an approximation-aware neural architecture search (NAS) to tune the learnable parameters used during approximation. Evaluations with Fashion-MNIST, CIFAR-10, and MARS, an open-source human pose estimation dataset, show eMamba achieves comparable accuracy to state-of-the-art techniques using 1.63-19.9$\times$ fewer parameters. In addition, it generalizes well to large-scale natural language tasks, demonstrating stable perplexity across varying sequence lengths on the WikiText2 dataset. We also quantize and implement the entire eMamba pipeline on an AMD ZCU102 FPGA and ASIC using GlobalFoundries (GF) 22 nm technology. Experimental results show 4.95-5.62$\times$ lower latency and 2.22-9.95$\times$ higher throughput, with 4.77$\times$ smaller area, 9.84$\times$ lower power, and 48.6$\times$ lower energy consumption than baseline solutions while maintaining competitive accuracy.
Abstract (translated)
基于状态空间模型（SSM）的机器学习架构最近在处理序列数据方面引起了广泛关注。Mamba，一种近期推出的序列到序列SSM，在准确性上与最先进的Transformer模型相当，但在计算效率上更胜一筹。虽然这一优势使得Mamba特别有希望应用于资源受限的边缘设备中，但目前还没有专门针对此类环境部署Mamba的硬件加速框架。本文介绍了eMamba，这是一个全面的端到端硬件加速框架，专为在边缘平台上部署Mamba模型而设计。 eMamba通过用轻量级且适合硬件实现的替代层替换复杂的归一化层，并对昂贵的操作（如SiLU激活和指数运算）进行近似处理来最大化计算效率。同时，它还执行了感知近似的神经架构搜索（NAS），以调整在近似过程中使用的可学习参数。使用Fashion-MNIST、CIFAR-10以及开源人体姿态估计数据集MARS的评估显示，eMamba实现了与最新技术相当的准确性，并且所需的模型参数减少了1.63到19.9倍。 此外，它能够很好地泛化至大规模自然语言任务，在WikiText2数据集中，无论序列长度如何，都能表现出稳定的困惑度。我们还使用GlobalFoundries（GF）22nm技术在AMD ZCU102 FPGA和ASIC上实现了整个eMamba管道的量化实施。实验结果表明，与基线解决方案相比，eMamba的延迟降低了4.95到5.62倍，吞吐量提高了2.22到9.95倍，并且面积减少了4.77倍，功耗降低至原来的十分之一（9.84倍），能耗更是减少到了之前的四十八点六倍，同时保持了相当的准确性。
URL
https://arxiv.org/abs/2508.10370
