Abstract
Recent advances in deep learning have made it possible to predict phenotypic measures directly from functional magnetic resonance imaging (fMRI) brain volumes, sparking significant interest in the neuroimaging community. However, existing approaches, primarily based on convolutional neural networks or transformer architectures, often struggle to model the complex relationships inherent in fMRI data, limited by their inability to capture long-range spatial and temporal dependencies. To overcome these shortcomings, we introduce BrainMT, a novel hybrid framework designed to efficiently learn and integrate long-range spatiotemporal attributes in fMRI data. Our framework operates in two stages: (1) a bidirectional Mamba block with a temporal-first scanning mechanism to capture global temporal interactions in a computationally efficient manner; and (2) a transformer block leveraging self-attention to model global spatial relationships across the deep features processed by the Mamba block. Extensive experiments on two large-scale public datasets, UKBioBank and the Human Connectome Project, demonstrate that BrainMT achieves state-of-the-art performance on both classification (sex prediction) and regression (cognitive intelligence prediction) tasks, outperforming existing methods by a significant margin. Our code and implementation details will be made publicly available at this this https URL
Abstract (translated)
最近在深度学习领域的进展使得直接从功能性磁共振成像（fMRI）脑体积数据中预测表型测量成为可能，这一成就引起了神经影像学界的极大兴趣。然而，现有的方法主要基于卷积神经网络或变压器架构，常常难以建模fMRI数据中存在的复杂关系，因为这些模型无法捕捉到长距离的空间和时间依赖性。为了解决这些问题，我们引入了BrainMT，这是一种新颖的混合框架，旨在高效地学习并整合fMRI数据中的长程时空属性。 我们的框架分为两个阶段运作： 1. 一个双向Mamba块，结合了一个以时间为先扫描机制来有效地捕捉全局时间交互。 2. 一个变压器模块利用自注意力机制建模Mamba块处理的深层特征之间的全球空间关系。 在两个大规模公开数据集（英国生物银行和人类连接组项目）上的广泛实验表明，BrainMT在分类任务（性别预测）和回归任务（认知智能预测）上均达到了最先进的性能水平，并且比现有方法有显著的优势。我们的代码和实现细节将在[此处提供链接]公开发布。
URL
https://arxiv.org/abs/2506.22591
