Abstract
Understanding and decoding brain activity into visual representations is a fundamental challenge at the intersection of neuroscience and artificial intelligence. While EEG-based visual decoding has shown promise due to its non-invasive, low-cost nature and millisecond-level temporal resolution, existing methods are limited by their reliance on flat neural representations that overlook the brain's inherent visual hierarchy. In this paper, we introduce ViEEG, a biologically inspired hierarchical EEG decoding framework that aligns with the Hubel-Wiesel theory of visual processing. ViEEG decomposes each visual stimulus into three biologically aligned components-contour, foreground object, and contextual scene-serving as anchors for a three-stream EEG encoder. These EEG features are progressively integrated via cross-attention routing, simulating cortical information flow from V1 to IT to the association cortex. We further adopt hierarchical contrastive learning to align EEG representations with CLIP embeddings, enabling zero-shot object recognition. Extensive experiments on the THINGS-EEG dataset demonstrate that ViEEG achieves state-of-the-art performance, with 40.9% Top-1 accuracy in subject-dependent and 22.9% Top-1 accuracy in cross-subject settings, surpassing existing methods by over 45%. Our framework not only advances the performance frontier but also sets a new paradigm for biologically grounded brain decoding in AI.
Abstract (translated)
理解并解码脑活动以生成视觉表示是神经科学与人工智能交叉领域中的一个基本挑战。尽管基于EEG的视觉解码由于其非侵入性、低成本以及毫秒级的时间分辨率而展现出巨大潜力，现有的方法却受到依赖于忽视大脑固有视觉层次结构的扁平化神经表示的限制。在本文中，我们提出了ViEEG——一种受生物启发的分层EEG解码框架，该框架与Hubel-Wiesel关于视觉处理理论相一致。ViEEG将每个视觉刺激分解为三个与生物学相符的部分：轮廓、前景对象和背景场景，作为三流EEG编码器的基础。这些EEG特征通过交叉注意力路由进行逐步融合，模拟从初级视皮层（V1）到颞叶内侧区域（IT）再到关联皮层的信息流动过程。我们进一步采用了层次对比学习方法来使EEG表示与CLIP嵌入对齐，从而实现零样本对象识别功能。在THINGS-EEG数据集上的大量实验表明，ViEEG取得了最先进的性能，在依赖于单个受试者的情况下实现了40.9%的Top-1准确率，并且在跨受试者设置中达到了22.9%的Top-1准确率，这比现有方法高出超过45%。我们的框架不仅提升了性能边界，还为人工智能领域中的生物基础脑解码设定了新的范式。
URL
https://arxiv.org/abs/2505.12408
