Abstract
Human language processing relies on the brain's capacity for predictive inference. We present a machine learning framework for decoding neural (EEG) responses to dynamic visual language stimuli in Deaf signers. Using coherence between neural signals and optical flow-derived motion features, we construct spatiotemporal representations of predictive neural dynamics. Through entropy-based feature selection, we identify frequency-specific neural signatures that differentiate interpretable linguistic input from linguistically disrupted (time-reversed) stimuli. Our results reveal distributed left-hemispheric and frontal low-frequency coherence as key features in language comprehension, with experience-dependent neural signatures correlating with age. This work demonstrates a novel multimodal approach for probing experience-driven generative models of perception in the brain.
Abstract (translated)
人类语言处理依赖于大脑进行预测性推理的能力。我们提出了一种机器学习框架，用于解码聋人手语使用者对动态视觉语言刺激的神经（EEG）反应。通过将神经信号与基于光流的运动特征之间的相干性结合，我们构建了预测性神经动力学的空间时间表示。利用熵为基础的特征选择方法，我们识别出频率特异性的神经标志物，这些标志物能够区分可理解的语言输入和语言被打乱（时间倒置）的刺激。我们的研究结果揭示了左半球广泛分布以及额叶低频相干是语言理解的关键特征，并且随着经验积累而变化的神经标志物与年龄相关联。这项工作展示了探索大脑中由经验驱动的感知生成模型的新颖多模态方法。
URL
https://arxiv.org/abs/2512.20929
