Abstract
Aphasia is an acquired language disorder caused by injury to the regions of the brain that are responsible for language. Aphasia may impair the use and comprehension of written and spoken language. The Western Aphasia Battery-Revised (WAB-R) is an assessment tool administered by speech-language pathologists (SLPs) to evaluate the aphasia type and severity. Because the WAB-R measures isolated linguistic skills, there has been growing interest in the assessment of discourse production as a more holistic representation of everyday language abilities. Recent advancements in speech analysis focus on automated estimation of aphasia severity from spontaneous speech, relying mostly in isolated linguistic or acoustical features. In this work, we propose a graph neural network-based framework for estimating aphasia severity. We represented each participant's discourse as a directed multi-modal graph, where nodes represent lexical items and gestures and edges encode word-word, gesture-word, and word-gesture transitions. GraphSAGE is employed to learn participant-level embeddings, thus integrating information from immediate neighbors and overall graph structure. Our results suggest that aphasia severity is not encoded in isolated lexical distribution, but rather emerges from structured interactions between speech and gesture. The proposed architecture offers a reliable automated aphasia assessment, with possible uses in bedside screening and telehealth-based monitoring.
Abstract (translated)
失语症是一种由于大脑语言区域受损而获得的语言障碍。它可能会影响口语和书面语言的使用与理解。西方失语症评估量表修订版（WAB-R）是由言语-语言治疗师（SLPs）用来评估失语类型及严重程度的一种评估工具。因为WAB-R测量的是孤立的语言技能，所以越来越多的人开始对话语产出的评估感兴趣，以作为日常语言能力更全面的表现形式。最近在语音分析方面的进展主要集中在从自然语言中自动估计失语症的严重程度上，这主要依赖于孤立的语言或声学特征。在这项工作中，我们提出了一种基于图神经网络框架来估算失语症严重程度的方法。我们将每个参与者的对话表示为一个有向多模态图，在这种图中节点代表词汇和手势，边则编码词-词、手势-词以及词-手势的转换关系。使用GraphSAGE来学习参与者级别的嵌入，从而整合了来自直接邻居的信息及整个图形结构的信息。我们的结果显示，失语症严重程度并非体现在孤立的词汇分布上，而是从言语与手势之间的有机构成互动中浮现出来。所提出的架构提供了一种可靠的自动化失语评估方法，可能在床边筛查和基于远程医疗监测方面有着潜在的应用价值。
URL
https://arxiv.org/abs/2602.20163
