Abstract
Musculoskeletal disorders pose significant risks to athletes, and assessing risk early is important for prevention. However, most existing methods are designed for controlled settings and fail to reliably assess risk in complex environments due to their reliance on a single type of data. This research proposes ViSK-GAT (Visual-Skeletal Geometric Attention Transformer), a novel multimodal deep learning framework designed to classify musculoskeletal risk using visual and skeletal coordinate-based features. In addition, a custom multimodal dataset is constructed by combining visual data and skeletal coordinates for risk assessment. Each sample is labeled into eight risk categories based on the Rapid Entire Body Assessment system. ViSK-GAT combines a Residual Block with a Lightweight Transformer Block to learn spatial and temporal dependencies jointly. It incorporates two novel modules: the Fine-Grained Attention Module (FGAM), which enables precise inter-modal feature refinement through cross-attention between visual and skeletal inputs, and the Multimodal Geometric Correspondence Module (MGCM), which enhances cross-modal coherence by aligning image features with coordinate-based representations. ViSK-GAT achieved strong performance with validation and test accuracies of 93.55\% and 93.89\%, respectively; a precision of 93.86\%; an F1 score of 93.85\%; and Cohen's Kappa and Matthews Correlation Coefficient of 93\%. The regression results also indicated a low Root Mean Square Error of the predicted probability distribution of 0.1205 and a corresponding Mean Absolute Error of 0.0156. Compared to nine popular transfer learning backbones, ViSK-GAT consistently outperformed previous methods. The ViSK-GAT model advances artificial intelligence implementation and application, transforming musculoskeletal risk classification and enabling impactful early interventions in sports.
Abstract (translated)
肌肉骨骼疾病对运动员构成了重大风险，及早评估这些风险对于预防至关重要。然而，大多数现有的方法设计用于受控环境，并且由于依赖单一类型的数据而无法可靠地在复杂环境中进行风险评估。这项研究提出了一种新的多模态深度学习框架——ViSK-GAT（视觉-骨骼几何注意力变换器），该框架旨在使用视觉和基于坐标的骨骼特征来分类肌肉骨骼风险。此外，还构建了一个自定义的多模态数据集，通过结合视觉数据和骨骼坐标来进行风险评估。每个样本根据快速全身评估系统被标记为八个风险类别。 ViSK-GAT 结合了残差块（Residual Block）与轻量级变换器块（Lightweight Transformer Block），以共同学习空间和时间依赖关系。它还包含了两个新颖的模块：细粒度注意力模块（FGAM），该模块通过视觉和骨骼输入之间的交叉注意来实现精确的跨模态特征细化；以及多模态几何对应模块（MGCM），该模块通过将图像特征与基于坐标的表示对齐，以增强跨模式的一致性。 ViSK-GAT 在验证集和测试集上分别实现了 93.55% 和 93.89% 的准确性；精度为 93.86%；F1 分数为 93.85%；并且 Cohen's Kappa 值和 Matthews 相关系数分别为 93%。回归结果也表明，预测概率分布的均方根误差（Root Mean Square Error）仅为 0.1205，相应的平均绝对误差（Mean Absolute Error）为 0.0156。 与九种流行的迁移学习骨干网络相比，ViSK-GAT 在各个指标上始终优于以前的方法。这一模型的进步推动了人工智能在肌肉骨骼风险分类中的实施和应用，并且通过早期干预使运动领域的成果具有深远的影响。
URL
https://arxiv.org/abs/2509.05913
