Abstract
Despite rapid advances in large language models (LLMs), their integration with traditional supervised machine learning (ML) techniques that have proven applicability to medical data remains underexplored. This is particularly true for psychiatric applications, where narrative data often exhibit nuanced linguistic and contextual complexity, and can benefit from the combination of multiple models with differing characteristics. In this study, we introduce an ensemble framework for automatically classifying Attention-Deficit/Hyperactivity Disorder (ADHD) diagnosis (binary) using narrative transcripts. Our approach integrates three complementary models: LLaMA3, an open-source LLM that captures long-range semantic structure; RoBERTa, a pre-trained transformer model fine-tuned on labeled clinical narratives; and a Support Vector Machine (SVM) classifier trained using TF-IDF-based lexical features. These models are aggregated through a majority voting mechanism to enhance predictive robustness. The dataset includes 441 instances, including 352 for training and 89 for validation. Empirical results show that the ensemble outperforms individual models, achieving an F$_1$ score of 0.71 (95\% CI: [0.60-0.80]). Compared to the best-performing individual model (SVM), the ensemble improved recall while maintaining competitive precision. This indicates the strong sensitivity of the ensemble in identifying ADHD-related linguistic cues. These findings demonstrate the promise of hybrid architectures that leverage the semantic richness of LLMs alongside the interpretability and pattern recognition capabilities of traditional supervised ML, offering a new direction for robust and generalizable psychiatric text classification.
Abstract (translated)
尽管在大型语言模型（LLMs）方面取得了快速进展，但它们与传统监督式机器学习（ML）技术的结合仍未被充分探索。这些传统方法已在医疗数据的应用中证明了其有效性，尤其是在精神科应用领域，其中叙述性数据常常表现出复杂的语义和上下文特征，并且可以从多个具有不同特性的模型组合中受益。在这项研究中，我们介绍了一种用于自动分类注意力缺陷多动障碍（ADHD）诊断（二元分类）的叙事转录文本的集成框架。我们的方法整合了三种互补性模型：LLaMA3，一个开源的大规模语言模型，能够捕捉长程语义结构；RoBERTa，一种在标记临床叙述上进行了微调的预训练变换器模型；以及支持向量机（SVM）分类器，使用基于TF-IDF的词汇特征进行训练。这些模型通过多数投票机制整合起来以增强预测稳健性。该数据集包括441个实例，其中352个用于训练，89个用于验证。实证结果表明，集成方法优于单一模型的表现，达到了0.71的F1分数（95%置信区间：[0.60-0.80]）。与表现最佳的单个模型（SVM）相比，集成方法提高了召回率同时保持了竞争力的精度。这说明该集成系统在识别ADHD相关语言线索方面具有较强的敏感性。这些发现展示了混合架构的潜力，即利用LLMs的语义丰富性和传统监督式机器学习的可解释性和模式识别能力，在稳健和泛化的精神科文本分类上开辟了一条新的方向。
URL
https://arxiv.org/abs/2505.21324
