Abstract
This study explores the optimization of the DRAGON Longformer base model for clinical text classification, specifically targeting the binary classification of medical case descriptions. A dataset of 500 clinical cases containing structured medical observations was used, with 400 cases for training and 100 for validation. Enhancements to the pre-trained joeranbosma/dragon-longformer-base-mixed-domain model included hyperparameter tuning, domain-specific preprocessing, and architectural adjustments. Key modifications involved increasing sequence length from 512 to 1024 tokens, adjusting learning rates from 1e-05 to 5e-06, extending training epochs from 5 to 8, and incorporating specialized medical terminology. The optimized model achieved notable performance gains: accuracy improved from 72.0% to 85.2%, precision from 68.0% to 84.1%, recall from 75.0% to 86.3%, and F1-score from 71.0% to 85.2%. Statistical analysis confirmed the significance of these improvements (p < .001). The model demonstrated enhanced capability in interpreting medical terminology, anatomical measurements, and clinical observations. These findings contribute to domain-specific language model research and offer practical implications for clinical natural language processing applications. The optimized model's strong performance across diverse medical conditions underscores its potential for broad use in healthcare settings.
Abstract (translated)
这项研究探讨了DRAGON Longformer基础模型在临床文本分类中的优化，特别是针对医疗案例描述的二元分类。使用了一个包含500个临床案例的数据集，其中包括结构化的医学观察数据，其中400个案例用于训练，100个案例用于验证。对预训练的joeranbosma/dragon-longformer-base-mixed-domain模型进行了改进，包括超参数调整、领域特定的预处理和架构修改。关键的改动涉及将序列长度从512增加到1024个标记，学习率从1e-05调整到5e-06，训练周期从5延长至8，并且加入了专门的医学术语。 优化后的模型取得了显著的性能提升：准确度从72.0%提高到了85.2%，精度从68.0%提高到了84.1%，召回率从75.0%提高到了86.3%，F1分数从71.0%提高到了85.2%。统计分析确认了这些改进的显著性（p < .001）。该模型在解释医学术语、解剖测量和临床观察方面表现出增强的能力。 这些发现为特定领域的语言模型研究做出了贡献，并对临床自然语言处理应用的实际操作具有重要意义。优化后的模型在各种医疗状况下的优异性能，强调了其在医疗保健环境中广泛使用的潜力。
URL
https://arxiv.org/abs/2507.09470
