Abstract
This paper presents system description for Arabic medical text classification across 82 distinct categories. Our primary architecture utilizes a fine-tuned AraBERTv2 encoder enhanced with a hybrid pooling strategies, combining attention and mean representations, and multi-sample dropout for robust regularization. We systematically benchmark this approach against a suite of multilingual and Arabic-specific encoders, as well as several large-scale causal decoders, including zero-shot re-ranking via Llama 3.3 70B and feature extraction from Qwen 3B hidden states. Our findings demonstrate that specialized bidirectional encoders significantly outperform causal decoders in capturing the precise semantic boundaries required for fine-grained medical text classification. We show that causal decoders, optimized for next-token prediction, produce sequence-biased embeddings that are less effective for categorization compared to the global context captured by bidirectional attention. Despite significant class imbalance and label noise identified within the training data, our results highlight the superior semantic compression of fine-tuned encoders for specialized Arabic NLP tasks. Final performance metrics on the test set, including Accuracy and Macro-F1, are reported and discussed.
Abstract (translated)
本文介绍了针对阿拉伯语医学文本分类的系统描述，涵盖了82个不同的类别。我们的主要架构使用了一种改进后的AraBERTv2编码器，该编码器融合了混合池化策略（结合注意力和平均表示）以及多样本 dropout 技术来增强鲁棒性正则化。我们通过一系列多语言和阿拉伯语特定的编码器，以及一些大规模因果解码器进行了系统的基准测试，包括零样本重排序方法Llama 3.3 70B及从Qwen 3B隐藏状态中提取特征的方法。我们的研究发现表明，在捕捉精细医学文本分类所需的精确语义边界方面，专业双向编码器显著优于因果解码器。 我们展示了优化于下一个标记预测的因果解码器会产生序列偏向性嵌入，这些嵌入在类别化任务中的效果不如双向注意力捕获的全局上下文。尽管训练数据中存在明显的类别不平衡和标签噪声问题，我们的结果强调了针对专业阿拉伯语自然语言处理（NLP）任务进行微调后的编码器所具有的卓越语义压缩能力。 最终，在测试集上报告并讨论了性能指标，包括准确率和宏F1值。
URL
https://arxiv.org/abs/2603.10008
