Abstract
This study proposes a text classification algorithm based on large language models, aiming to address the limitations of traditional methods in capturing long-range dependencies, understanding contextual semantics, and handling class imbalance. The framework includes text encoding, contextual representation modeling, attention-based enhancement, feature aggregation, and classification prediction. In the representation stage, deep semantic embeddings are obtained through large-scale pretrained language models, and attention mechanisms are applied to enhance the selective representation of key features. In the aggregation stage, global and weighted strategies are combined to generate robust text-level vectors. In the classification stage, a fully connected layer and Softmax output are used to predict class distributions, and cross-entropy loss is employed to optimize model parameters. Comparative experiments introduce multiple baseline models, including recurrent neural networks, graph neural networks, and Transformers, and evaluate them on Precision, Recall, F1-Score, and AUC. Results show that the proposed method outperforms existing models on all metrics, with especially strong improvements in Recall and AUC. In addition, sensitivity experiments are conducted on hyperparameters and data conditions, covering the impact of hidden dimensions on AUC and the impact of class imbalance ratios on Recall. The findings demonstrate that proper model configuration has a significant effect on performance and reveal the adaptability and stability of the model under different conditions. Overall, the proposed text classification method not only achieves effective performance improvement but also verifies its robustness and applicability in complex data environments through systematic analysis.
Abstract (translated)
这项研究提出了一种基于大规模语言模型的文本分类算法，旨在解决传统方法在捕捉长距离依赖性、理解上下文语义以及处理类别不平衡方面的局限。该框架包括文本编码、上下文表示建模、注意力机制增强、特征聚合和分类预测阶段。 在表示阶段，通过大规模预训练的语言模型获得深层语义嵌入，并应用注意机制来增强对关键特征的选择性表示。在聚合阶段，结合全局和加权策略生成稳健的文本级向量。在分类阶段，使用全连接层和Softmax输出来预测类别分布，并采用交叉熵损失来优化模型参数。 比较实验引入了多个基准模型，包括递归神经网络、图神经网络和Transformer，并从精度（Precision）、召回率（Recall）、F1-Score以及AUC等指标评估它们的性能。结果显示，所提出的方法在所有度量标准上均优于现有模型，特别是在召回率和AUC方面有显著改进。 此外，还进行了对超参数和数据条件进行敏感性实验，涵盖了隐藏维度对AUC的影响及类别不平衡比例对召回率的影响。研究发现表明，合适的模型配置会对性能产生重要影响，并揭示了该模型在不同条件下具有适应性和稳定性。 总体而言，所提出的文本分类方法不仅实现了有效的性能提升，还通过系统分析验证了其在复杂数据环境中的鲁棒性和适用性。
URL
https://arxiv.org/abs/2512.09444
