Abstract
This paper studies a text classification algorithm based on an improved Transformer to improve the performance and efficiency of the model in text classification tasks. Aiming at the shortcomings of the traditional Transformer model in capturing deep semantic relationships and optimizing computational complexity, this paper introduces a multi-level attention mechanism and a contrastive learning strategy. The multi-level attention mechanism effectively models the global semantics and local features in the text by combining global attention with local attention; the contrastive learning strategy enhances the model's ability to distinguish between different categories by constructing positive and negative sample pairs while improving the classification effect. In addition, in order to improve the training and inference efficiency of the model on large-scale text data, this paper designs a lightweight module to optimize the feature transformation process and reduce the computational cost. Experimental results on the dataset show that the improved Transformer model outperforms the comparative models such as BiLSTM, CNN, standard Transformer, and BERT in terms of classification accuracy, F1 score, and recall rate, showing stronger semantic representation ability and generalization performance. The method proposed in this paper provides a new idea for algorithm optimization in the field of text classification and has good application potential and practical value. Future work will focus on studying the performance of this model in multi-category imbalanced datasets and cross-domain tasks and explore the integration wi
Abstract (translated)
这篇论文研究了一种基于改进Transformer的文本分类算法，以提高模型在文本分类任务中的性能和效率。针对传统Transformer模型在捕捉深层次语义关系及优化计算复杂度方面的不足，本文引入了多层次注意力机制以及对比学习策略。多层次注意力机制通过结合全局注意力与局部注意力有效建模文本的全局语义和局部特征；对比学习策略则通过构造正负样本对来增强模型区分不同类别的能力，并提升分类效果的同时改进模型性能。 此外，为了提高大规模文本数据上模型训练及推理效率，本文设计了一个轻量级模块以优化特征转换过程并降低计算成本。实验结果表明，在测试集上的分类准确率、F1值和召回率等方面，改进后的Transformer模型优于BiLSTM、CNN、标准Transformer以及BERT等对比模型，显示出更强的语义表示能力和泛化性能。 该文中提出的这种方法为文本分类领域的算法优化提供了新的思路，并具有良好的应用潜力与实际价值。未来的工作将集中于研究此模型在多类别不平衡数据集及跨域任务中的表现，并探索与其他技术的集成应用可能性。
URL
https://arxiv.org/abs/2501.13467
