Abstract
This article presents an evaluation of several machine learning methods applied to automated text classification, alongside the design of a demonstrative system for unbalanced document categorization and distribution. The study focuses on balancing classification accuracy with computational efficiency, a key consideration when integrating AI into real world automation pipelines. Three models of varying complexity were examined: a Naive Bayes classifier, a bidirectional LSTM network, and a fine tuned transformer based BERT model. The experiments reveal substantial differences in performance. BERT achieved the highest accuracy, consistently exceeding 99\%, but required significantly longer training times and greater computational resources. The BiLSTM model provided a strong compromise, reaching approximately 98.56\% accuracy while maintaining moderate training costs and offering robust contextual understanding. Naive Bayes proved to be the fastest to train, on the order of milliseconds, yet delivered the lowest accuracy, averaging around 94.5\%. Class imbalance influenced all methods, particularly in the recognition of minority categories. A fully functional demonstrative system was implemented to validate practical applicability, enabling automated routing of technical requests with throughput unattainable through manual processing. The study concludes that BiLSTM offers the most balanced solution for the examined scenario, while also outlining opportunities for future improvements and further exploration of transformer architectures.
Abstract (translated)
本文评估了几种机器学习方法在自动文本分类中的应用，并设计了一个用于不平衡文档分类和分布的演示系统。研究重点在于平衡分类准确性和计算效率，这是将AI集成到现实世界自动化流水线中时的关键考虑因素。研究人员考察了三种不同复杂度的模型：朴素贝叶斯分类器、双向LSTM网络以及经过微调的基于Transformer的BERT模型。实验结果显示性能存在显著差异。 BERT取得了最高的准确性，超过了99%，但需要更长的训练时间和更多的计算资源。BiLSTM模型则提供了一个强有力的折中方案，在保持适中的训练成本的同时达到了大约98.56%的准确率，并且具有较强的理解上下文的能力。相比之下，朴素贝叶斯分类器训练速度最快，仅需毫秒级时间，然而其准确性最低，平均在94.5%左右。 类别不平衡影响了所有方法，尤其在识别少数类别的表现上更加明显。为验证实际应用的可行性，实施了一个完全功能的演示系统，该系统能够自动处理技术请求，大大提高了处理效率并达到了手动处理无法企及的速度水平。研究结论认为BiLSTM模型是目前所考察场景中的最优方案，同时指出未来改进和进一步探索Transformer架构的机会。
URL
https://arxiv.org/abs/2602.20336
