Abstract
Large Language Models revolutionized NLP and showed dramatic performance improvements across several tasks. In this paper, we investigated the role of such language models in text classification and how they compare with other approaches relying on smaller pre-trained language models. Considering 32 datasets spanning 8 languages, we compared zero-shot classification, few-shot fine-tuning and synthetic data based classifiers with classifiers built using the complete human labeled dataset. Our results show that zero-shot approaches do well for sentiment classification, but are outperformed by other approaches for the rest of the tasks, and synthetic data sourced from multiple LLMs can build better classifiers than zero-shot open LLMs. We also see wide performance disparities across languages in all the classification scenarios. We expect that these findings would guide practitioners working on developing text classification systems across languages.
Abstract (translated)
大型语言模型在自然语言处理（NLP）领域引发了革命，展现了多项任务上的显著性能提升。在这篇论文中，我们研究了此类语言模型在文本分类中的作用，并将其与依赖于较小预训练语言模型的其他方法进行了比较。考虑到涵盖8种语言的32个数据集，我们将零样本分类、少量样本微调和基于合成数据的分类器与使用完整人工标注数据集构建的分类器进行对比。我们的结果显示，对于情感分类，零样本方法表现良好；但对于其余任务，则被其他方法超越。此外，来自多个大型语言模型（LLM）的合成数据可以建立比零样本开放语言模型更好的分类器。在所有文本分类场景中，我们还观察到了跨不同语言间的广泛性能差异。我们认为这些发现将有助于指导在多语言环境下开发文本分类系统的研究人员和实践者。
URL
https://arxiv.org/abs/2502.11830
