Abstract
We extend and study a semi-supervised model for text classification proposed earlier by Hatefi et al. for classification tasks in which document classes are described by a small number of gold-labeled examples, while the majority of training examples is unlabeled. The model leverages the teacher-student architecture of Meta Pseudo Labels in which a ''teacher'' generates labels for originally unlabeled training data to train the ''student'' and updates its own model iteratively based on the performance of the student on the gold-labeled portion of the data. We extend the original model of Hatefi et al. by an unsupervised pre-training phase based on objective masking, and conduct in-depth performance evaluations of the original model, our extension, and various independent baselines. Experiments are performed using three different datasets in two different languages (English and Swedish).
Abstract (translated)
我们扩展并研究了Hatefi等人之前提出的用于文本分类的半监督模型，该模型适用于文档类别由少量带标签的示例描述、而大多数训练样本未标记的情况。此模型利用Meta Pseudo Labels（伪标签）中的教师-学生架构，在该架构中，“老师”为原本没有标签的训练数据生成标签以训练“学生”，并根据“学生”在带有黄金标签的数据部分上的表现迭代更新其自身的模型。我们通过基于目标掩码的无监督预训练阶段扩展了Hatefi等人的原始模型，并对原始模型、我们的改进版本以及各种独立基线进行了深入的性能评估。实验使用了三种不同数据集（英语和瑞典语两种语言）进行。
URL
https://arxiv.org/abs/2505.06624
