Abstract
Training deep learning networks with minimal supervision has gained significant research attention due to its potential to reduce reliance on extensive labelled data. While self-training methods have proven effective in semi-supervised learning, they remain vulnerable to errors from noisy pseudo labels. Moreover, most recent approaches to the few-label classification problem are either designed for resource-rich languages such as English or involve complex cascading models that are prone to overfitting. To address the persistent challenge of few-label text classification in truly low-resource linguistic contexts, where existing methods often struggle with noisy pseudo-labels and domain adaptation, we propose Flick. Unlike prior methods that rely on generic multi-cluster pseudo-labelling or complex cascading architectures, Flick leverages the fundamental insight that distilling high-confidence pseudo-labels from a broader set of initial clusters can dramatically improve pseudo-label quality, particularly for linguistically diverse, low-resource settings. Flick introduces a novel pseudo-label refinement component, a departure from traditional pseudo-labelling strategies by identifying and leveraging top-performing pseudo-label clusters. This component specifically learns to distil highly reliable pseudo-labels from an initial broad set by focusing on single-cluster cohesion and leveraging an adaptive top-k selection mechanism. This targeted refinement process is crucial for mitigating the propagation of errors inherent in low-resource data, allowing for robust fine-tuning of pre-trained language models with only a handful of true labels. We demonstrate Flick's efficacy across 14 diverse datasets, encompassing challenging low-resource languages such as Arabic, Urdu, and Setswana, alongside English, showcasing its superior performance and adaptability.
Abstract (translated)
使用最少监督训练深度学习网络因其能够减少对大量标注数据的依赖而受到了广泛关注。虽然自训练方法在半监督学习中证明了其有效性，但它们仍然容易受到由噪声伪标签引发错误的影响。此外，大多数近期解决少量标签分类问题的方法要么是为资源丰富的语言（如英语）设计的，要么涉及可能导致过拟合的复杂级联模型。为了应对真正低资源语言环境中的少量文本分类挑战，在这种环境中现有方法往往难以处理噪声伪标签和领域适应的问题，我们提出了Flick。 不同于以往依赖于通用多聚类伪标签或复杂级联架构的方法，Flick 利用了这样一个基本见解：从一个更广泛的初始聚类集中提炼出高可信度的伪标签可以显著提高伪标签的质量，特别是在语言多样且资源有限的情况下。Flick 引入了一种新颖的伪标签精炼组件，这与传统的伪标签策略有所不同，通过识别和利用表现最佳的伪标签聚类来实现改进。该组件专门学习从一个初始宽泛集中的单个聚类中提炼出高度可靠的伪标签，并通过采用自适应Top-K选择机制来提高单一聚类内的一致性。这种针对性的精炼过程对于缓解低资源数据中错误传播的问题至关重要，使得仅使用少量真实标签即可实现预训练语言模型的稳健微调成为可能。 我们展示了Flick 在涵盖包括阿拉伯语、乌尔都语和塞茨瓦纳语等具有挑战性的低资源语言以及英语在内的14个多样化的数据集上的有效性，证明了其卓越性能及适应性。
URL
https://arxiv.org/abs/2506.10292
