Abstract
Distance-based unsupervised text classification is a method within text classification that leverages the semantic similarity between a label and a text to determine label relevance. This method provides numerous benefits, including fast inference and adaptability to expanding label sets, as opposed to zero-shot, few-shot, and fine-tuned neural networks that require re-training in such cases. In multi-label distance-based classification and information retrieval algorithms, thresholds are required to determine whether a text instance is "similar" to a label or query. Similarity between a text and label is determined in a dense embedding space, usually generated by state-of-the-art sentence encoders. Multi-label classification complicates matters, as a text instance can have multiple true labels, unlike in multi-class or binary classification, where each instance is assigned only one label. We expand upon previous literature on this underexplored topic by thoroughly examining and evaluating the ability of sentence encoders to perform distance-based classification. First, we perform an exploratory study to verify whether the semantic relationships between texts and labels vary across models, datasets, and label sets by conducting experiments on a diverse collection of realistic multi-label text classification (MLTC) datasets. We find that similarity distributions show statistically significant differences across models, datasets and even label sets. We propose a novel method for optimizing label-specific thresholds using a validation set. Our label-specific thresholding method achieves an average improvement of 46% over normalized 0.5 thresholding and outperforms uniform thresholding approaches from previous work by an average of 14%. Additionally, the method demonstrates strong performance even with limited labeled examples.
Abstract (translated)
基于距离的无监督文本分类是文本分类领域的一种方法，该方法利用标签与文本之间的语义相似性来判断标签的相关性。相比于零样本、小样本和微调神经网络需要重新训练的情况，这种方法提供了许多优势，包括快速推理以及对不断扩大的标签集具有良好的适应能力。 在多标签距离基于的分类及信息检索算法中，为了确定某个文本实例是否与一个标签或查询“相似”，都需要设定阈值。文本和标签之间的相似性是在由先进句子编码器生成的稠密嵌入空间内进行判定的。与多类或者二元分类不同，在多标签分类中，一个文本实例可能有多个真实标签。 我们在之前较少探索的主题上进行了拓展研究，详细地考察并评估了句子编码器在基于距离的分类中的性能。首先，我们通过一系列实验来验证模型、数据集和标签集合之间的语义关系是否存在差异。这些实验是在一组多样化的现实多标签文本分类（MLTC）数据集上进行的。我们发现相似性的分布统计上显著地不同，这表明了在不同的模型和数据集中存在着变化。 基于这一观察，我们提出了一个新的方法来利用验证集优化特定于标签的阈值。通过这种方法，相较于标准化后的0.5阈值法，我们的特定标签阈值设定平均提升了46%，并且相比于以往研究中的统一阈值设定策略，该方法也表现出平均14%的优势。此外，此方法即使在标注样本有限的情况下也能展现出强大的性能。 这种方法不仅提供了改进的分类准确性，还为复杂的多标签场景提供了解决方案，并且证明了针对不同数据集和标签优化特定阈值的重要性。
URL
https://arxiv.org/abs/2510.11160
