Abstract
Many text classification methods usually introduce external information (e.g., label descriptions and knowledge bases) to improve the classification performance. Compared to external information, some internal information generated by the model itself during training, like text embeddings and predicted label probability distributions, are exploited poorly when predicting the outcomes of some texts. In this paper, we focus on leveraging this internal information, proposing a dual $k$ nearest neighbor (D$k$NN) framework with two $k$NN modules, to retrieve several neighbors from the training set and augment the distribution of labels. For the $k$NN module, it is easily confused and may cause incorrect predictions when retrieving some nearest neighbors from noisy datasets (datasets with labeling errors) or similar datasets (datasets with similar labels). To address this issue, we also introduce a label distribution learning module that can learn label similarity, and generate a better label distribution to help models distinguish texts more effectively. This module eases model overfitting and improves final classification performance, hence enhancing the quality of the retrieved neighbors by $k$NN modules during inference. Extensive experiments on the benchmark datasets verify the effectiveness of our method.
Abstract (translated)
许多文本分类方法通常会引入外部信息（例如标签描述和知识库）来提高分类性能。相比之下，模型在训练过程中生成的内部信息（如文本嵌入和预测的概率分布），在预测某些文本的结果时被利用得较少。在这篇论文中，我们专注于利用这种内部信息，提出了一个双重$k$最近邻（D$k$NN）框架，该框架包含两个$k$近邻模块，用于从训练集中检索多个邻居并增强标签的分布。 对于$k$最近邻模块，在从有噪声的数据集（带有标注错误的数据集）或相似数据集（具有类似标签的数据集）中检索一些最近邻时，它容易混淆并且可能导致不正确的预测。为了解决这个问题，我们还引入了一个标签分布学习模块，该模块可以学习标签之间的相似性，并生成一个更好的标签分布以帮助模型更有效地区分文本。这个模块有助于缓解模型过拟合问题并提高最终的分类性能，从而在推理过程中通过$k$近邻模块增强邻居检索的质量。 我们在基准数据集上进行了广泛的实验验证了我们方法的有效性。
URL
https://arxiv.org/abs/2503.04869
