Abstract
Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets.
Abstract (translated)
短文本分类作为自然语言处理领域的研究子课题，因其语义稀疏性和实际场景中标签样本不足而更具挑战性。在本工作中，我们提出了一种名为MI-DELIGHT的新模型，用于解决短文本分类问题。具体来说，该模型首先进行多源信息（即统计信息、语言学信息和事实信息）的探索以缓解语义稀疏性的问题。然后，采用图学习方法来学习用图形式表示的短文本的表征。此外，我们引入了一种双层次（实例级和聚类级）对比学习辅助任务，有效捕获大量未标记数据中的不同粒度的对比信息。同时，先前的模型仅在并行执行主要任务与辅助任务时，并不考虑各任务之间的关系。因此，我们引入了分层架构以明确建模各个任务间的相关性。我们在多个基准数据集上进行了广泛的实验，结果表明MI-DELIGHT显著超越了以前的竞争模型，在某些数据集中甚至超过了流行的大规模语言模型的性能。
URL
https://arxiv.org/abs/2501.09214
