Abstract
Contrastive learning has gained significant attention in short text clustering, yet it has an inherent drawback of mistakenly identifying samples from the same category as negatives and then separating them in the feature space (false negative separation), which hinders the generation of superior representations. To generate more discriminative representations for efficient clustering, we propose a novel short text clustering method, called Discriminative Representation learning via \textbf{A}ttention-\textbf{E}nhanced \textbf{C}ontrastive \textbf{L}earning for Short Text Clustering (\textbf{AECL}). The \textbf{AECL} consists of two modules which are the pseudo-label generation module and the contrastive learning module. Both modules build a sample-level attention mechanism to capture similarity relationships between samples and aggregate cross-sample features to generate consistent representations. Then, the former module uses the more discriminative consistent representation to produce reliable supervision information for assist clustering, while the latter module explores similarity relationships and consistent representations optimize the construction of positive samples to perform similarity-guided contrastive learning, effectively addressing the false negative separation issue. Experimental results demonstrate that the proposed \textbf{AECL} outperforms state-of-the-art methods. If the paper is accepted, we will open-source the code.
Abstract (translated)
对比学习在短文本聚类中获得了广泛关注,但它有一个内在的缺点:可能会错误地将同一类别中的样本识别为负例,并将其在特征空间中分离(假阴性分离),这阻碍了生成优质表示的能力。为了生成更具有区分性的表示以实现高效的聚类,我们提出了一种新的短文本聚类方法,称为通过注意力增强对比学习的判别式表示学习(Attention-Enhanced Contrastive Learning for Short Text Clustering, AECL)。AECL包含两个模块:伪标签生成模块和对比学习模块。这两个模块都构建了样本级别的注意机制来捕捉样本之间的相似关系,并聚合跨样本特征以生成一致的表示。前者使用更具区分性的稳定表示生成可靠的信息监督,辅助聚类;后者探索相似关系,并通过优化正例构造执行指导相似度的学习,有效解决了假阴性分离问题。实验结果表明,所提出的AECL优于最先进的方法。如果论文被接受,我们将开源代码。
URL
https://arxiv.org/abs/2501.03584