Abstract
Detecting diseases from social media has diverse applications, such as public health monitoring and disease spread detection. While language models (LMs) have shown promising performance in this domain, there remains ongoing research aimed at refining their discriminating representations. In this paper, we propose a novel method that integrates Contrastive Learning (CL) with language modeling to address this challenge. Our approach introduces a self-augmentation method, wherein hidden representations of the model are augmented with their own representations. This method comprises two branches: the first branch, a traditional LM, learns features specific to the given data, while the second branch incorporates augmented representations from the first branch to encourage generalization. CL further refines these representations by pulling pairs of original and augmented versions closer while pushing other samples away. We evaluate our method on three NLP datasets encompassing binary, multi-label, and multi-class classification tasks involving social media posts related to various diseases. Our approach demonstrates notable improvements over traditional fine-tuning methods, achieving up to a 2.48% increase in F1-score compared to baseline approaches and a 2.1% enhancement over state-of-the-art methods.
Abstract (translated)
检测社交媒体上的疾病具有多种应用,例如公共卫生监测和疾病传播检测。虽然语言模型(LMs)在此领域显示出有前景的表现,但仍然有持续的研究旨在改进它们的区分性表示。在本文中,我们提出了一种将对比学习(CL)与语言建模相结合的新方法,以解决这个挑战。我们的方法引入了一种自增强方法,其中模型的隐藏表示被自身表示增强。这个方法包括两个分支:第一个分支是一个传统的LM,学习给定数据的特征,而第二个分支从第一个分支中引入增强表示,以鼓励泛化。CL通过将原始和增强版本之间的对对距离拉近,同时将其他样本推开,进一步优化这些表示。我们在三个涵盖二元、多标签和多分类任务的社会媒体帖子相关的NLP数据集上评估我们的方法。我们的方法在传统微调方法上取得了显著的改进,与基线方法相比,F1分数提高了2.48%,与最先进的 methods相比,F1分数提高了2.1%。
URL
https://arxiv.org/abs/2405.01597
