Abstract
Deep learning with noisy labels is an interesting challenge in weakly supervised learning. Despite their significant learning capacity, CNNs have a tendency to overfit in the presence of samples with noisy labels. Alleviating this issue, the well known Co-Training framework is used as a fundamental basis for our work. In this paper, we introduce a Contrastive Co-Transformer framework, which is simple and fast, yet able to improve the performance by a large margin compared to the state-of-the-art approaches. We argue the robustness of transformers when dealing with label noise. Our Contrastive Co-Transformer approach is able to utilize all samples in the dataset, irrespective of whether they are clean or noisy. Transformers are trained by a combination of contrastive loss and classification loss. Extensive experimental results on corrupted data from six standard benchmark datasets including Clothing1M, demonstrate that our Contrastive Co-Transformer is superior to existing state-of-the-art methods.
Abstract (translated)
带有噪声标签的深度学习是弱监督学习中的一个有趣挑战。尽管卷积神经网络（CNNs）具有强大的学习能力，但在存在带噪标签样本的情况下，它们倾向于过拟合。为缓解这一问题，本工作以著名的Co-Training框架作为基础。在本文中，我们引入了一种对比共变换器（Contrastive Co-Transformer）框架，该框架简单且快速，能够大幅优于现有方法的表现。我们认为，当处理标签噪声时，变压器具有鲁棒性。我们的对比共变换器方法可以利用数据集中所有的样本，无论它们是干净的还是带噪的。通过结合对比损失和分类损失，对变压器进行训练。在六个标准基准数据集（包括Clothing1M）上的大量实验结果表明，我们提出的对比共变换器优于现有的最先进方法。
URL
https://arxiv.org/abs/2503.03042
