Abstract
In the field of visual representation learning, performance of contrastive learning has been catching up with the supervised method which is commonly a classification convolutional neural network. However, most of the research work focuses on improving the accuracy of downstream tasks such as image classification and object detection. For visual contrastive learning, the influences of individual image features (e.g., color and shape) to model performance remain ambiguous. This paper investigates such influences by designing various ablation experiments, the results of which are evaluated by specifically designed metrics. While these metrics are not invented by us, we first use them in the field of representation evaluation. Specifically, we assess the contribution of two primary image features (i.e., color and shape) in a quantitative way. Experimental results show that compared with supervised representations, contrastive representations tend to cluster with objects of similar color in the representation space, and contain less shape information than supervised representations. Finally, we discuss that the current data augmentation is responsible for these results. We believe that exploring an unsupervised augmentation method that
Abstract (translated)
在视觉表示学习领域,对比学习的性能已经赶上监督方法,通常是一种分类卷积神经网络。然而,大多数研究重点都是改进后续任务的准确性,例如图像分类和物体检测。对于视觉对比学习,个体图像特征(例如颜色和形状)对模型性能的影响仍然不明确。本文通过设计各种烧灼实验来研究这些影响,实验结果由专门设计的指标进行评估。尽管这些指标不是我们发明的,我们首先将其应用于表示评估领域。具体来说,我们评估两个主要图像特征(即颜色和形状)的贡献,以定量方式。实验结果显示,与监督表示相比,对比表示在表示空间中倾向于与相似颜色的物体聚集在一起,并且包含比监督表示更少的形状信息。最后,我们讨论了当前数据增强方法如何导致这些结果。我们相信探索一种无监督增强方法将解决这些问题。
URL
https://arxiv.org/abs/2301.12459
