Abstract
Self-supervised learning (SSL) has emerged as a promising solution for addressing the challenge of limited labeled data in deep neural networks (DNNs), offering scalability potential. However, the impact of design dependencies within the SSL framework remains insufficiently investigated. In this study, we comprehensively explore SSL behavior across a spectrum of augmentations, revealing their crucial role in shaping SSL model performance and learning mechanisms. Leveraging these insights, we propose a novel learning approach that integrates prior knowledge, with the aim of curtailing the need for extensive data augmentations and thereby amplifying the efficacy of learned representations. Notably, our findings underscore that SSL models imbued with prior knowledge exhibit reduced texture bias, diminished reliance on shortcuts and augmentations, and improved robustness against both natural and adversarial corruptions. These findings not only illuminate a new direction in SSL research, but also pave the way for enhancing DNN performance while concurrently alleviating the imperative for intensive data augmentation, thereby enhancing scalability and real-world problem-solving capabilities.
Abstract (translated)
自监督学习（SSL）作为一种解决深度神经网络（DNN）中有限标注数据问题的有前途的解决方案，具有可扩展性潜力。然而，在SSL框架内设计依赖关系的影响仍然不够深入研究。在本文中，我们全面探讨了SSL在各种增强方法上的行为，揭示了它们在塑造SSL模型性能和学习机制中的关键作用。利用这些见解，我们提出了一个新学习方法，旨在结合先验知识，以抑制对广泛数据增强的需求，从而提高所学表示的效力。值得注意的是，我们的研究结果表明，预先训练的SSL模型具有降低纹理偏差、减小对短路和增强的依赖，以及对抗自然和对抗性失真增强的改善的特性。这些发现不仅阐明了SSL研究的新方向，而且为同时减轻数据增强的必要性，提高DNN性能，增强可扩展性和现实问题解决能力铺平道路。
URL
https://arxiv.org/abs/2404.09752
