Abstract
Features of the same sample generated by different pretrained models often exhibit inherently distinct feature distributions because of discrepancies in the model pretraining objectives or architectures. Learning invariant representations from large-scale unlabeled visual data with various pretrained models in a fully unsupervised transfer manner remains a significant challenge. In this paper, we propose a multiview self-representation learning (MSRL) method in which invariant representations are learned by exploiting the self-representation property of features across heterogeneous views. The features are derived from large-scale unlabeled visual data through transfer learning with various pretrained models and are referred to as heterogeneous multiview data. An individual linear model is stacked on top of its corresponding frozen pretrained backbone. We introduce an information-passing mechanism that relies on self-representation learning to support feature aggregation over the outputs of the linear model. Moreover, an assignment probability distribution consistency scheme is presented to guide multiview self-representation learning by exploiting complementary information across different views. Consequently, representation invariance across different linear models is enforced through this scheme. In addition, we provide a theoretical analysis of the information-passing mechanism, the assignment probability distribution consistency and the incremental views. Extensive experiments with multiple benchmark visual datasets demonstrate that the proposed MSRL method consistently outperforms several state-of-the-art approaches.
Abstract (translated)
不同预训练模型生成的同一样本特征通常会表现出固有的分布差异，这是由于这些模型在预训练目标或架构上的不同。从大规模未标记视觉数据中使用各种预训练模型通过完全无监督的方式学习不变表示仍然是一项重大挑战。在这篇论文中，我们提出了一种多视图自表征学习（MSRL）方法，在这种方法中，通过利用异构视角下特征的自表征特性来学习不变表示。这些特征是从大规模未标记视觉数据中通过对不同预训练模型进行迁移学习得到，并被称作异构多视角数据。在其对应的冻结的预训练骨干网络之上堆叠一个独立的线性模型。我们引入了一种基于自表征学习的信息传递机制，通过该机制支持在线性模型输出上的特征聚合。此外，还提出了一种分配概率分布一致性方案，利用不同视图间的互补信息来指导多视角自表征学习，并且通过这种方案强制执行不同线性模型之间的表示不变性。另外，我们提供了关于信息传递机制、分配概率分布一致性和增量视图的理论分析。 在多个基准视觉数据集上的广泛实验表明，所提出的MSRL方法始终优于几种最先进的方法。
URL
https://arxiv.org/abs/2602.04328
