Abstract
Factor analysis, often regarded as a Bayesian variant of matrix factorization, offers superior capabilities in capturing uncertainty, modeling complex dependencies, and ensuring robustness. As the deep learning era arrives, factor analysis is receiving less and less attention due to their limited expressive ability. On the contrary, contrastive learning has emerged as a potent technique with demonstrated efficacy in unsupervised representational learning. While the two methods are different paradigms, recent theoretical analysis has revealed the mathematical equivalence between contrastive learning and matrix factorization, providing a potential possibility for factor analysis combined with contrastive learning. Motivated by the interconnectedness of contrastive learning, matrix factorization, and factor analysis, this paper introduces a novel Contrastive Factor Analysis framework, aiming to leverage factor analysis's advantageous properties within the realm of contrastive learning. To further leverage the interpretability properties of non-negative factor analysis, which can learn disentangled representations, contrastive factor analysis is extended to a non-negative version. Finally, extensive experimental validation showcases the efficacy of the proposed contrastive (non-negative) factor analysis methodology across multiple key properties, including expressiveness, robustness, interpretability, and accurate uncertainty estimation.
Abstract (translated)
因素分析，通常被认为是矩阵分解的贝叶斯变体，在捕捉不确定性、建模复杂依赖关系和确保鲁棒性方面具有卓越的性能。随着深度学习时代的到来，因素分析因其表达能力有限而受到越来越少的关注。相反，对比学习作为一种在无监督表示学习上表现出强大效果的技术而得到了越来越多的关注。虽然这两种方法是不同的范式，但最近的理论分析揭示了对比学习和矩阵分解之间的数学等价性，为将因素分析与对比学习相结合提供了潜在可能性。受到对比学习、矩阵分解和因素分析之间相互联系的启发，本文提出了一种新颖的对比因子分析框架，旨在在对比学习的领域充分利用因素分析的优势。为了进一步利用非负因素分析的可解释性特性（可以学习解离表示），对比因子分析被扩展到了非负版本。最后，广泛的实验验证展示了所提出的对比（非负）因子分析方法在多个关键属性（包括表现力、鲁棒性、可解释性和准确的不确定性估计）上的有效性。
URL
https://arxiv.org/abs/2407.21740
