Abstract
This paper introduces Least Volume-a simple yet effective regularization inspired by geometric intuition-that can reduce the necessary number of latent dimensions needed by an autoencoder without requiring any prior knowledge of the intrinsic dimensionality of the dataset. We show that the Lipschitz continuity of the decoder is the key to making it work, provide a proof that PCA is just a linear special case of it, and reveal that it has a similar PCA-like importance ordering effect when applied to nonlinear models. We demonstrate the intuition behind the regularization on some pedagogical toy problems, and its effectiveness on several benchmark problems, including MNIST, CIFAR-10 and CelebA.
Abstract (translated)
本文介绍了一种简单而有效的正则化方法，名为 Least Volume，该方法受到几何直觉的启发，可以减少不需要的潜在维度，同时不需要任何关于数据集固有维度的先验知识。我们证明，解码器的Lipschitz连续性是其工作的关键，给出PCA只是其线性特殊情况的证明，并揭示其在非线性模型上具有与PCA类似的重要性排序效应。我们证明了在某些教育玩具问题和多个基准问题上的直觉，包括MNIST，CIFAR-10和CelebA。
URL
https://arxiv.org/abs/2404.17773
