Abstract
We consider the problem of estimating a regularization parameter, or a shrinkage coefficient $\alpha \in (0,1)$ for Regularized Tyler's M-estimator (RTME). In particular, we propose to estimate an optimal shrinkage coefficient by setting $\alpha$ as the solution to a suitably chosen objective function; namely the leave-one-out cross-validated (LOOCV) log-likelihood loss. Since LOOCV is computationally prohibitive even for moderate sample size $n$, we propose a computationally efficient approximation for the LOOCV log-likelihood loss that eliminates the need for invoking the RTME procedure $n$ times for each sample left out during the LOOCV procedure. This approximation yields an $O(n)$ reduction in the running time complexity for the LOOCV procedure, which results in a significant speedup for computing the LOOCV estimate. We demonstrate the efficiency and accuracy of the proposed approach on synthetic high-dimensional data sampled from heavy-tailed elliptical distributions, as well as on real high-dimensional datasets for object recognition, face recognition, and handwritten digit's recognition. Our experiments show that the proposed approach is efficient and consistently more accurate than other methods in the literature for shrinkage coefficient estimation.
Abstract (translated)
我们考虑了估计正则化参数或Regularized Tyler的M-估计量(RTME)中的收缩系数$\alpha \in (0,1)$的问题。具体而言，我们建议通过将$\alpha$设置为一个合适的目标函数的解来估计最优的收缩系数；即，leave-one-out交叉验证(LOOCV)对数似然损失函数。由于即使对于中等样本量$n$，LOOCV在计算上也是不可行的，我们提出了一种用于LOOCV对数似然损失的计算效率高的近似方法，该方法消除了在LOOCV过程中每次留出一个样本时调用RTME程序$n$次的需求。这种近似方法将LOOCV过程的时间复杂度降低了$O(n)$，从而大大加快了LOOCV估计的计算速度。我们在从重型长尾椭圆分布中采样的合成高维数据以及用于物体识别、人脸识别和手写数字识别的实际高维数据集上展示了所提出的方法的有效性和准确性。我们的实验表明，与文献中的其他收缩系数估算方法相比，所提出的方法不仅效率更高，而且始终更准确。
URL
https://arxiv.org/abs/2505.24781
