Abstract
Machine Learning often involves various imprecise labels, leading to diverse weakly supervised settings. While recent methods aim for universal handling, they usually suffer from complex manual pre-work, ignore the relationships between associated labels, or are unable to batch process due to computational design flaws, resulting in long running times. To address these limitations, we propose a novel general framework that efficiently infers latent true label distributions across various weak supervisions. Our key idea is to express the label brute-force search process as a probabilistic transition of label variables, compressing diverse weakly supervised DFS tree structures into a shared Bayesian network. From this, we derived a latent probability calculation algorithm based on generalized belief propagation and proposed two joint acceleration strategies: 1) introducing a low-rank assumption to approximate the transition matrix, reducing time complexity; 2) designing an end-to-end state evolution module to learn batch-scale transition matrices, facilitating multi-category batch processing. In addition, the equivalence of our method with the EM algorithm in most scenarios is further demonstrated. Extensive experiments show that our method achieves SOTA results under most weakly supervised settings, and achieves up to hundreds of times faster acceleration in running time compared to other general methods.
Abstract (translated)
机器学习经常涉及各种不精确的标签，导致了多种弱监督设置。尽管最近的方法旨在实现通用处理，但它们通常需要复杂的手动前期工作、忽略相关标签之间的关系，或者由于计算设计缺陷而无法批量处理，从而导致运行时间过长。为了解决这些限制，我们提出了一种新的通用框架，该框架能够高效地在各种弱监督下推断潜在的真实标签分布。 我们的核心思想是将标签暴力搜索过程表达为标签变量的概率转换，并将多样的弱监督DFS（深度优先搜索）树结构压缩成一个共享的贝叶斯网络。基于此，我们从广义信念传播中导出了一个潜在概率计算算法，并提出了两种联合加速策略： 1. 引入低秩假设来近似转移矩阵，从而降低时间复杂度； 2. 设计了一个端到端状态演化模块以学习批量规模的转移矩阵，促进多类别批量处理。 此外，我们的方法在大多数情况下与EM（期望最大化）算法等价性也得到了进一步证明。广泛的实验表明，在大多数弱监督设置下，我们的方法实现了最先进的结果，并且相比其他通用方法能够实现运行时间最多快几百倍的加速效果。
URL
https://arxiv.org/abs/2603.00517
