Abstract
Most pseudo-label selection strategies in semi-supervised learning rely on fixed confidence thresholds, implicitly assuming that prediction confidence reliably indicates correctness. In practice, deep networks are often overconfident: high-confidence predictions can still be wrong, while informative low-confidence samples near decision boundaries are discarded. This paper introduces a Confidence-Variance (CoVar) theory framework that provides a principled joint reliability criterion for pseudo-label selection. Starting from the entropy minimization principle, we derive a reliability measure that combines maximum confidence (MC) with residual-class variance (RCV), which characterizes how probability mass is distributed over non-maximum classes. The derivation shows that reliable pseudo-labels should have both high MC and low RCV, and that the influence of RCV increases as confidence grows, thereby correcting overconfident but unstable predictions. From this perspective, we cast pseudo-label selection as a spectral relaxation problem that maximizes separability in a confidence-variance feature space, and design a threshold-free selection mechanism to distinguish high- from low-reliability predictions. We integrate CoVar as a plug-in module into representative semi-supervised semantic segmentation and image classification methods. Across PASCAL VOC 2012, Cityscapes, CIFAR-10, and Mini-ImageNet with varying label ratios and backbones, it consistently improves over strong baselines, indicating that combining confidence with residual-class variance provides a more reliable basis for pseudo-label selection than fixed confidence thresholds. (Code: this https URL)
Abstract (translated)
半监督学习中大多数伪标签选择策略依赖于固定的置信度阈值，暗含地假设预测的置信度可靠地指示了正确性。在实践中，深度网络往往过于自信：即使高置信度的预测也可能出错，而位于决策边界附近的低置信度但信息量大的样本则被丢弃。本文介绍了一种基于置信度-方差（Confidence-Variance, CoVar）理论框架，该框架提供了一个原则性的联合可靠性标准用于伪标签选择。从熵最小化原理出发，我们推导出一种结合最大置信度（MC）和残余类方差（RCV）的可靠性测量方法，其中RCV描述了概率质量在非最大类中的分布情况。 我们的推导表明，可靠的伪标签应具有高MC和低RCV，并且随着置信度的增长，RCV的影响增大，从而纠正过于自信但不稳定的预测。从这个角度来看，我们将伪标签选择视为一种谱松弛问题，在置信度-方差特征空间中最大化可分离性，并设计了一种无需阈值的机制来区分高可靠性与低可靠性预测。 我们将CoVar作为插件模块整合进代表性的半监督语义分割和图像分类方法中。在PASCAL VOC 2012、Cityscapes、CIFAR-10和Mini-ImageNet数据集上，通过不同的标签比例和骨干网络进行实验，结果表明：CoVar的一致性改进了强大的基准模型，这表明结合置信度与残余类方差为伪标签选择提供了比固定置信度阈值更可靠的依据。 (代码地址: this https URL)
URL
https://arxiv.org/abs/2601.11670
