Abstract
Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches
Abstract (translated)
深度神经网络在诸如医疗诊断和自动驾驶等安全关键型应用中的使用越来越广泛。然而，许多研究表明这些模型容易出现校准不良的问题，并且倾向于过度自信，这可能会导致灾难性后果。在这篇论文中，不同于标准的训练方法如随机梯度下降，我们展示了最近提出的敏锐度感知最小化（SAM）可以对抗这种过度自信的趋势。理论分析表明，SAM允许我们学习出已经具有良好校准性的模型，通过隐式地最大化预测分布的熵来实现这一点。受到这一发现的启发，我们进一步提出了一种名为CSAM的SAM变体，以改进模型校准。在包括ImageNet-1K在内的多个数据集上的大量实验表明，SAM能够减少校准误差。与此同时，CSAM的表现甚至比SAM还要好，并且始终优于其他方法，在所有情况下均实现了更低的校准误差。
URL
https://arxiv.org/abs/2505.23866
