Abstract
Fully test-time adaptation aims at adapting a pre-trained model to the test stream during real-time inference, which is urgently required when the test distribution differs from the training distribution. Several efforts have been devoted to improving adaptation performance. However, we find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning. First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates. Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes. In addition to the extensively studied test stream with independent and class-balanced samples, we further observe that the defects can be exacerbated in more complicated test environments, such as (time) dependent or class-imbalanced data. We observe that previous approaches work well in certain scenarios while show performance degradation in others due to their faults. In this paper, we provide a plug-in solution called DELTA for Degradation-freE fuLly Test-time Adaptation, which consists of two components: (i) Test-time Batch Renormalization (TBR), introduced to improve the estimated normalization statistics. (ii) Dynamic Online re-weighTing (DOT), designed to address the class bias within optimization. We investigate various test-time adaptation methods on three commonly used datasets with four scenarios, and a newly introduced real-world dataset. DELTA can help them deal with all scenarios simultaneously, leading to SOTA performance.
Abstract (translated)
完全测试时适应旨在在实时推理时适应预训练模型测试流,当测试分布与训练分布不同时,迫切需要适应。已花费大量努力来提高适应性能。然而,我们发现,现有的适应方法中存在两个不利的的缺陷,如测试时批量归一化(BN)和自学习。首先,我们揭示了测试时BN的归一化统计完全受到当前接收的测试样本的影响,导致不准确的估计。其次,我们表明,在测试时适应时,参数更新是偏向某些主要类别的。除了研究了独立和类平衡测试流之外,我们还进一步观察到,由于缺陷的存在,测试环境变得更加复杂,例如时间依赖或类不平衡数据。我们观察到,以前的方法和某些场景表现良好,而在其他情况下由于它们的缺陷而表现出性能下降。在本文中,我们提供了一种插件解决方案,称为DELTA,以退化测试时完全适应,它由两个组件组成:(一)测试时批量归一化(TBR),引入以提高估计的归一化统计。(二)动态在线重加权(DOT),旨在解决优化中的类偏差。我们研究了几种常用的数据集和四个场景,以及一个新引入的现实世界数据集,DELTA可以帮助它们同时处理所有场景,从而获得卓越的性能。
URL
https://arxiv.org/abs/2301.13018
