Abstract
Distribution shift is a common challenge in medical images obtained from different clinical centers, significantly hindering the deployment of pre-trained semantic segmentation models in real-world applications across multiple domains. Continual Test-Time Adaptation(CTTA) has emerged as a promising approach to address cross-domain shifts during continually evolving target domains. Most existing CTTA methods rely on incrementally updating model parameters, which inevitably suffer from error accumulation and catastrophic forgetting, especially in long-term adaptation. Recent prompt-tuning-based works have shown potential to mitigate the two issues above by updating only visual prompts. While these approaches have demonstrated promising performance, several limitations remain:1)lacking multi-scale prompt diversity, 2)inadequate incorporation of instance-specific knowledge, and 3)risk of privacy leakage. To overcome these limitations, we propose Multi-scale Global-Instance Prompt Tuning(MGIPT), to enhance scale diversity of prompts and capture both global- and instance-level knowledge for robust CTTA. Specifically, MGIPT consists of an Adaptive-scale Instance Prompt(AIP) and a Multi-scale Global-level Prompt(MGP). AIP dynamically learns lightweight and instance-specific prompts to mitigate error accumulation with adaptive optimal-scale selection mechanism. MGP captures domain-level knowledge across different scales to ensure robust adaptation with anti-forgetting capabilities. These complementary components are combined through a weighted ensemble approach, enabling effective dual-level adaptation that integrates both global and local information. Extensive experiments on medical image segmentation benchmarks demonstrate that our MGIPT outperforms state-of-the-art methods, achieving robust adaptation across continually changing target domains.
Abstract (translated)
医学图像中由于不同临床中心获取的数据分布差异（即数据偏移）是应用预训练的语义分割模型到实际跨域任务中的一个常见挑战。连续测试时自适应(CTTA)作为一种有前景的方法，旨在解决在持续变化的目标领域间迁移过程中遇到的跨域偏移问题。尽管现有的大部分CTTA方法通过逐步更新模型参数来应对这一挑战，但这种方法在长期自适应中不可避免地会累积误差，并且容易出现灾难性遗忘的问题，尤其是在长期内。最近基于提示微调的方法显示出减少上述两类问题（即错误累积和灾难性遗忘）的潜力，它们仅对视觉提示进行更新。虽然这些方法已经展现了令人鼓舞的表现，但仍存在一些局限：1)缺乏多尺度提示多样性；2)未能充分整合实例特定知识；3)隐私泄露风险。 为了解决这些问题，我们提出了多尺度全局-实例提示微调(MGIPT)，以增强提示的规模多样性并捕捉全局和实例级别的知识，从而实现鲁棒性的CTTA。MGIPT由自适应尺度实例提示(AIP)和多尺度全局级提示(MGP)组成。AIP通过动态学习轻量且特定于实例的提示，并结合自适应最优尺度选择机制来减少错误累积的影响。而MGP则在不同尺度上捕获领域级别的知识，以确保具有防遗忘能力的鲁棒性自适应。 这两种互补组件通过加权集成方法相结合，从而能够有效地进行双级别调整，整合全局和局部信息。我们在医学图像分割基准数据集上的大量实验表明，我们的MGIPT优于现有最佳方法，在持续变化的目标领域中实现了稳健的适应性。
URL
https://arxiv.org/abs/2602.05937
