Abstract
Domain shift, characterized by degraded model performance during transition from labeled source domains to unlabeled target domains, poses a persistent challenge for deploying deep learning systems. Current unsupervised domain adaptation (UDA) methods predominantly rely on fine-tuning feature extractors - an approach limited by inefficiency, reduced interpretability, and poor scalability to modern architectures. Our analysis reveals that models pretrained on large-scale data exhibit domain-invariant geometric patterns in their feature space, characterized by intra-class clustering and inter-class separation, thereby preserving transferable discriminative structures. These findings indicate that domain shifts primarily manifest as boundary misalignment rather than feature degradation. Unlike fine-tuning entire pre-trained models - which risks introducing unpredictable feature distortions - we propose the Feature-space Planes Searcher (FPS): a novel domain adaptation framework that optimizes decision boundaries by leveraging these geometric patterns while keeping the feature encoder frozen. This streamlined approach enables interpretative analysis of adaptation while substantially reducing memory and computational costs through offline feature extraction, permitting full-dataset optimization in a single computation cycle. Evaluations on public benchmarks demonstrate that FPS achieves competitive or superior performance to state-of-the-art methods. FPS scales efficiently with multimodal large models and shows versatility across diverse domains including protein structure prediction, remote sensing classification, and earthquake detection. We anticipate FPS will provide a simple, effective, and generalizable paradigm for transfer learning, particularly in domain adaptation tasks. .
Abstract (translated)
领域转移（Domain Shift）是指在从标记的源域过渡到未标记的目标域时，模型性能下降的问题。这对部署深度学习系统构成了持续挑战。目前的无监督领域适应（UDA）方法主要依赖于微调特征提取器，这种方法由于效率低下、可解释性差以及难以扩展至现代架构而受到限制。我们的分析发现，基于大规模数据预训练的模型在它们的特征空间中表现出域不变的几何模式，表现为类内聚类和类间分离，从而保留了可转移的判别结构。这些发现表明领域转换主要表现为边界错位而非特征退化。与微调整个预训练模型相比（这可能会引入不可预测的特征扭曲），我们提出了一种新颖的领域适应框架——Feature-space Planes Searcher (FPS)，它通过利用这些几何模式来优化决策边界，同时保持特征编码器冻结不变。这种简化的途径不仅可以实现对适应过程的理解性分析，并且通过离线特征提取大幅减少内存和计算成本，在单次计算周期中即可进行全数据集优化。在公共基准测试中的评估表明，FPS能够达到与最新方法相当或更好的性能。此外，FPS可以高效地扩展到多模态大型模型，并表现出跨蛋白质结构预测、远程传感分类以及地震检测等多样化领域的灵活性和通用性。我们预期FPS将为迁移学习（尤其是领域适应任务）提供一种简单、有效且可推广的范式。
URL
https://arxiv.org/abs/2508.18693
