Abstract
This paper presents a robust fine-tuning method designed for pre-trained 3D point cloud models, to enhance feature robustness in downstream fine-tuned models. We highlight the limitations of current fine-tuning methods and the challenges of learning robust models. The proposed method, named Weight-Space Ensembles for Fine-Tuning then Linear Probing (WiSE-FT-LP), integrates the original pre-training and fine-tuning models through weight space integration followed by Linear Probing. This approach significantly enhances the performance of downstream fine-tuned models under distribution shifts, improving feature robustness while maintaining high performance on the target distribution. We apply this robust fine-tuning method to mainstream 3D point cloud pre-trained models and evaluate the quality of model parameters and the degradation of downstream task performance. Experimental results demonstrate the effectiveness of WiSE-FT-LP in enhancing model robustness, effectively balancing downstream task performance and model feature robustness without altering the model structures.
Abstract (translated)
本文提出了一种用于预训练3D点云模型的稳健微调方法，以提高下游微调模型的特征鲁棒性。我们重点介绍了当前微调方法的局限性和学习稳健模型的挑战。所提出的方法，名为权重空间集成用于微调线性探测（WiSE-FT-LP），通过权重空间整合和线性探测来整合原始预训练和微调模型。这种方法在分布变化下显著增强了下游微调模型的性能，同时保持目标分布上的高性能。我们将这种稳健微调方法应用于主流3D点云预训练模型，并评估模型的参数质量和下游任务性能的退化。实验结果表明，WiSE-FT-LP在增强模型鲁棒性方面非常有效，有效地平衡了下游任务性能和模型特征鲁棒性，同时不改变模型结构。
URL
https://arxiv.org/abs/2404.16422
