Abstract
Unsupervised domain adaptation leverages abundant labeled data from various source domains to generalize onto unlabeled target data. Prior research has primarily focused on learning domain-invariant features across the source and target domains. However, these methods often require training a model using source domain data, which is time-consuming and can limit model usage for applications with different source data. This paper introduces a simple framework that utilizes the impressive generalization capabilities of Large Language Models (LLMs) for target data annotation without the need of source model training, followed by a novel similarity-based knowledge distillation loss. Our extensive experiments on cross-domain text classification reveal that our framework achieves impressive performance, specifically, 2.44\% accuracy improvement when compared to the SOTA method.
Abstract (translated)
无监督领域自适应利用来自多个源领域的大量标记数据来推广到未标记的目标数据。先前的研究主要集中在学习源域和目标域之间的领域不变特征上。然而，这些方法通常需要使用源域数据训练模型，这既耗时又限制了在不同源数据应用中的模型使用。本文介绍了一个简单的框架，该框架利用大型语言模型（LLM）出色的泛化能力为目标数据进行标注，并且无需对源模型进行训练，随后提出了一种基于相似性的知识蒸馏损失函数。我们在跨域文本分类的广泛实验中发现，我们的框架实现了令人印象深刻的性能，在与最先进方法相比时，准确率提高了2.44%。
URL
https://arxiv.org/abs/2503.05281
