Abstract
The advent of large language models (LLMs) capable of producing general-purpose representations lets us revisit the practicality of deep active learning (AL): By leveraging frozen LLM embeddings, we can mitigate the computational costs of iteratively fine-tuning large backbones. This study establishes a benchmark and systematically investigates the influence of LLM embedding quality on query strategies in deep AL. We employ five top-performing models from the massive text embedding benchmark (MTEB) leaderboard and two baselines for ten diverse text classification tasks. Our findings reveal key insights: First, initializing the labeled pool using diversity-based sampling synergizes with high-quality embeddings, boosting performance in early AL iterations. Second, the choice of the optimal query strategy is sensitive to embedding quality. While the computationally inexpensive Margin sampling can achieve performance spikes on specific datasets, we find that strategies like Badge exhibit greater robustness across tasks. Importantly, their effectiveness is often enhanced when paired with higher-quality embeddings. Our results emphasize the need for context-specific evaluation of AL strategies, as performance heavily depends on embedding quality and the target task.
Abstract (translated)
大型语言模型（LLMs）的出现，这些模型能够生成通用表示，使我们重新审视深度主动学习（AL）的实际可行性：通过利用冻结的LLM嵌入，我们可以减轻在大规模骨干网络上迭代微调所带来的计算成本。本研究建立了一个基准，并系统地调查了LLM嵌入质量对深度AL中查询策略的影响。我们在十个多样化的文本分类任务中使用来自大规模文本嵌入基准（MTEB）排行榜上的五个高性能模型和两个基线模型。 我们的发现揭示了一些关键的见解： 首先，基于多样性采样的方法来初始化标记池与高质量的嵌入协同工作，在早期AL迭代中的性能得到了显著提升。 其次，最佳查询策略的选择对嵌入质量敏感。虽然计算成本低廉的Margin抽样在某些数据集上可以实现性能峰值，但我们发现像Badge这样的策略在整个任务中表现出更强的稳健性。值得注意的是，当与更高质量的嵌入结合使用时，它们的有效性往往得到增强。 我们的结果强调了根据具体上下文评估AL策略的必要性，因为其表现很大程度上依赖于嵌入质量和目标任务的要求。
URL
https://arxiv.org/abs/2506.01992
