Abstract
Recent progress in large language models (LLMs) has leveraged their in-context learning (ICL) abilities to enable quick adaptation to unseen biomedical NLP tasks. By incorporating only a few input-output examples into prompts, LLMs can rapidly perform these new tasks. While the impact of these demonstrations on LLM performance has been extensively studied, most existing approaches prioritize representativeness over diversity when selecting examples from large corpora. To address this gap, we propose Dual-Div, a diversity-enhanced data-efficient framework for demonstration selection in biomedical ICL. Dual-Div employs a two-stage retrieval and ranking process: First, it identifies a limited set of candidate examples from a corpus by optimizing both representativeness and diversity (with optional annotation for unlabeled data). Second, it ranks these candidates against test queries to select the most relevant and non-redundant demonstrations. Evaluated on three biomedical NLP tasks (named entity recognition (NER), relation extraction (RE), and text classification (TC)) using LLaMA 3.1 and Qwen 2.5 for inference, along with three retrievers (BGE-Large, BMRetriever, MedCPT), Dual-Div consistently outperforms baselines-achieving up to 5% higher macro-F1 scores-while demonstrating robustness to prompt permutations and class imbalance. Our findings establish that diversity in initial retrieval is more critical than ranking-stage optimization, and limiting demonstrations to 3-5 examples maximizes performance efficiency.
Abstract (translated)
最近在大型语言模型（LLMs）方面的进展，利用了它们的上下文学习（ICL）能力来快速适应未见过的生物医学自然语言处理任务。通过将少量输入输出示例整合到提示中，LLMs能够迅速执行这些新任务。尽管已经对这些示范如何影响LLM性能进行了广泛研究，但现有方法在从大型语料库中选择示例时通常优先考虑代表性而非多样性。为了解决这一空白，我们提出了Dual-Div，这是一种增强多样性的高效数据框架，用于生物医学ICL中的示范选择。Dual-Div采用两阶段的检索和排序过程：首先，通过优化代表性和多样性（可选地针对未标记数据进行注释），它从语料库中识别出一组有限的候选示例；其次，在测试查询的基础上对这些候选者进行排名，以选择最相关且非冗余的示范。在三项生物医学NLP任务（命名实体识别(NER)、关系抽取(RE)和文本分类(TC)）上进行了评估，使用LLaMA 3.1 和 Qwen 2.5 进行推理，并结合了三种检索器（BGE-Large、BMRetriever、MedCPT），Dual-Div始终超越基线方法，在所有任务中宏F1分数提高了高达5%，同时展示了对提示排列和类别不平衡的鲁棒性。我们的研究结果表明，初始检索阶段中的多样性比排序阶段优化更为关键，并且将示范限制为3-5个示例可以最大化性能效率。
URL
https://arxiv.org/abs/2508.08140
