Abstract
The advancement of Large Language Models (LLMs) has significantly impacted biomedical Natural Language Processing (NLP), enhancing tasks such as named entity recognition, relation extraction, event extraction, and text classification. In this context, the DeepSeek series of models have shown promising potential in general NLP tasks, yet their capabilities in the biomedical domain remain underexplored. This study evaluates multiple DeepSeek models (Distilled-DeepSeek-R1 series and Deepseek-LLMs) across four key biomedical NLP tasks using 12 datasets, benchmarking them against state-of-the-art alternatives (Llama3-8B, Qwen2.5-7B, Mistral-7B, Phi-4-14B, Gemma-2-9B). Our results reveal that while DeepSeek models perform competitively in named entity recognition and text classification, challenges persist in event and relation extraction due to precision-recall trade-offs. We provide task-specific model recommendations and highlight future research directions. This evaluation underscores the strengths and limitations of DeepSeek models in biomedical NLP, guiding their future deployment and optimization.
Abstract (translated)
大型语言模型（LLMs）的发展在生物医学自然语言处理（NLP）领域产生了显著影响，提升了诸如命名实体识别、关系抽取、事件抽取和文本分类等任务的效果。在此背景下，DeepSeek系列模型在通用NLP任务中展示了巨大的潜力，但在生物医学领域的应用能力仍有待探索。本研究评估了多个DeepSeek模型（Distilled-DeepSeek-R1系列和Deepseek-LLMs）在四个关键的生物医学NLP任务中的表现，并使用12个数据集将它们与最先进的替代模型（如Llama3-8B、Qwen2.5-7B、Mistral-7B、Phi-4-14B、Gemma-2-9B）进行基准测试。我们的研究结果表明，尽管DeepSeek模型在命名实体识别和文本分类任务上表现出色，但在事件抽取和关系提取方面仍面临精度与召回率之间的权衡问题。我们提供了针对特定任务的模型推荐，并指出了未来的研究方向。这一评估强调了DeepSeek模型在生物医学NLP领域的优势和局限性，为它们未来的部署和优化提供指导。
URL
https://arxiv.org/abs/2503.00624
