Abstract
This paper presents a comprehensive evaluation of cost-efficient Large Language Models (LLMs) for diverse biomedical tasks spanning both text and image modalities. We evaluated a range of closed-source and open-source LLMs on tasks such as biomedical text classification and generation, question answering, and multimodal image processing. Our experimental findings indicate that there is no single LLM that can consistently outperform others across all tasks. Instead, different LLMs excel in different tasks. While some closed-source LLMs demonstrate strong performance on specific tasks, their open-source counterparts achieve comparable results (sometimes even better), with additional benefits like faster inference and enhanced privacy. Our experimental results offer valuable insights for selecting models that are optimally suited for specific biomedical applications.
Abstract (translated)
本文对适用于多样化的生物医学任务（涵盖文本和图像模式）的成本效益型大型语言模型（LLMs）进行了全面评估。我们针对生物医学文本分类与生成、问答以及多模态图像处理等任务，测试了一系列的闭源和开源LLMs。我们的实验结果表明，并不存在一个能在所有任务中持续胜出其他模型的单一LLM。相反，不同的LLM在各自的任务上表现出色。尽管某些闭源LLM在特定任务上展示了强大的性能，但其对应的开源版本也能达到类似甚至更好的效果，同时还具备诸如推理速度更快和隐私保护更佳等额外优势。我们的实验结果为选择最适合特定生物医学应用的模型提供了宝贵的见解。
URL
https://arxiv.org/abs/2507.14045
