Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities on general text; however, their proficiency in specialized scientific domains that require deep, interconnected knowledge remains largely uncharacterized. Metabolomics presents unique challenges with its complex biochemical pathways, heterogeneous identifier systems, and fragmented databases. To systematically evaluate LLM capabilities in this domain, we introduce MetaBench, the first benchmark for metabolomics assessment. Curated from authoritative public resources, MetaBench evaluates five capabilities essential for metabolomics research: knowledge, understanding, grounding, reasoning, and research. Our evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks: while models perform well on text generation tasks, cross-database identifier grounding remains challenging even with retrieval augmentation. Model performance also decreases on long-tail metabolites with sparse annotations. With MetaBench, we provide essential infrastructure for developing and evaluating metabolomics AI systems, enabling systematic progress toward reliable computational tools for metabolomics research.
Abstract (translated)
大型语言模型（LLMs）在通用文本方面展示了卓越的能力；然而，它们在需要深厚、相互关联知识的专门化科学领域中的专业能力仍然缺乏系统性的表征。代谢组学因其复杂的生化途径、异质性标识符系统和碎片化的数据库而提出了独特的挑战。为了系统地评估LLM在该领域的表现，我们引入了MetaBench——第一个用于代谢组学评估的基准测试工具。MetaBench从权威的公共资源中精心挑选内容，以评估代谢组学研究所需的五种关键能力：知识、理解、实体关联、推理和科研支持。对25个开源及闭源LLM进行的评估显示，在代谢组学任务上表现出不同的性能模式：尽管模型在文本生成任务上表现良好，但在跨数据库标识符关联的任务中仍然面临挑战，即使增加了检索增强也无法显著改善这一状况。此外，对于那些注释稀疏的长尾代谢物，模型的表现也会下降。通过MetaBench，我们为开发和评估代谢组学AI系统提供了重要的基础设施，从而推动了可靠计算工具在代谢组学研究中的系统性进步。
URL
https://arxiv.org/abs/2510.14944
