Abstract
The integration of Large Language Models (LLMs) into healthcare promises to transform medical diagnostics, research, and patient care. Yet, the progression of medical LLMs faces obstacles such as complex training requirements, rigorous evaluation demands, and the dominance of proprietary models that restrict academic exploration. Transparent, comprehensive access to LLM resources is essential for advancing the field, fostering reproducibility, and encouraging innovation in healthcare AI. We present Hippocrates, an open-source LLM framework specifically developed for the medical domain. In stark contrast to previous efforts, it offers unrestricted access to its training datasets, codebase, checkpoints, and evaluation protocols. This open approach is designed to stimulate collaborative research, allowing the community to build upon, refine, and rigorously evaluate medical LLMs within a transparent ecosystem. Also, we introduce Hippo, a family of 7B models tailored for the medical domain, fine-tuned from Mistral and LLaMA2 through continual pre-training, instruction tuning, and reinforcement learning from human and AI feedback. Our models outperform existing open medical LLMs models by a large-margin, even surpassing models with 70B parameters. Through Hippocrates, we aspire to unlock the full potential of LLMs not just to advance medical knowledge and patient care but also to democratize the benefits of AI research in healthcare, making them available across the globe.
Abstract (translated)
将大型语言模型（LLMs）融入医疗保健行业有望彻底改变医疗诊断、研究和患者护理。然而，医疗LLMs的发展面临着一些障碍，如复杂的训练要求、严格的评估需求以及 proprietary模型的主导地位，这些模型限制了学术探索。透明、全面的访问LLM资源对于推动该领域的发展、促进可重复性以及鼓励医疗保健领域的人工创新至关重要。我们推出了Hippocrates，一个专为医疗领域而设计的开源LLM框架。与之前的努力相比，它提供了无限制的访问其训练数据集、代码库、检查点以及评估协议。这种开放方法旨在鼓励协同研究，让社区在透明的生态系统中构建、改进和严格评估医疗LLM。此外，我们还介绍了Hippo家族7B模型，这些模型针对医疗领域进行了微调和优化，通过持续的预训练、指令调整和强化学习从人类和AI反馈中进行微调。我们的模型在现有开放医疗LLM模型的性能优势基础上，性能优势巨大，甚至超过了具有70B参数的模型。通过Hippocrates，我们渴望利用LLMs不仅推动医疗知识和患者护理的发展，还将促进医疗保健领域的人工研究民主化，使它们在全球范围内可用。
URL
https://arxiv.org/abs/2404.16621
