Abstract
PET/CT imaging is pivotal in oncology and nuclear medicine, yet summarizing complex findings into precise diagnostic impressions is labor-intensive. While LLMs have shown promise in medical text generation, their capability in the highly specialized domain of PET/CT remains underexplored. We introduce PET-F2I-41K (PET Findings-to-Impression Benchmark), a large-scale benchmark for PET/CT impression generation using LLMs, constructed from over 41k real-world reports. Using PET-F2I-41K, we conduct a comprehensive evaluation of 27 models across proprietary frontier LLMs, open-source generalist models, and medical-domain LLMs, and we develop a domain-adapted 7B model (PET-F2I-7B) fine-tuned from Qwen2.5-7B-Instruct via LoRA. Beyond standard NLG metrics (e.g., BLEU-4, ROUGE-L, BERTScore), we propose three clinically grounded metrics - Entity Coverage Rate (ECR), Uncovered Entity Rate (UER), and Factual Consistency Rate (FCR) - to assess diagnostic completeness and factual reliability. Experiments reveal that neither frontier nor medical-domain LLMs perform adequately in zero-shot settings. In contrast, PET-F2I-7B achieves substantial gains (e.g., 0.708 BLEU-4) and a 3.0x improvement in entity coverage over the strongest baseline, while offering advantages in cost, latency, and privacy. Beyond this modeling contribution, PET-F2I-41K establishes a standardized evaluation framework to accelerate the development of reliable and clinically deployable reporting systems for PET/CT.
Abstract (translated)
PET/CT成像在肿瘤学和核医学领域至关重要，但将复杂的发现总结为精确的诊断结论是一项繁重的任务。尽管大语言模型（LLMs）在医疗文本生成方面显示出潜力，但在高度专业化的PET/CT领域中其能力尚待探索。我们推出了PET-F2I-41K (PET发现到印象基准)，这是一个使用大规模语言模型进行PET/CT印象生成的大规模基准测试数据集，构建自超过41,000份真实世界报告。利用PET-F2I-41K，我们在专有前沿大语言模型、开源通用模型和医学领域大语言模型中对27种不同的模型进行了全面评估，并开发了一个经过领域适应的7B模型（PET-F2I-7B），它是从Qwen2.5-7B-Instruct通过低秩适应（LoRA）微调而来。除了标准自然语言生成（NLG）指标（例如BLEU-4、ROUGE-L和BERTScore）之外，我们还提出了三种基于临床实践的度量标准——实体覆盖率率（ECR）、未覆盖实体率（UER）和事实一致性率（FCR），以评估诊断完整性和事实可靠性。实验结果显示，在零样本设置下，无论是前沿模型还是医学领域专用的大语言模型都无法充分表现。相比之下，PET-F2I-7B在诸如BLEU-4这样的标准指标上取得了显著的提升（如0.708 BLEU-4），并且在实体覆盖率方面比最强基线提高了3.0倍，同时提供了成本、延迟和隐私方面的优势。除了模型贡献外，PET-F2I-41K还建立了一个标准化评估框架，以加速可靠且临床可部署的报告系统的发展，用于PET/CT。
URL
https://arxiv.org/abs/2603.10560
