Abstract
Background: Lung cancer ranks as the leading cause of cancer-related mortality worldwide. The complexity of tumor delineation, crucial for radiation therapy, requires expertise often unavailable in resource-limited settings. Artificial Intelligence(AI), particularly with advancements in deep learning (DL) and natural language processing (NLP), offers potential solutions yet is challenged by high false positive rates. Purpose: The Oncology Contouring Copilot (OCC) system is developed to leverage oncologist expertise for precise tumor contouring using textual descriptions, aiming to increase the efficiency of oncological workflows by combining the strengths of AI with human oversight. Methods: Our OCC system initially identifies nodule candidates from CT scans. Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively reduces false positives with clinical descriptive texts, merging textual and visual data to automate tumor delineation, designed to elevate the quality of oncology care by incorporating knowledge from experienced domain experts. Results: Deployments of the OCC system resulted in a significant reduction in the false discovery rate by 35.0%, a 72.4% decrease in false positives per scan, and an F1-score of 0.652 across our dataset for unbiased evaluation. Conclusions: OCC represents a significant advance in oncology care, particularly through the use of the latest LVMs to improve contouring results by (1) streamlining oncology treatment workflows by optimizing tumor delineation, reducing manual processes; (2) offering a scalable and intuitive framework to reduce false positives in radiotherapy planning using LVMs; (3) introducing novel medical language vision prompt techniques to minimize LVMs hallucinations with ablation study, and (4) conducting a comparative analysis of LVMs, highlighting their potential in addressing medical language vision challenges.
Abstract (translated)
背景：肺癌是全球癌症死亡的主要原因。肿瘤轮廓的复杂性对于放射治疗至关重要，但在资源有限的情况下，这种专业知识往往不可用。人工智能（AI），特别是深度学习（DL）和自然语言处理（NLP）的进步，提供了潜在解决方案，但面临高假阳性率的问题。 目的：我们开发了Oncology Contouring Copilot (OCC)系统，利用肿瘤学家的专业知识通过文本描述进行精确的肿瘤轮廓勾画，旨在通过结合AI的优势与人类监督来提高肿瘤学工作流程的效率。 方法：我们的OCC系统首先从CT扫描中识别出结节候选者。然后，它使用像GPT-4V这样的语言视觉模型（LVM）有效地利用临床描述性文本减少假阳性，并将文本和图像数据融合以自动化肿瘤勾画过程，旨在通过融入经验丰富的领域专家的知识来提升肿瘤学护理质量。 结果：OCC系统的部署在我们的数据集中显著降低了35.0%的错误发现率，每个扫描减少了72.4%的假阳性，并且F1分数为0.652。这证明了系统评估中的无偏性。 结论：OCC代表了肿瘤学护理的重要进展，特别是通过使用最新的LVM来改善轮廓绘制结果： (1) 通过优化肿瘤勾画过程，减少手动操作，从而简化肿瘤治疗工作流程； (2) 提供一个可扩展且直观的框架，利用LVM减少放射治疗计划中的假阳性； (3) 引入新的医疗语言视觉提示技术以最小化LVM幻觉，并进行消融研究验证； (4) 对LVM进行比较分析，突显其在解决医学语言视觉挑战方面的潜力。
URL
https://arxiv.org/abs/2503.14933
