Abstract
Medical image analysis increasingly relies on large vision-language models (VLMs), yet most systems remain single-pass black boxes that offer limited control over reasoning, safety, and spatial grounding. We propose R^4, an agentic framework that decomposes medical imaging workflows into four coordinated agents: a Router that configures task- and specialization-aware prompts from the image, patient history, and metadata; a Retriever that uses exemplar memory and pass@k sampling to jointly generate free-text reports and bounding boxes; a Reflector that critiques each draft-box pair for key clinical error modes (negation, laterality, unsupported claims, contradictions, missing findings, and localization errors); and a Repairer that iteratively revises both narrative and spatial outputs under targeted constraints while curating high-quality exemplars for future cases. Instantiated on chest X-ray analysis with multiple modern VLM backbones and evaluated on report generation and weakly supervised detection, R^4 consistently boosts LLM-as-a-Judge scores by roughly +1.7-+2.5 points and mAP50 by +2.5-+3.5 absolute points over strong single-VLM baselines, without any gradient-based fine-tuning. These results show that agentic routing, reflection, and repair can turn strong but brittle VLMs into more reliable and better grounded tools for clinical image interpretation. Our code can be found at: this https URL
Abstract (translated)
医学图像分析越来越依赖于大型视觉语言模型（VLM），然而，大多数系统仍然是一次性黑盒操作，对推理、安全和空间定位的控制有限。我们提出了R^4框架，这是一个代理框架，将医学成像工作流程分解为四个协调的代理：一个路由器，根据从图像、患者历史记录和元数据中提取的任务和专业信息来配置提示；一个检索器，使用范例记忆和pass@k抽样同时生成自由文本报告和边界框；一个反思者，批评每对草案-边框的关键临床错误模式（否定、侧向性、缺乏支持的主张、矛盾、遗漏发现以及定位错误）；以及一个修复者，在有针对性的约束下迭代修订叙述性和空间输出的同时为未来案例策划高质量范例。在胸部X光片分析中使用多个现代VLM后端，并通过报告生成和弱监督检测进行评估，R^4 在不进行任何基于梯度的微调的情况下，相对于强大的单一VLM基线，显著提高了LLM作为裁判（LLM-as-a-Judge）评分约1.7至2.5分，并将mAP50绝对提高了2.5至3.5个点。这些结果表明，代理路由、反思和修复可以将强大但脆弱的VLM转变为更可靠且更好地定位的临床图像解释工具。我们的代码可在以下链接找到：[此URL]
URL
https://arxiv.org/abs/2601.08192
