Abstract
Quality Assurance (QA) for radiology reports refers to judging whether the junior reports (written by junior doctors) are qualified. The QA scores of one junior report are given by the senior doctor(s) after reviewing the image and junior report. This process requires intensive labor costs for senior doctors. Additionally, the QA scores may be inaccurate for reasons like diagnosis bias, the ability of senior doctors, and so on. To address this issue, we propose a Span-level Quality Assurance EvaluaTOR (Sqator) to mark QA scores automatically. Unlike the common document-level semantic comparison method, we try to analyze the semantic difference by exploring more fine-grained text spans. Unlike the common document-level semantic comparison method, we try to analyze the semantic difference by exploring more fine-grained text spans. Specifically, Sqator measures QA scores by measuring the importance of revised spans between junior and senior reports, and outputs the final QA scores by merging all revised span scores. We evaluate Sqator using a collection of 12,013 radiology reports. Experimental results show that Sqator can achieve competitive QA scores. Moreover, the importance scores of revised spans can be also consistent with the judgments of senior doctors.
Abstract (translated)
放射学报告的质量保证（QA）是指判断由初级医生撰写的报告是否合格。一份初级报告的QA评分是由资深医生在审查影像资料和报告后给出的，这需要大量资深医生的工作成本。此外，由于诊断偏见、资深医生的能力等因素，可能导致QA评分不准确。为解决这一问题，我们提出了一种名为“Span-level Quality Assurance EvaluaTOR”（Sqator）的方法，旨在自动标注QA评分。 不同于常见的文档级语义比较方法，我们尝试通过探索更细粒度的文本片段来分析语义差异。具体来说，Sqator通过测量初级和资深报告之间修订部分的重要性来衡量QA分数，并将所有修订部分的得分合并输出最终的QA分数。我们在12,013份放射学报告的数据集上评估了Sqator的效果。实验结果表明，Sqator能够实现具有竞争力的QA评分效果。此外，修订片段的重要程度评分与资深医生的评判也保持了一致性。
URL
https://arxiv.org/abs/2508.08876
