Abstract
Deep learning (DL) methods are currently being explored to restore images from sparse-view-, limited-data-, and undersampled-based acquisitions in medical applications. Although outputs from DL may appear visually appealing based on likability/subjective criteria (such as less noise, smooth features), they may also suffer from hallucinations. This issue is further exacerbated by a lack of easy-to-use techniques and robust metrics for the identification of hallucinations in DL outputs. In this work, we propose performing Fourier Ring Correlation (FRC) analysis over small patches and concomitantly (s)canning across DL outputs and their reference counterparts to detect hallucinations (termed as sFRC). We describe the rationale behind sFRC and provide its mathematical formulation. The parameters essential to sFRC may be set using predefined hallucinated features annotated by subject matter experts or using imaging theory-based hallucination maps. We use sFRC to detect hallucinations for three undersampled medical imaging problems: CT super-resolution, CT sparse view, and MRI subsampled restoration. In the testing phase, we demonstrate sFRC's effectiveness in detecting hallucinated features for the CT problem and sFRC's agreement with imaging theory-based outputs on hallucinated feature maps for the MR problem. Finally, we quantify the hallucination rates of DL methods on in-distribution versus out-of-distribution data and under increasing subsampling rates to characterize the robustness of DL methods. Beyond DL-based methods, sFRC's effectiveness in detecting hallucinations for a conventional regularization-based restoration method and a state-of-the-art unrolled method is also shown.
Abstract (translated)
深度学习（DL）方法目前被探索用于从稀疏视图、有限数据和欠采样获取的图像中恢复医学应用中的影像。虽然基于美观性/主观标准（如减少噪声，平滑特征），深度学习输出可能看起来很有吸引力，但它们也可能存在幻觉问题。这一问题由于缺乏易于使用的技术和稳健指标来识别深度学习输出中的幻觉而进一步加剧。在本工作中，我们提出对小块进行傅里叶环相关性（FRC）分析，并同时扫描深度学习输出及其参考图像以检测幻觉（称为sFRC）。我们描述了sFRC的原理并提供了其数学公式化方法。用于设置sFRC的关键参数可以通过使用领域专家注释的幻觉特征或基于成像理论的幻觉图来设定。 我们利用sFRC对三种欠采样医学影像问题进行幻觉检测：CT超分辨率、CT稀疏视图以及MRI欠采样恢复。在测试阶段，我们展示了sFRC在CT问题中检测到幻觉特征的有效性，并且对于MR问题，显示了sFRC与基于成像理论的输出对幻觉特征图的一致性。 最后，我们通过量化深度学习方法在分布内与分布外数据以及随着欠采样率增加时的幻觉频率来表征这些方法的鲁棒性。除深度学习方法外，还展示了sFRC对于传统正则化恢复方法和最新展开式方法检测幻觉的有效性。
URL
https://arxiv.org/abs/2603.04673
