Abstract
Generative artificial intelligence (AI) is rapidly populating medical records with synthetic content, creating a feedback loop where future models are increasingly at risk of training on uncurated AI-generated data. However, the clinical consequences of this AI-generated data contamination remain unexplored. Here, we show that in the absence of mandatory human verification, this self-referential cycle drives a rapid erosion of pathological variability and diagnostic reliability. By analysing more than 800,000 synthetic data points across clinical text generation, vision-language reporting, and medical image synthesis, we find that models progressively converge toward generic phenotypes regardless of the model architecture. Specifically, rare but critical findings, including pneumothorax and effusions, vanish from the synthetic content generated by AI models, while demographic representations skew heavily toward middle-aged male phenotypes. Crucially, this degradation is masked by false diagnostic confidence; models continue to issue reassuring reports while failing to detect life-threatening pathology, with false reassurance rates tripling to 40%. Blinded physician evaluation confirms that this decoupling of confidence and accuracy renders AI-generated documentation clinically useless after just two generations. We systematically evaluate three mitigation strategies, finding that while synthetic volume scaling fails to prevent collapse, mixing real data with quality-aware filtering effectively preserves diversity. Ultimately, our results suggest that without policy-mandated human oversight, the deployment of generative AI threatens to degrade the very healthcare data ecosystems it relies upon.
Abstract (translated)
生成式人工智能（AI）正迅速用合成内容填充医疗记录，形成一个反馈循环，在此过程中未来的模型越来越有可能在未经筛选的自动生成数据上进行训练。然而，这种由AI生成的数据污染所导致的临床后果尚不清楚。在此研究中，我们展示了当缺乏强制性的人类验证时，这一自我参照循环会导致病理变化多样性和诊断可靠性的迅速下降。 通过分析超过80万个合成数据点（涵盖临床文本生成、视觉-语言报告和医学图像合成），我们发现无论模型架构如何，这些模型会逐渐趋向于通用的表型。具体而言，罕见但关键的发现——如气胸和积液——从AI生成的内容中消失，而人口统计学代表则严重偏向于中年男性表型。 至关重要的是，这种退化被虚假的诊断信心所掩盖；尽管模型未能检测到危及生命的病理变化，它们仍会继续发出令人安心的报告，这导致虚假安慰率增至原来的三倍达到40%。盲法医生评估确认，在仅经过两代发展后，AI生成文档由于自信度和准确性的脱节变得在临床上毫无用处。 我们系统地评估了三种缓解策略，发现尽管扩大合成数据量无法防止退化，但将真实数据与质量感知筛选相结合的方法能有效保持多样性。最终，我们的结果表明，在没有政策强制的人类监督的情况下，生成式AI的部署威胁到了它所依赖的医疗健康数据生态系统的质量。
URL
https://arxiv.org/abs/2601.12946
