Abstract
We present AIForge-Doc, the first dedicated benchmark targeting exclusively diffusion-model-based inpainting in financial and form documents with pixel-level annotation. Existing document forgery datasets rely on traditional digital editing tools (e.g., Adobe Photoshop, GIMP), creating a critical gap: state-of-the-art detectors are blind to the rapidly growing threat of AI-forged document fraud. AIForge-Doc addresses this gap by systematically forging numeric fields in real-world receipt and form images using two AI inpainting APIs -- Gemini 2.5 Flash Image and Ideogram v2 Edit -- yielding 4,061 forged images from four public document datasets (CORD, WildReceipt, SROIE, XFUND) across nine languages, annotated with pixel-precise tampered-region masks in DocTamper-compatible format. We benchmark three representative detectors -- TruFor, DocTamper, and a zero-shot GPT-4o judge -- and find that all existing methods degrade substantially: TruFor achieves AUC=0.751 (zero-shot, out-of-distribution) vs. AUC=0.96 on NIST16; DocTamper achieves AUC=0.563 vs. AUC=0.98 in-distribution, with pixel-level IoU=0.020; GPT-4o achieves only 0.509 -- essentially at chance -- confirming that AI-forged values are indistinguishable to automated detectors and VLMs. These results demonstrate that AIForge-Doc represents a qualitatively new and unsolved challenge for document forensics.
Abstract (translated)
我们介绍了AIForge-Doc，这是第一个专注于使用扩散模型进行图像修复的文档伪造基准测试，该基准包含针对财务和表格文档的像素级注释。现有的文档伪造数据集依赖于传统的数字编辑工具（例如Adobe Photoshop、GIMP），这造成了一个关键差距：最先进的检测器对于日益增长的人工智能伪造文档欺诈威胁视而不见。AIForge-Doc通过使用两个AI修复API——Gemini 2.5 Flash Image和Ideogram v2 Edit，在真实世界的收据和表格图像中系统性地伪造数字字段，从而填补了这一空白。这产生了4,061张伪造的图片，这些图片来自四个公开的数据集（CORD、WildReceipt、SROIE、XFUND），涵盖九种语言，并以DocTamper兼容格式进行了像素级篡改区域标注。 我们评估了三种代表性的检测器——TruFor、DocTamper以及零样本GPT-4o法官，并发现所有现有方法在效果上都有显著下降：TruFor的AUC值为0.751（零样本，分布外）对比NIST16上的0.96；DocTamper的AUC值为0.563（分布内），与之相比其像素级IoU仅为0.020；GPT-4o仅达到0.509——基本等同于随机猜测。这些结果证实了AI伪造的价值对于自动化检测器和视觉语言模型而言是无法区分的。 这一成果表明，AIForge-Doc代表了一个文档取证领域中的全新且未解决的重大挑战。
URL
https://arxiv.org/abs/2602.20569
