Abstract
Deep learning-based image manipulation localization (IML) methods have achieved remarkable performance in recent years, but typically rely on large-scale pixel-level annotated datasets. To address the challenge of acquiring high-quality annotations, some recent weakly supervised methods utilize image-level labels to segment manipulated regions. However, the performance is still limited due to insufficient supervision signals. In this study, we explore a form of weak supervision that improves the annotation efficiency and detection performance, namely scribble annotation supervision. We re-annotated mainstream IML datasets with scribble labels and propose the first scribble-based IML (Sc-IML) dataset. Additionally, we propose the first scribble-based weakly supervised IML framework. Specifically, we employ self-supervised training with a structural consistency loss to encourage the model to produce consistent predictions under multi-scale and augmented inputs. In addition, we propose a prior-aware feature modulation module (PFMM) that adaptively integrates prior information from both manipulated and authentic regions for dynamic feature adjustment, further enhancing feature discriminability and prediction consistency in complex scenes. We also propose a gated adaptive fusion module (GAFM) that utilizes gating mechanisms to regulate information flow during feature fusion, guiding the model toward emphasizing potential tampered regions. Finally, we propose a confidence-aware entropy minimization loss (${\mathcal{L}}_{ {CEM }}$). This loss dynamically regularizes predictions in weakly annotated or unlabeled regions based on model uncertainty, effectively suppressing unreliable predictions. Experimental results show that our method outperforms existing fully supervised approaches in terms of average performance both in-distribution and out-of-distribution.
Abstract (translated)
基于深度学习的图像篡改定位（IML）方法近年来取得了显著的成绩，但通常依赖大规模像素级别的标注数据集。为了解决获取高质量注释的挑战，一些最近的弱监督方法利用图级别标签来分割篡改区域。然而，由于缺乏足够的监督信号，性能仍然有限。在本研究中，我们探索了一种能够提高注解效率和检测性能的弱监督形式——即草稿标注（scribble annotation）。我们重新用草稿标签注释了主流IML数据集，并提出了首个基于草稿的IML（Sc-IML）数据集。此外，我们还提出了一种基于草稿的第一种弱监督IML框架。 具体而言，我们采用自监督训练方法，在多尺度和增强输入的情况下使用结构一致性损失来鼓励模型生成一致的预测结果。除此之外，我们提出了一个先验感知特征调制模块（PFMM），该模块能够从篡改区域和真实区域中自适应地整合先验信息来进行动态特性调整，进一步增强了复杂场景中的特异性区分能力和预测的一致性。另外，我们提出了一种门控自适应融合模块（GAFM），利用门控机制来调节特征融合过程中的信息流，指导模型着重于潜在的篡改区域。最后，我们提出了一个基于置信度感知熵最小化损失（${\mathcal{L}}_{ {CEM }}$）。这种损失根据模型不确定性动态地对弱标注或未标注区域进行预测调整，有效地抑制了不可靠的预测。 实验结果表明，我们的方法在分布内和分布外的数据上均优于现有的全监督方法，在平均性能方面取得了显著提升。
URL
https://arxiv.org/abs/2507.13018
