Abstract
Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods. Recent approaches in image manipulation detection have largely been driven by fully supervised approaches, which require labor-intensive pixel-level annotations. Thus, it is essential to explore weakly supervised image manipulation localization methods that only require image-level binary labels for training. However, existing weakly supervised image manipulation methods overlook the importance of edge information for accurate localization, leading to suboptimal localization performance. To address this, we propose a Context-Aware Boundary Localization (CABL) module to aggregate boundary features and learn context-inconsistency for localizing manipulated areas. Furthermore, by leveraging Class Activation Mapping (CAM) and Segment Anything Model (SAM), we introduce the CAM-Guided SAM Refinement (CGSR) module to generate more accurate manipulation localization maps. By integrating two modules, we present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture. Our method achieves outstanding localization performance across multiple datasets.
Abstract (translated)
恶意图像篡改带来了社会风险，从而增强了有效图像篡改检测方法的重要性。近期的图像篡改检测方法主要依赖于完全监督的方法，这需要耗费大量人力进行像素级别的标注。因此，探索仅需基于图片级别二元标签训练的弱监督图像篡改定位方法至关重要。然而，现有的弱监督图像篡改方法忽视了边缘信息对于精确定位的重要性，导致其在定位性能上不尽如人意。为此，我们提出了一种上下文感知边界定位（CABL）模块，用于聚合边界特征并学习不一致的背景以精确定位篡改区域。此外，通过利用类激活映射（CAM）和分割一切模型（SAM），我们引入了CAM引导下的SAM细化（CGSR）模块，生成更精确的篡改定位图。结合这两种模块，基于双分支Transformer-CNN架构，我们提出了一个新颖的弱监督框架。我们的方法在多个数据集上实现了卓越的定位性能。
URL
https://arxiv.org/abs/2503.20294
