Abstract
Image forgery localization aims to precisely identify tampered regions within images, but it commonly depends on costly pixel-level annotations. To alleviate this annotation burden, weakly supervised image forgery localization (WSIFL) has emerged, yet existing methods still achieve limited localization performance as they mainly exploit intra-image consistency clues and lack external semantic guidance to compensate for weak supervision. In this paper, we propose ViLaCo, a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models (VLMs), enabling accurate pixel-level localization using only image-level labels. Specifically, ViLaCo first incorporates semantic knowledge through a vision-language feature modeling network, which jointly extracts textual and visual priors using pre-trained VLMs. Next, an adaptive vision-language reasoning network aligns textual semantics and visual features through mutual interactions, producing semantically aligned representations. Subsequently, these representations are passed into dual prediction heads, where the coarse head performs image-level classification and the fine head generates pixel-level localization masks, thereby bridging the gap between weak supervision and fine-grained localization. Moreover, a contrastive patch consistency module is introduced to cluster tampered features while separating authentic ones, facilitating more reliable forgery discrimination. Extensive experiments on multiple public datasets demonstrate that ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.
Abstract (translated)
图像伪造定位旨在精确识别图片中的篡改区域，但通常依赖于昂贵的像素级标注。为减轻这种标注负担，弱监督下的图像伪造定位（WSIFL）应运而生，然而现有方法仍因主要利用内部一致性线索并缺乏外部语义指导来弥补弱监督的不足，从而导致有限的定位性能。本文中我们提出了一种视觉-语言协作推理框架ViLaCo，它引入了从预训练的视觉-语言模型（VLM）蒸馏出的辅助语义监督，仅使用图像级标签即可实现准确的像素级定位。 具体来说，ViLaCo首先通过视觉-语言特征建模网络整合语义知识，该网络利用预先训练好的VLM联合提取文本和视觉先验。接下来，自适应视觉-语言推理网络通过相互作用对齐文本语义与视觉特征，生成语义一致的表示。随后，这些表示被传递到双预测头中：粗略头部执行图像级分类而精细头部产生像素级定位掩模，从而弥合了弱监督和细粒度定位之间的差距。此外，引入了一种对比式补丁一致性模块来聚类篡改特征并分离真实特征，以便更可靠地进行伪造鉴别。 在多个公共数据集上的广泛实验表明，ViLaCo显著优于现有的WSIFL方法，在检测和定位精度方面均达到了最先进的性能。
URL
https://arxiv.org/abs/2508.01338
