Abstract
We present Common Inpainted Objects In-N-Out of Context (COinCO), a novel dataset addressing the scarcity of out-of-context examples in existing vision datasets. By systematically replacing objects in COCO images through diffusion-based inpainting, we create 97,722 unique images featuring both contextually coherent and inconsistent scenes, enabling effective context learning. Each inpainted object is meticulously verified and categorized as in- or out-of-context through a multimodal large language model assessment. Our analysis reveals significant patterns in semantic priors that influence inpainting success across object categories. We demonstrate three key tasks enabled by COinCO: (1) training context classifiers that effectively determine whether existing objects belong in their context; (2) a novel Objects-from-Context prediction task that determines which new objects naturally belong in given scenes at both instance and clique levels, and (3) context-enhanced fake detection on state-of-the-art methods without fine-tuning. COinCO provides a controlled testbed with contextual variations, establishing a foundation for advancing context-aware visual understanding in computer vision and image forensics. Our code and data are at: this https URL.
Abstract (translated)
我们介绍了“Common Inpainted Objects In-N-Out of Context”(简称COinCO)，这是一个新颖的数据集，旨在解决现有视觉数据集中缺少上下文外示例的问题。通过利用基于扩散的修复技术系统性地替换COCO图像中的对象，我们创建了97,722张独特的图像，这些图像是经过精心设计的，并且包含在上下文中一致和不一致的场景，从而能够有效地进行上下文学习。每个被修复的对象都经过多模态大型语言模型的评估，以确保其分类为上下文内或上下文外对象。我们的分析揭示了影响不同类别物体修复成功的语义先验模式。 COinCO支持三个关键任务：(1) 训练能够有效判断现有物体是否属于它们所处环境中的上下文分类器；(2) 一种新的“从上下文中预测物体”任务，该任务确定哪些新物体在给定场景中自然地存在，并且以实例和团簇级别进行评估；以及 (3) 在无需微调的情况下使用最先进的方法进行增强的假图像检测。 COinCO提供了一个具有上下文变化的受控测试平台，为计算机视觉和图像取证中的上下文感知视觉理解研究奠定了基础。我们的代码和数据可在以下网址获取：this https URL.
URL
https://arxiv.org/abs/2506.00721
