Abstract
Instruction-based image editing models offer increased personalization opportunities in generative tasks. However, properly evaluating their results is challenging, and most of the existing metrics lag in terms of alignment with human judgment and explainability. To tackle these issues, we introduce DICE (DIfference Coherence Estimator), a model designed to detect localized differences between the original and the edited image and to assess their relevance to the given modification request. DICE consists of two key components: a difference detector and a coherence estimator, both built on an autoregressive Multimodal Large Language Model (MLLM) and trained using a strategy that leverages self-supervision, distillation from inpainting networks, and full supervision. Through extensive experiments, we evaluate each stage of our pipeline, comparing different MLLMs within the proposed framework. We demonstrate that DICE effectively identifies coherent edits, effectively evaluating images generated by different editing models with a strong correlation with human judgment. We publicly release our source code, models, and data.
Abstract (translated)
指令驱动的图像编辑模型为生成任务提供了更多的个性化机会。然而，对其结果进行恰当评估颇具挑战性，并且现有的大多数指标在与人类判断的一致性和可解释性方面存在不足。为了应对这些问题，我们引入了DICE（DIfference Coherence Estimator），这是一种旨在检测原始图像和编辑后图像之间的局部差异并评估这些差异对给定修改请求的相关性的模型。DICE由两个关键组件组成：一个差分检测器和一个一致性估算器，它们都基于自回归多模态大型语言模型（MLLM）构建，并通过一种利用自我监督、从修复网络的蒸馏以及完全监督的方法进行训练。 通过广泛的实验，我们评估了管道中的每个阶段，并在所提出的框架内比较了不同的MLLM。结果显示，DICE能够有效识别连贯的编辑，并且可以有效地对由不同编辑模型生成的图像进行评价，其结果与人类判断有很强的相关性。我们将源代码、模型和数据公开发布。
URL
https://arxiv.org/abs/2505.20405
