Abstract
Object removal, a challenging task within image inpainting, involves seamlessly filling the removed region with content that matches the surrounding context. Despite advancements in diffusion models, current methods still face two critical challenges. The first is mask hallucination, where the model generates irrelevant or spurious content inside the masked region, and the second is mask-shape bias, where the model fills the masked area with an object that mimics the mask's shape rather than surrounding content. To address these issues, we propose Mask Consistency Regularization (MCR), a novel training strategy designed specifically for object removal tasks. During training, our approach introduces two mask perturbations: dilation and reshape, enforcing consistency between the outputs of these perturbed branches and the original mask. The dilated masks help align the model's output with the surrounding content, while reshaped masks encourage the model to break the mask-shape bias. This combination of strategies enables MCR to produce more robust and contextually coherent inpainting results. Our experiments demonstrate that MCR significantly reduces hallucinations and mask-shape bias, leading to improved performance in object removal.
Abstract (translated)
对象移除是图像修复中的一项具有挑战性的任务，它涉及到无缝地填充被移除的区域，并使其内容与周围环境相匹配。尽管在扩散模型方面已经取得了进展，但目前的方法仍然面临两个关键挑战：一是“掩码幻觉”，即模型会在遮罩区域内生成不相关或虚假的内容；二是“掩码形状偏见”，即模型会用与其掩码形状相似的对象填充被遮盖的区域，而不是根据周围环境进行填充。为了解决这些问题，我们提出了一种新的训练策略——Mask Consistency Regularization (MCR)，专门针对对象移除任务设计。 在训练过程中，我们的方法引入了两种掩码扰动：膨胀和重塑。前者通过使模型输出与周围内容对齐来强制这些扰动分支的输出与原始掩码保持一致性；后者则促使模型打破掩模形状偏见。这种策略组合使得MCR能够生成更加稳健且上下文一致的图像修复结果。 实验表明，MCR显著减少了幻觉和掩模形状偏差的发生，从而提高了对象移除任务中的性能。
URL
https://arxiv.org/abs/2509.10259
