Abstract
Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance, and it is one area where image inpainting is widely used in real-world applications. The performance of an object remover is quantitatively evaluated by measuring the quality of object removal results, similar to how the performance of an image inpainter is gauged. Current works reporting quantitative performance evaluations utilize original images as references. In this letter, to validate the current evaluation methods cannot properly evaluate the performance of an object remover, we create a dataset with object removal ground truth and compare the evaluations made by the current methods using original images to those utilizing object removal ground truth images. The disparities between two evaluation sets validate that the current methods are not suitable for measuring the performance of an object remover. Additionally, we propose new evaluation methods tailored to gauge the performance of an object remover. The proposed methods evaluate the performance through class-wise object removal results and utilize images without the target class objects as a comparison set. We confirm that the proposed methods can make judgments consistent with human evaluators in the COCO dataset, and that they can produce measurements aligning with those using object removal ground truth in the self-acquired dataset.
Abstract (translated)
对象移除是指在保留图像整体外观的情况下，从图像中删除指定对象的过程，它是图像修复在现实应用中得到广泛使用的一个领域。对象移除算法的性能通过测量对象移除结果的质量进行定量评估，就像衡量图像修复性能一样。目前的工作报道了定量性能评估，它们使用原始图像作为参考。在本文中，为了验证当前评估方法不能正确评估对象移除算法的性能，我们创建了一个带有对象移除真实值的 dataset，并使用原始图像比较当前方法得到的评估结果和利用对象移除真实值图像得到的评估结果。两个评估集中的差异证实了当前方法不适合测量对象移除算法的性能。此外，我们提出了针对对象移除算法的性能评估的新方法。这些方法通过类级对象移除结果进行评估，并利用没有目标类物体作为比较集的图像。我们证实，所提出的方法可以在COCO数据集中让评估者做出一致的判断，并且可以产生与利用对象移除真实值数据集中的测量结果相一致的测量值。
URL
https://arxiv.org/abs/2404.11104
