Abstract
We propose \textbf{A$^2$-Edit}, a unified inpainting framework for arbitrary object categories, which allows users to replace any target region with a reference object using only a coarse mask. To address the issues of severe homogenization and limited category coverage in existing datasets, we construct a large-scale, multi-category dataset \textbf{UniEdit-500K}, which includes 8 major categories, 209 fine-grained subcategories, and a total of 500,104 image pairs. Such rich category diversity poses new challenges for the model, requiring it to automatically learn semantic relationships and distinctions across categories. To this end, we introduce the \textbf{Mixture of Transformer} module, which performs differentiated modeling of various object categories through dynamic expert selection, and further enhances cross-category semantic transfer and generalization through collaboration among experts. In addition, we propose a \textbf{Mask Annealing Training Strategy} (MATS) that progressively relaxes mask precision during training, reducing the model's reliance on accurate masks and improving robustness across diverse editing tasks. Extensive experiments on benchmarks such as VITON-HD and AnyInsertion demonstrate that A$^2$-Edit consistently outperforms existing approaches across all metrics, providing a new and efficient solution for arbitrary object editing.
Abstract (translated)
我们提出了一种名为\textbf{A$^2$-Edit}的统一修补框架，该框架针对任意对象类别设计，允许用户仅通过一个粗略掩码就能将目标区域替换为参考对象。为了应对现有数据集中严重同质化和有限类别覆盖的问题，我们构建了一个大规模、多类别的数据集\textbf{UniEdit-500K}，它包括8个主要类别，209个细粒度子类别以及总共500,104对图像。这种丰富的类别多样性给模型带来了新的挑战，需要模型自动学习不同类别间的语义关系和区分性。为此，我们引入了\textbf{Transformer混合}模块，该模块通过动态专家选择来进行各种对象类别的差异化建模，并通过专家之间的协作进一步增强跨类别的语义传递和泛化能力。此外，我们还提出了一种名为\textbf{掩码退火训练策略}(MATS)的方法，在训练过程中逐步放松对掩码精度的要求，减少模型对准确掩码的依赖并提高其在各种编辑任务中的鲁棒性。在VITON-HD和AnyInsertion等基准测试上的广泛实验表明，A$^2$-Edit在所有指标上都始终优于现有方法，为任意对象编辑提供了一种新的高效解决方案。
URL
https://arxiv.org/abs/2603.10685
