Abstract
Recent advances in diffusion models (DMs) have achieved exceptional visual quality in image editing tasks. However, the global denoising dynamics of DMs inherently conflate local editing targets with the full-image context, leading to unintended modifications in non-target regions. In this paper, we shift our attention beyond DMs and turn to Masked Generative Transformers (MGTs) as an alternative approach to tackle this challenge. By predicting multiple masked tokens rather than holistic refinement, MGTs exhibit a localized decoding paradigm that endows them with the inherent capacity to explicitly preserve non-relevant regions during the editing process. Building upon this insight, we introduce the first MGT-based image editing framework, termed EditMGT. We first demonstrate that MGT's cross-attention maps provide informative localization signals for localizing edit-relevant regions and devise a multi-layer attention consolidation scheme that refines these maps to achieve fine-grained and precise localization. On top of these adaptive localization results, we introduce region-hold sampling, which restricts token flipping within low-attention areas to suppress spurious edits, thereby confining modifications to the intended target regions and preserving the integrity of surrounding non-target areas. To train EditMGT, we construct CrispEdit-2M, a high-resolution dataset spanning seven diverse editing categories. Without introducing additional parameters, we adapt a pre-trained text-to-image MGT into an image editing model through attention injection. Extensive experiments across four standard benchmarks demonstrate that, with fewer than 1B parameters, our model achieves similarity performance while enabling 6 times faster editing. Moreover, it delivers comparable or superior editing quality, with improvements of 3.6% and 17.6% on style change and style transfer tasks, respectively.
Abstract (translated)
近期，扩散模型（DMs）在图像编辑任务中取得了卓越的视觉效果。然而，DMs 的全局去噪动态特性会将局部编辑目标与整幅图像上下文混淆在一起，导致非目标区域出现未预期的修改。为此，本文转向探讨遮罩生成变换器（MGTs），作为一种替代方法来应对这一挑战。通过预测多个被屏蔽的标记而非整体细化，MGT 展现了本地化解码范式，并因此具备在编辑过程中固有的能力，能够显式地保留非相关区域。 基于此见解，我们引入首个基于 MGT 的图像编辑框架——EditMGT。首先，我们证明了 MGT 的交叉注意力图提供了有关编辑相关区域定位的丰富信息信号，并设计了一种多层注意力整合方案以细化这些地图，从而实现精细化和精准化定位。在此基础上，我们提出了区域保持采样策略，在低关注度区域内限制标记翻转操作，抑制意外修改，使修改局限于目标区域并保留周围非目标区域的完整性。 为了训练 EditMGT，我们构建了 CrispEdit-2M 数据集，这是一个涵盖七类多样化编辑任务的高分辨率数据集。在不增加额外参数的情况下，通过注意力注入的方式将预训练的文字到图像 MGT 适配为图像编辑模型。跨四个标准基准进行的广泛实验表明，在少于10亿个参数的情况下，我们的模型实现了相似的表现水平，并使编辑速度提高了六倍。此外，它提供了相当或更好的编辑质量，风格变化任务和风格迁移任务分别提高了3.6% 和 17.6% 的性能。
URL
https://arxiv.org/abs/2512.11715
