Abstract
Existing text-guided image editing methods primarily rely on end-to-end pixel-level inpainting paradigm. Despite its success in simple scenarios, this paradigm still significantly struggles with compositional editing tasks that require precise local control and complex multi-object spatial reasoning. This paradigm is severely limited by 1) the implicit coupling of planning and execution, 2) the lack of object-level control granularity, and 3) the reliance on unstructured, pixel-centric modeling. To address these limitations, we propose I2E, a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions into a series of atomic actions via Chain-of-Thought reasoning. Further, we also construct I2E-Bench, a benchmark designed for multi-instance spatial reasoning and high-precision editing. Experimental results on I2E-Bench and multiple public benchmarks demonstrate that I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.
Abstract (translated)
现有的文本引导图像编辑方法主要依赖于端到端的像素级修复范式。尽管这种方法在简单场景中取得了成功，但在需要精确局部控制和复杂多对象空间推理的组合编辑任务上仍然面临重大挑战。这一范式受到以下限制：1）规划与执行之间隐式的耦合；2）缺乏以物体级别为单位的操作精细度；3）依赖于无结构、像素中心化的建模方式。为了克服这些局限，我们提出了I2E（图像到编辑），一种新的“分解然后行动”范式，将图像编辑重新审视为在结构化环境中的可操作交互过程。I2E利用一个分解器将非结构化的图片转换成离散的、可以操控的对象层，并引入了具有物理感知能力的视觉-语言-动作代理来通过链式思维推理解析复杂的指令并将其转化为一系列原子行动。此外，我们还构建了I2E-Bench，这是一个专门设计用于多实例空间推理和高精度编辑任务的基准测试平台。 在I2E-Bench和其他多个公开基准上的实验结果表明，I2E在处理复杂组合指令、保持物理合理性以及保证多次编辑稳定性方面显著优于现有最先进的方法。
URL
https://arxiv.org/abs/2601.03741
