Abstract
Recent advances in diffusion models have significantly improved image editing. However, challenges persist in handling geometric transformations, such as translation, rotation, and scaling, particularly in complex scenes. Existing approaches suffer from two main limitations: (1) difficulty in achieving accurate geometric editing of object translation, rotation, and scaling; (2) inadequate modeling of intricate lighting and shadow effects, leading to unrealistic results. To address these issues, we propose GeoEdit, a framework that leverages in-context generation through a diffusion transformer module, which integrates geometric transformations for precise object edits. Moreover, we introduce Effects-Sensitive Attention, which enhances the modeling of intricate lighting and shadow effects for improved realism. To further support training, we construct RS-Objects, a large-scale geometric editing dataset containing over 120,000 high-quality image pairs, enabling the model to learn precise geometric editing while generating realistic lighting and shadows. Extensive experiments on public benchmarks demonstrate that GeoEdit consistently outperforms state-of-the-art methods in terms of visual quality, geometric accuracy, and realism.
Abstract (translated)
最近在扩散模型方面取得的进展显著提高了图像编辑的质量。然而，在处理几何变换（如平移、旋转和缩放）时，特别是在复杂场景中，仍然面临挑战。现有方法存在两大主要局限性：(1) 难以实现对象平移、旋转和缩放的精确几何编辑；(2) 不充分地模拟复杂的光照和阴影效果，导致生成的结果不够真实。 为了解决这些问题，我们提出了GeoEdit框架，该框架利用扩散变换器模块通过上下文生成来整合几何变换，从而实现对物体的精准编辑。此外，我们引入了敏感效应注意力机制（Effects-Sensitive Attention），以增强复杂照明和阴影效果的建模，提高现实感。 为了进一步支持训练，我们构建了一个大规模的几何编辑数据集RS-Objects，包含超过120,000对高质量图像对，使模型能够学习精准的几何编辑并生成逼真的光照与阴影。在公共基准测试中的广泛实验表明，GeoEdit在视觉质量、几何精度和现实感方面始终优于当前最先进的方法。
URL
https://arxiv.org/abs/2602.08388
