Abstract
We introduce PhotoDoodle, a novel image editing framework designed to facilitate photo doodling by enabling artists to overlay decorative elements onto photographs. Photo doodling is challenging because the inserted elements must appear seamlessly integrated with the background, requiring realistic blending, perspective alignment, and contextual coherence. Additionally, the background must be preserved without distortion, and the artist's unique style must be captured efficiently from limited training data. These requirements are not addressed by previous methods that primarily focus on global style transfer or regional inpainting. The proposed method, PhotoDoodle, employs a two-stage training strategy. Initially, we train a general-purpose image editing model, OmniEditor, using large-scale data. Subsequently, we fine-tune this model with EditLoRA using a small, artist-curated dataset of before-and-after image pairs to capture distinct editing styles and techniques. To enhance consistency in the generated results, we introduce a positional encoding reuse mechanism. Additionally, we release a PhotoDoodle dataset featuring six high-quality styles. Extensive experiments demonstrate the advanced performance and robustness of our method in customized image editing, opening new possibilities for artistic creation.
Abstract (translated)
我们介绍了PhotoDoodle，这是一个新颖的图像编辑框架，旨在通过允许艺术家将装饰性元素叠加到照片上来促进照片涂鸦。然而，由于插入的元素必须与背景无缝融合，这使得照片涂鸦具有挑战性，需要实现逼真的混合、透视对齐和语境一致性。此外，还需要保持背景不被扭曲，并且要高效地捕捉艺术家独特的风格，即使训练数据量有限也是如此。这些需求并未被以往主要关注全局样式转换或局部修复的方法所解决。 我们的方法PhotoDoodle采用了一个两阶段的训练策略。首先，我们使用大规模的数据集来训练一个通用目的图像编辑模型OmniEditor。然后，我们通过EditLoRA微调该模型，并使用艺术家精心挑选的小规模“前”、“后”图像对数据集来捕捉独特的编辑风格和技巧。为了提高生成结果的一致性，我们引入了一种位置编码重用机制。 此外，我们发布了包含六种高质量样式的PhotoDoodle数据集。广泛的实验表明了我们的方法在定制化图像编辑中的高级性能和鲁棒性，为艺术创作开辟了新的可能性。
URL
https://arxiv.org/abs/2502.14397
