Abstract
Recent advancements in diffusion-based generative image editing have sparked a profound revolution, reshaping the landscape of image outpainting and inpainting tasks. Despite these strides, the field grapples with inherent challenges, including: i) inferior quality; ii) poor consistency; iii) insufficient instrcution adherence; iv) suboptimal generation efficiency. To address these obstacles, we present ByteEdit, an innovative feedback learning framework meticulously designed to Boost, Comply, and Accelerate Generative Image Editing tasks. ByteEdit seamlessly integrates image reward models dedicated to enhancing aesthetics and image-text alignment, while also introducing a dense, pixel-level reward model tailored to foster coherence in the output. Furthermore, we propose a pioneering adversarial and progressive feedback learning strategy to expedite the model's inference speed. Through extensive large-scale user evaluations, we demonstrate that ByteEdit surpasses leading generative image editing products, including Adobe, Canva, and MeiTu, in both generation quality and consistency. ByteEdit-Outpainting exhibits a remarkable enhancement of 388% and 135% in quality and consistency, respectively, when compared to the baseline model. Experiments also verfied that our acceleration models maintains excellent performance results in terms of quality and consistency.
Abstract (translated)
近年来，基于扩散的生成图像编辑的进步引发了一场深刻的变化，重新塑造了图像修复和去修复任务的格局。尽管取得了这些进步，该领域仍面临固有挑战，包括：i）低质量；ii） poor consistency；iii） insufficient instruction adherence；iv） suboptimal generation efficiency。为了应对这些障碍，我们提出了ByteEdit，一种专门设计的反馈学习框架，旨在提高、符合和加速生成图像编辑任务。ByteEdit无缝地将专为增强美观和图像文本对齐的图像奖励模型集成在一起，同时引入了一个密集的像素级别奖励模型，以促进输出的一致性。此外，我们提出了一个里程碑式的对抗和 progressive feedback learning 策略，以加速模型的推理速度。通过大量的大型用户评估，我们证明了ByteEdit在生成质量和一致性方面超过了包括Adobe、Canva和MeiTu在内的领先生成图像编辑产品。ByteEdit-Outpainting在质量和一致性方面分别显示了388%和135%的显著增强。实验还验证了我们的加速模型在质量和一致性方面的优异表现。
URL
https://arxiv.org/abs/2404.04860
