Abstract
Image aesthetic enhancement aims to perceive aesthetic deficiencies in images and perform corresponding editing operations, which is highly challenging and requires the model to possess creativity and aesthetic perception capabilities. Although recent advancements in image editing models have significantly enhanced their controllability and flexibility, they struggle with enhancing image aesthetic. The primary challenges are twofold: first, following editing instructions with aesthetic perception is difficult, and second, there is a scarcity of "perfectly-paired" images that have consistent content but distinct aesthetic qualities. In this paper, we propose Dual-supervised Image Aesthetic Enhancement (DIAE), a diffusion-based generative model with multimodal aesthetic perception. First, DIAE incorporates Multimodal Aesthetic Perception (MAP) to convert the ambiguous aesthetic instruction into explicit guidance by (i) employing detailed, standardized aesthetic instructions across multiple aesthetic attributes, and (ii) utilizing multimodal control signals derived from text-image pairs that maintain consistency within the same aesthetic attribute. Second, to mitigate the lack of "perfectly-paired" images, we collect "imperfectly-paired" dataset called IIAEData, consisting of images with varying aesthetic qualities while sharing identical semantics. To better leverage the weak matching characteristics of IIAEData during training, a dual-branch supervision framework is also introduced for weakly supervised image aesthetic enhancement. Experimental results demonstrate that DIAE outperforms the baselines and obtains superior image aesthetic scores and image content consistency scores.
Abstract (translated)
图像美学增强的目标是识别图片中的审美缺陷，并执行相应的编辑操作，这是一项极具挑战性的任务，要求模型具备创造力和审美感知能力。尽管最近的图像编辑模型在可控性和灵活性方面取得了显著进展，但它们仍难以提升图像的美感。主要面临的两个挑战是：首先，根据具有审美意识的指令进行编辑很困难；其次，“完美配对”的图像数据（即内容一致而美学特征不同的图像）稀缺。为此，在本文中我们提出了基于扩散模型的多模态审美感知图像美学增强方法(Dual-supervised Image Aesthetic Enhancement, DIAE)。 DIAE 方法主要包括两个方面： 1. **Multi-modal Aesthetic Perception (MAP)**：这种方法将模糊的审美指令转化为具体的指导信号，通过使用详细的、标准化的跨多个审美属性的指令，并利用从文本-图像对中提取出的一致于同一美学属性中的多模态控制信号。 2. 为了缓解“完美配对”数据稀缺的问题，我们收集了一个名为IIAEData的数据集，该数据集中包含了具有不同美学品质但语义一致性的图片。同时，为更好地利用IIAEData的弱匹配特性进行训练，我们还引入了一种双分支监督框架来实现弱监督图像美学增强。 实验结果表明，DIAE 方法不仅超越了基准方法，还在图像美学评分和图像内容一致性评分上取得了更好的成绩。
URL
https://arxiv.org/abs/2603.11556
