Abstract
Deep generative models have the capacity to render high fidelity images of content like human faces. Recently, there has been substantial progress in conditionally generating images with specific quantitative attributes, like the emotion conveyed by one's face. These methods typically require a user to explicitly quantify the desired intensity of a visual attribute. A limitation of this method is that many attributes, like how "angry" a human face looks, are difficult for a user to precisely quantify. However, a user would be able to reliably say which of two faces seems "angrier". Following this premise, we develop the $\textit{PrefGen}$ system, which allows users to control the relative attributes of generated images by presenting them with simple paired comparison queries of the form "do you prefer image $a$ or image $b$?" Using information from a sequence of query responses, we can estimate user preferences over a set of image attributes and perform preference-guided image editing and generation. Furthermore, to make preference localization feasible and efficient, we apply an active query selection strategy. We demonstrate the success of this approach using a StyleGAN2 generator on the task of human face editing. Additionally, we demonstrate how our approach can be combined with CLIP, allowing a user to edit the relative intensity of attributes specified by text prompts. Code at this https URL.
Abstract (translated)
深度生成模型能够生成类似于人类面部的图像内容，最近，在 conditional 生成具有特定量化属性的图像方面，取得了显著进展，例如，像人类面部传达的情感一样，这些方法通常要求用户明确地量化视觉属性的 desired 强度。这种方法的限制是，许多属性，例如人类面部看起来很“愤怒”的属性，很难精确量化。然而，用户能够可靠地地说，哪个面部似乎更“愤怒”。基于这一假设，我们开发了 $\textit{PrefGen}$ 系统，该系统允许用户通过简单的一对比较查询“do you prefer image $a$ or image $b$?” 来控制生成图像的相对属性。使用查询响应序列中的信息，我们可以估计用户对一组图像属性的偏好，并使用偏好指导的图像编辑和生成。此外，为了让偏好集中化可行和高效，我们应用了一种主动查询选择策略。我们使用 StyleGAN2 生成器演示了这种方法的成功，在人类面部编辑任务中。此外，我们还演示了如何将我们的方法和 CLIP 相结合，允许用户编辑由文本提示指定的属性相对强度。代码在此 https URL 上。
URL
https://arxiv.org/abs/2304.00185
