Abstract
The StyleGAN family succeed in high-fidelity image generation and allow for flexible and plausible editing of generated images by manipulating the semantic-rich latent style space.However, projecting a real image into its latent space encounters an inherent trade-off between inversion quality and editability. Existing encoder-based or optimization-based StyleGAN inversion methods attempt to mitigate the trade-off but see limited performance. To fundamentally resolve this problem, we propose a novel two-phase framework by designating two separate networks to tackle editing and reconstruction respectively, instead of balancing the two. Specifically, in Phase I, a W-space-oriented StyleGAN inversion network is trained and used to perform image inversion and editing, which assures the editability but sacrifices reconstruction quality. In Phase II, a carefully designed rectifying network is utilized to rectify the inversion errors and perform ideal reconstruction. Experimental results show that our approach yields near-perfect reconstructions without sacrificing the editability, thus allowing accurate manipulation of real images. Further, we evaluate the performance of our rectifying network, and see great generalizability towards unseen manipulation types and out-of-domain images.
Abstract (translated)
风格GAN家族成功地生成高保真的图像,并通过操纵语义丰富的潜在风格空间,实现了灵活且可信的图像编辑。然而,将真实图像引入潜在空间则会遇到编辑质量与可编辑性之间的固有权衡。现有的编码器-based或优化-based的风格GAN逆过程方法试图减轻权衡,但表现有限。要根本解决这个问题,我们提出了一种 novel 的双向框架,指定两个独立的网络分别处理编辑和重建任务,而不是平衡两者。具体来说,在第I阶段,一个以W空间为中心的风格GAN逆网络被训练和用于图像逆与编辑,确保了可编辑性但牺牲了重建质量。在第II阶段,一个 carefully designed 纠正器网络被利用来纠正逆错误并实现理想的重建。实验结果显示,我们的方法产生几乎完美的重构,而无需牺牲可编辑性,从而允许对真实图像进行准确的操纵。此外,我们评估了我们的纠正器网络的性能,并看到了对未曾见过的操纵类型和跨域图像的极大泛化能力。
URL
https://arxiv.org/abs/2301.13402
