Abstract
Facial parts swapping aims to selectively transfer regions of interest from the source image onto the target image while maintaining the rest of the target image unchanged. Most studies on face swapping designed specifically for full-face swapping, are either unable or significantly limited when it comes to swapping individual facial parts, which hinders fine-grained and customized character designs. However, designing such an approach specifically for facial parts swapping is challenged by a reasonable multiple reference feature fusion, which needs to be both efficient and effective. To overcome this challenge, FuseAnyPart is proposed to facilitate the seamless "fuse-any-part" customization of the face. In FuseAnyPart, facial parts from different people are assembled into a complete face in latent space within the Mask-based Fusion Module. Subsequently, the consolidated feature is dispatched to the Addition-based Injection Module for fusion within the UNet of the diffusion model to create novel characters. Extensive experiments qualitatively and quantitatively validate the superiority and robustness of FuseAnyPart. Source codes are available at this https URL.
Abstract (translated)
面部部分交换旨在将源图像中感兴趣的区域选择性地转移到目标图像上，同时保持目标图像的其余部分不变。大多数专门针对全脸交换的研究要么无法进行，要么在单独交换面部部位时受到显著限制，这阻碍了精细和定制化的角色设计。然而，为面部部位交换专门设计这样的方法面临着合理多参考特征融合的挑战，这种融合需要既高效又有效。为克服这一挑战，提出了FuseAnyPart来促进面部“任意部分融合”的无缝自定义。在FuseAnyPart中，不同人的面部部位被组装成基于掩码融合模块内的潜在空间中的完整面孔。随后，整合后的特征被发送到基于加法的注入模块，在扩散模型的UNet内进行融合以创造新的角色。广泛的实验定性和定量验证了FuseAnyPart的优势和鲁棒性。源代码可以在以下链接获取：[此处提供的https URL]。
URL
https://arxiv.org/abs/2410.22771
