Abstract
We propose ZeST, a method for zero-shot material transfer to an object in the input image given a material exemplar image. ZeST leverages existing diffusion adapters to extract implicit material representation from the exemplar image. This representation is used to transfer the material using pre-trained inpainting diffusion model on the object in the input image using depth estimates as geometry cue and grayscale object shading as illumination cues. The method works on real images without any training resulting a zero-shot approach. Both qualitative and quantitative results on real and synthetic datasets demonstrate that ZeST outputs photorealistic images with transferred materials. We also show the application of ZeST to perform multiple edits and robust material assignment under different illuminations. Project Page: this https URL
Abstract (translated)
我们提出了ZeST，一种在给定材料示例图像的输入图像中实现零散材料传输的方法。ZeST利用现有的扩散适配器从示例图像中提取隐含材料表示。这个表示用于在输入图像中的物体上使用预训练的修复扩散模型进行材料转移。该方法对真实图像进行处理，没有任何训练，实现零散传输。在真实和合成数据集上，ZeST生成的图像具有转移的材料。我们还展示了ZeST在不同的光照条件下进行多个编辑和鲁棒材料分配的应用。项目页面：https:// this URL
URL
https://arxiv.org/abs/2404.06425
