Abstract
This paper handles the problem of converting real pictures into traditional Chinese ink-wash paintings, i.e., Chinese ink-wash painting style transfer. Though this problem could be realized by a wide range of image-to-image translation models, a notable issue with all these methods is that the original image content details could be easily erased or corrupted due to transfer of ink-wash style elements. To solve or ameliorate this issue, we propose to incorporate saliency detection into the unpaired image-to-image translation framework to regularize content information of the generated paintings. The saliency map is utilized for content regularization from two aspects, both explicitly and implicitly: (\romannumeral1) we propose saliency IOU (SIOU) loss to explicitly regularize saliency consistency before and after stylization; (\romannumeral2) we propose saliency adaptive normalization (SANorm) which implicitly enhances content integrity of the generated paintings by injecting saliency information to the generator network to guide painting generation. Besides, we also propose saliency attended discriminator network which harnesses saliency mask to focus generative adversarial attention onto salient image regions, it contributes to producing finer ink-wash stylization effect for salient objects of images. Qualitative and quantitative experiments consistently demonstrate superiority of our model over related advanced methods for Chinese ink-wash painting style transfer.
Abstract (translated)
本文处理将真实图像转换成传统中国水墨画的问题，即水墨画风格的图像-to-图像转换问题。尽管这个问题可以通过广泛的图像到图像转换模型来解决，但所有这些方法的一个显着的问题是，由于墨水风格元素的转移，原始图像内容细节很容易被轻松地擦除或损坏。为解决这个问题或减轻这个问题，我们提出将局部可用性检测引入无配对图像到图像转换框架中，以规范化生成的绘画的内容信息。局部可用性图用于从两个方面进行内容规范化：（罗马数字1）我们提出局部可用性IOU（SIOU）损失，以明确规范化和 stylization 之前的和之后的 saliency 一致性；（罗马数字2）我们提出局部可用性自适应规范化（SANorm），它通过向生成网络注入 saliency 信息来增强生成的绘画的内容完整性，从而指导绘画生成。此外，我们还提出了 saliency 注意的判别器网络，它利用 saliency 掩码将生成对抗注意力聚焦于显眼的图像区域，有助于产生更清晰的墨水画风格效果。定性和定量的实验结果都一致地证明了我们的模型在相关先进方法中的优越性。
URL
https://arxiv.org/abs/2404.15743
