Abstract
Colorizing grayscale images offers an engaging visual experience. Existing automatic colorization methods often fail to generate satisfactory results due to incorrect semantic colors and unsaturated colors. In this work, we propose an automatic colorization pipeline to overcome these challenges. We leverage the extraordinary generative ability of the diffusion prior to synthesize color with plausible semantics. To overcome the artifacts introduced by the diffusion prior, we apply the luminance conditional guidance. Moreover, we adopt multimodal high-level semantic priors to help the model understand the image content and deliver saturated colors. Besides, a luminance-aware decoder is designed to restore details and enhance overall visual quality. The proposed pipeline synthesizes saturated colors while maintaining plausible semantics. Experiments indicate that our proposed method considers both diversity and fidelity, surpassing previous methods in terms of perceptual realism and gain most human preference.
Abstract (translated)
给灰度图像上色提供了一个有趣的视觉体验。然而，现有的自动上色方法由于错误的语义颜色和不饱和的颜色而往往无法产生令人满意的结果。在这项工作中，我们提出了一个自动上色管道来克服这些挑战。我们利用扩散前的无与伦比的生成能力来合成具有合理语义的颜色。为了克服扩散前的伪影，我们应用了亮度条件指导。此外，我们还采用了多模态高级语义 prior 来帮助模型理解图像内容，并呈现饱和的颜色。此外，还设计了一个亮度感知解码器，以恢复细节并提高整体视觉质量。我们提出的管道在保持合理语义的同时合成饱和的颜色。实验结果表明，与以前的方法相比，我们的方法考虑了多样性和准确性，在感知真实感和人偏好方面超过了以前的方法。
URL
https://arxiv.org/abs/2404.16678
