Abstract
Incorporating diffusion models in the image compression domain has the potential to produce realistic and detailed reconstructions, especially at extremely low bitrates. Previous methods focus on using diffusion models as expressive decoders robust to quantization errors in the conditioning signals, yet achieving competitive results in this manner requires costly training of the diffusion model and long inference times due to the iterative generative process. In this work we formulate the removal of quantization error as a denoising task, using diffusion to recover lost information in the transmitted image latent. Our approach allows us to perform less than 10\% of the full diffusion generative process and requires no architectural changes to the diffusion model, enabling the use of foundation models as a strong prior without additional fine tuning of the backbone. Our proposed codec outperforms previous methods in quantitative realism metrics, and we verify that our reconstructions are qualitatively preferred by end users, even when other methods use twice the bitrate.
Abstract (translated)
将扩散模型融入图像压缩领域，有望产生真实和详细的重构，尤其是在极低比特率的情况下。以前的方法侧重于使用扩散模型作为具有条件信号量化误差稳健的表达编码器，然而以这种方式实现竞争力的结果需要对扩散模型进行昂贵的训练，并由于递归生成过程，导致推理时间较长。在这项工作中，我们将量化误差消除视为去噪任务，利用扩散来恢复在传输图像潜在中丢失的信息。我们的方法允许我们执行不到10%的完整扩散生成过程，并且不需要对扩散模型进行架构更改，使得基础模型可以作为强大的先验，无需额外对骨干模型进行微调。我们提出的编码在量化现实指标上优于以前的方法，而且我们验证，即使其他方法使用两倍的比特率，我们的重构仍然具有用户满意的质量。
URL
https://arxiv.org/abs/2404.08580
