Abstract
High-resolution (HR) images are usually downscaled to low-resolution (LR) ones for better display and afterward upscaled back to the original size to recover details. Recent work in image rescaling formulates downscaling and upscaling as a unified task and learns a bijective mapping between HR and LR via invertible networks. However, in real-world applications (e.g., social media), most images are compressed for transmission. Lossy compression will lead to irreversible information loss on LR images, hence damaging the inverse upscaling procedure and degrading the reconstruction accuracy. In this paper, we propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware image rescaling. To tackle the distribution shift, we first develop an end-to-end asymmetric framework with two separate bijective mappings for high-quality and compressed LR images, respectively. Then, based on empirical analysis of this framework, we model the distribution of the lost information (including downscaling and compression) using isotropic Gaussian mixtures and propose the Enhanced Invertible Block to derive high-quality/compressed LR images in one forward pass. Besides, we design a set of losses to regularize the learned LR images and enhance the invertibility. Extensive experiments demonstrate the consistent improvements of SAIN across various image rescaling datasets in terms of both quantitative and qualitative evaluation under standard image compression formats (i.e., JPEG and WebP).
Abstract (translated)
高分辨率(HR)图像通常被向下拉伸到低分辨率(LR)图像以更好地显示,然后再向上拉伸回原始大小以恢复细节。最近的图像重排工作将下拉和上拉作为一项统一任务,并使用可逆网络学习HR和LR之间的互逆映射。然而,在实际应用(如社交媒体)中,大多数图像都被压缩以传输。lossy压缩会导致LR图像的不可逆信息损失,因此破坏逆拉伸过程并降低重构精度。在本文中,我们提出了一种自对称逆转网络(SAIN),以压缩 aware 的图像重排。为了应对分布迁移,我们首先开发了一个端到端的自对称框架,分别用于高质量的和压缩的LR图像。然后,基于该框架的实证分析,我们使用 isotropic 均匀混合物建模丢失信息分布(包括下拉和压缩)并提出了增强逆转块,在一次forward过程中生成高质量的/压缩的LR图像。此外,我们设计了一套损失来 regularize 学习到的LR图像,并增强逆转性。广泛的实验结果表明,SAIN在各种图像重排数据集上在量化和定性评估的标准图像压缩格式(如JPEG和WebP)中表现出一致性的改进。
URL
https://arxiv.org/abs/2303.02353
