Abstract
Deep learning models for image compression often face practical limitations in hardware-constrained applications. Although these models achieve high-quality reconstructions, they are typically complex, heavyweight, and require substantial training data and computational resources. We propose a methodology to partially compress these networks by reducing the size of their encoders. Our approach uses a simplified knowledge distillation strategy to approximate the latent space of the original models with less data and shorter training, yielding lightweight encoders from heavyweight ones. We evaluate the resulting lightweight encoders across two different architectures on the image compression task. Experiments show that our method preserves reconstruction quality and statistical fidelity better than training lightweight encoders with the original loss, making it practical for resource-limited environments.
Abstract (translated)
深度学习模型在图像压缩方面的应用经常面临硬件受限条件下的实际限制。尽管这些模型能够实现高质量的重建，但它们通常复杂且占用大量资源，需要大量的训练数据和计算资源。我们提出了一种方法，通过减小这些网络编码器的大小来对其进行部分压缩。我们的方法使用简化的知识蒸馏策略，在较少的数据和较短的训练时间下近似原始模型的潜在空间，从而从重型编码器生成轻量级编码器。 我们在两种不同架构上对产生的轻量级编码器在图像压缩任务中的效果进行了评估。实验表明，与使用原始损失训练轻量级编码器相比，我们的方法更好地保持了重建质量和统计保真度，使其适用于资源受限的环境。
URL
https://arxiv.org/abs/2601.05639
