Abstract
Existing learning-based stereo image codec adopt sophisticated transformation with simple entropy models derived from single image codecs to encode latent representations. However, those entropy models struggle to effectively capture the spatial-disparity characteristics inherent in stereo images, which leads to suboptimal rate-distortion results. In this paper, we propose a stereo image compression framework, named CAMSIC. CAMSIC independently transforms each image to latent representation and employs a powerful decoder-free Transformer entropy model to capture both spatial and disparity dependencies, by introducing a novel content-aware masked image modeling (MIM) technique. Our content-aware MIM facilitates efficient bidirectional interaction between prior information and estimated tokens, which naturally obviates the need for an extra Transformer decoder. Experiments show that our stereo image codec achieves state-of-the-art rate-distortion performance on two stereo image datasets Cityscapes and InStereo2K with fast encoding and decoding speed.
Abstract (translated)
现有的基于学习的立体图像编码器采用简单的源于单张图像编码器的熵模型对潜在表示进行编码。然而，这些熵模型很难有效地捕捉立体图像固有的空间差异特点，导致速率失真结果。在本文中，我们提出了一个名为CAMSIC的立体图像压缩框架。CAMSIC将每个图像独立转换为潜在表示，并采用一种强大的无解码器-无关Transformer熵模型来捕捉空间和差异依赖，通过引入一种新颖的内容感知掩膜图像建模（MIM）技术。我们内容感知MIM使前后信息之间实现高效的双向交互，自然地消除了需要额外Transformer解码器的需要。实验结果表明，我们的立体图像编码器在两个立体图像数据集Cityscapes和InStereo2K上实现了与最先进的速率失真性能相同的水平，具有快速的编码和解码速度。
URL
https://arxiv.org/abs/2403.08505
