Abstract
While the performance of deep convolutional neural networks for image super-resolution (SR) has improved significantly, the rapid increase of memory and computation requirements hinders their deployment on resource-constrained devices. Quantized networks, especially binary neural networks (BNN) for SR have been proposed to significantly improve the model inference efficiency but suffer from large performance degradation. We observe the activation distribution of SR networks demonstrates very large pixel-to-pixel, channel-to-channel, and image-to-image variation, which is important for high performance SR but gets lost during binarization. To address the problem, we propose two effective methods, including the spatial re-scaling as well as channel-wise shifting and re-scaling, which augments binary convolutions by retaining more spatial and channel-wise information. Our proposed models, dubbed EBSR, demonstrate superior performance over prior art methods both quantitatively and qualitatively across different datasets and different model sizes. Specifically, for x4 SR on Set5 and Urban100, EBSRlight improves the PSNR by 0.31 dB and 0.28 dB compared to SRResNet-E2FIF, respectively, while EBSR outperforms EDSR-E2FIF by 0.29 dB and 0.32 dB PSNR, respectively.
Abstract (translated)
虽然深度学习卷积神经网络的图像超分辨率(SR)性能已经显著改善,但内存和计算要求的快速增加阻碍了它们在资源受限的设备上的部署。提议了量化网络,特别是SR中的二进制神经网络(BNN),以提高模型推理效率,但出现了显著的性能下降。我们观察了SR网络的激活分布,显示它们具有巨大的像素到像素、通道到通道和图像到图像变异,这对于高性能SR非常重要,但在二进制分类中迷失了。为了解决这个问题,我们提出了两个有效方法,包括空间重排和通道重排,这增加了二进制卷积核的数量,并通过保留更多的空间和情感通道信息来增强它们。我们提出的模型被称为EBSR,在不同数据集和模型大小上表现出了比现有方法更高的性能和质量。具体来说,对于Set5和Urban100中的x4SR,EBSRlight比SRResNet-E2FIF提高了0.31 dB的PSNR,而EBSR比EDSR-E2FIF提高了0.29 dB的PSNR。
URL
https://arxiv.org/abs/2303.12270
