Abstract
Generative adversarial network (GAN)-based image inpainting methods which utilize coarse-to-fine network with a contextual attention module (CAM) have shown remarkable performance. However, they require numerous computational resources such as convolution operations and network parameters due to two stacked generative networks, which results in a low speed. To address this problem, we propose a novel network structure called PEPSI: parallel extended-decoder path for semantic inpainting network, which aims at not only reducing hardware costs but also improving the inpainting performance. The PEPSI consists of a single shared encoding network and parallel decoding networks with coarse and inpainting paths. The coarse path generates a preliminary inpainting result to train the encoding network for prediction of features for the CAM. At the same time, the inpainting path results in higher inpainting quality with refined features reconstructed using the CAM. In addition, we propose a Diet-PEPSI which significantly reduces the network parameters while maintaining the performance. In the proposed method, we present a Diet-PEPSI unit (DPU) which effectively aggregates the global contextual information with a small number of parameters. Extensive experiments and comparisons with state-of-the-art image inpainting methods demonstrate that both PEPSI and Diet-PEPSI achieve significant improvements in qualitative scores and reduced computation cost.
Abstract (translated)
基于生成对抗网络（gan）的图像修复方法，利用粗到细的网络和上下文注意模块（cam），具有显著的性能。然而，由于两个叠加生成网络的存在，它们需要大量的计算资源，如卷积运算和网络参数，从而导致速度低下。针对这一问题，我们提出了一种新的网络结构pepsi：并行扩展译码器路径用于语义修复网络，它不仅降低了硬件成本，而且提高了修复性能。百事可乐由一个单一的共享编码网络和并行解码网络组成，具有粗糙和不可绘制的路径。粗路径生成一个初步的修复结果来训练编码网络，以预测凸轮的特征。同时，通过利用凸轮重构的精细特征，使补漆路径具有更高的补漆质量。此外，我们还提出了一种饮食百事可乐，它在保持性能的同时显著降低了网络参数。在该方法中，我们提出了一种饮食-百事可乐单位（DPU），它能有效地将全球背景信息与少量参数进行聚合。大量的实验和与最先进的图像修复方法的比较表明，百事可乐和减肥百事可乐在定性评分方面都取得了显著的改善，并降低了计算成本。
URL
https://arxiv.org/abs/1905.09010
