Abstract
Feature upsampling is a key operation in a number of modern convolutional network architectures, e.g. feature pyramids. Its design is critical for dense prediction tasks such as object detection and semantic/instance segmentation. In this work, we propose Content-Aware ReAssembly of FEatures (CARAFE), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE has several appealing properties: (1) Large field of view. Unlike previous works (e.g. bilinear interpolation) that only exploit sub-pixel neighborhood, CARAFE can aggregate contextual information within a large receptive field. (2) Content-aware handling. Instead of using a fixed kernel for all samples (e.g. deconvolution), CARAFE enables instance specific content-aware handling, which generates adaptive kernels on-the-fly. (3) Lightweight and fast to compute. CARAFE introduces little computational overhead and can be readily integrated into modern network architectures. We conduct comprehensive evaluations on standard benchmarks in object detection, instance/semantic segmentation and inpainting. CARAFE shows consistent and substantial gains across all the tasks (1.2%, 1.3%, 1.8%, 1.1db respectively) with negligible computational overhead. It has great potential to serve as a strong building block for future research.
Abstract (translated)
特征向上采样是许多现代卷积网络结构（如特征金字塔）中的关键操作。它的设计对于密集的预测任务（如对象检测和语义/实例分割）至关重要。在这项工作中，我们提出了内容感知功能重组（carafe），一个通用的、轻量级的、高效的操作员来实现这一目标。玻璃瓶有几个吸引人的特性：（1）视野开阔。不像以前的作品（例如双线性插值）只利用亚像素邻域，Carafe可以在一个大的接受域内聚集上下文信息。（2）内容感知处理。Carafe不需要对所有样本使用固定的内核（例如反褶积），而是支持特定于实例的内容感知处理，从而在运行中生成自适应内核。（3）重量轻，计算速度快。Carafe引入的计算开销很少，可以很容易地集成到现代网络架构中。我们对目标检测、实例/语义分割和修复中的标准基准进行了全面评估。Carafe显示了所有任务（分别为1.2%、1.3%、1.8%、1.1db）的一致性和实质性收益，计算开销可忽略不计。它具有巨大的潜力，可以作为未来研究的有力基石。
URL
https://arxiv.org/abs/1905.02188
