Abstract
Recently, there have been tremendous efforts in developing lightweight Deep Neural Networks (DNNs) with satisfactory accuracy, which can enable the ubiquitous deployment of DNNs in edge devices. The core challenge of developing compact and efficient DNNs lies in how to balance the competing goals of achieving high accuracy and high efficiency. In this paper we propose two novel types of convolutions, dubbed \emph{Pixel Difference Convolution (PDC) and Binary PDC (Bi-PDC)} which enjoy the following benefits: capturing higher-order local differential information, computationally efficient, and able to be integrated with existing DNNs. With PDC and Bi-PDC, we further present two lightweight deep networks named \emph{Pixel Difference Networks (PiDiNet)} and \emph{Binary PiDiNet (Bi-PiDiNet)} respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition. Extensive experiments on popular datasets (BSDS500, ImageNet, LFW, YTF, \emph{etc.}) show that PiDiNet and Bi-PiDiNet achieve the best accuracy-efficiency trade-off. For edge detection, PiDiNet is the first network that can be trained without ImageNet, and can achieve the human-level performance on BSDS500 at 100 FPS and with $<$1M parameters. For object recognition, among existing Binary DNNs, Bi-PiDiNet achieves the best accuracy and a nearly $2\times$ reduction of computational cost on ResNet18. Code available at \href{this https URL}{this https URL}.
Abstract (translated)
近年来，在轻量化的Deep Neural Networks（DNNs）的开发中，已经取得了巨大的进展，这些DNNs具有令人满意的精度，可以实现分布式部署到边缘设备。开发轻量且高效的DNN的核心挑战在于如何平衡实现高准确性和高效率的竞争目标。在本文中，我们提出了两种新的卷积操作，称为\emph{Pixel Difference Convolution（PDC）和Binary PDC（Bi-PDC）}，它们具有以下优点：捕获更高阶局部差分信息，计算高效，并能够与现有的DNN集成。通过PDC和Bi-PDC，我们进一步提出了两个轻量化的深度网络，分别称为\emph{Pixel Difference Networks（PiDiNet）和Binary PiDiNet（Bi-PiDiNet）}，用于学习包括边缘检测和物体识别在内的视觉任务的准确且高效的表示。在流行的数据集（如BSDS500、ImageNet、LFW、YTF等）上进行的大量实验证明，PiDiNet和Bi-PiDiNet在准确性和效率之间实现了最佳平衡。对于边缘检测，PiDiNet是第一个在没有ImageNet训练的训练过程中达到人类水平性能的网络，同时可以在BSDS500上以100 FPS的速度实现1M参数以下的超人类性能。对于物体识别，在现有的二进制DNN中，Bi-PiDiNet实现了最佳准确性和与ResNet18相当2倍计算成本的降低。代码可在此处下载：\href{this <https://this URL>}。
URL
https://arxiv.org/abs/2402.00422
