Abstract
Vision backbone networks play a central role in modern computer vision. Enhancing their efficiency directly benefits a wide range of downstream applications. To measure efficiency, many publications rely on MACs (Multiply Accumulate operations) as a predictor of execution time. In this paper, we experimentally demonstrate the shortcomings of such a metric, especially in the context of edge devices. By contrasting the MAC count and execution time of common architectural design elements, we identify key factors for efficient execution and provide insights to optimize backbone design. Based on these insights, we present LowFormer, a novel vision backbone family. LowFormer features a streamlined macro and micro design that includes Lowtention, a lightweight alternative to Multi-Head Self-Attention. Lowtention not only proves more efficient, but also enables superior results on ImageNet. Additionally, we present an edge GPU version of LowFormer, that can further improve upon its baseline's speed on edge GPU and desktop GPU. We demonstrate LowFormer's wide applicability by evaluating it on smaller image classification datasets, as well as adapting it to several downstream tasks, such as object detection, semantic segmentation, image retrieval, and visual object tracking. LowFormer models consistently achieve remarkable speed-ups across various hardware platforms compared to recent state-of-the-art backbones. Code and models are available at this https URL.
Abstract (translated)
视觉骨干网络在现代计算机视觉中扮演着核心角色。提升其效率可直接惠及众多下游应用。为衡量效率，许多出版物依赖乘加运算次数（MACs）作为执行时间的预测指标。本文通过实验证明该指标存在缺陷，尤其在边缘设备场景下。通过对比常见架构设计元素的MACs数量与执行时间，我们识别出高效执行的关键因素，并为骨干网络设计优化提供见解。基于这些见解，我们提出LowFormer——一种新型视觉骨干网络家族。LowFormer采用精简的宏观与微观设计，包含Lowtention（多头自注意力机制的轻量化替代方案）。Lowtention不仅效率更高，还在ImageNet上取得更优结果。此外，我们推出了LowFormer的边缘GPU版本，可在边缘GPU和桌面GPU上进一步超越基线速度。通过在较小图像分类数据集上的评估，以及将其适配到目标检测、语义分割、图像检索、视觉目标跟踪等多个下游任务，我们验证了LowFormer的广泛适用性。与近期最先进骨干网络相比，LowFormer模型在各类硬件平台上均实现了显著加速。代码与模型详见链接。
URL
https://arxiv.org/abs/2603.26551
