Abstract
Since the introduction of Vision Transformers, the landscape of many computer vision tasks (e.g., semantic segmentation), which has been overwhelmingly dominated by CNNs, recently has significantly revolutionized. However, the computational cost and memory requirement render these methods unsuitable on the mobile device, especially for the high-resolution per-pixel semantic segmentation task. In this paper, we introduce a new method squeeze-enhanced Axial TransFormer (SeaFormer) for mobile semantic segmentation. Specifically, we design a generic attention block characterized by the formulation of squeeze Axial and detail enhancement. It can be further used to create a family of backbone architectures with superior cost-effectiveness. Coupled with a light segmentation head, we achieve the best trade-off between segmentation accuracy and latency on the ARM-based mobile devices on the ADE20K and Cityscapes datasets. Critically, we beat both the mobile-friendly rivals and Transformer-based counterparts with better performance and lower latency without bells and whistles. Beyond semantic segmentation, we further apply the proposed SeaFormer architecture to image classification problem, demonstrating the potentials of serving as a versatile mobile-friendly backbone.
Abstract (translated)
自从视觉转换器引入以来,许多计算机视觉任务(例如语义分割)的景象已经显著地被卷积神经网络所支配,最近已经发生了重大的变革。然而,计算成本和精神计算要求使得这些方法在移动设备上不适合,特别是在高分辨率每个像素的语义分割任务上。在本文中,我们提出了一种新的方法,称为Squeeze-enhanced Axial Transformer(海洋 former),用于移动语义分割。具体来说,我们设计了一个通用的关注块,其特点在于squeeze Axial和细节增强的制备。它还可以进一步用于创建一个成本效益更高的骨架架构家族。与轻量级分割头结合,我们在ARM基座移动设备上的ADE20K和城市风景数据集上实现了分割准确性和延迟的最佳权衡。是至关重要的是,我们超越了移动友好竞争对手和基于Transformer的替代品,以更好的性能和更低的延迟表现优异,而无需添加 bells 和 whistles。除了语义分割,我们还将海洋 former架构应用于图像分类问题,展示了作为多功能移动友好骨架的潜力。
URL
https://arxiv.org/abs/2301.13156
