Abstract
The deployment of transformer-based models on resource-constrained edge devices represents a critical challenge in enabling real-time artificial intelligence applications. This comprehensive survey examines lightweight transformer architectures specifically designed for edge deployment, analyzing recent advances in model compression, quantization, pruning, and knowledge distillation techniques. We systematically review prominent lightweight variants including MobileBERT, TinyBERT, DistilBERT, EfficientFormer, EdgeFormer, and MobileViT, providing detailed performance benchmarks on standard datasets such as GLUE, SQuAD, ImageNet-1K, and COCO. Our analysis encompasses current industry adoption patterns across major hardware platforms (NVIDIA Jetson, Qualcomm Snapdragon, Apple Neural Engine, ARM architectures), deployment frameworks (TensorFlow Lite, ONNX Runtime, PyTorch Mobile, CoreML), and optimization strategies. Experimental results demonstrate that modern lightweight transformers can achieve 75-96% of full-model accuracy while reducing model size by 4-10x and inference latency by 3-9x, enabling deployment on devices with as little as 2-5W power consumption. We identify sparse attention mechanisms, mixed-precision quantization (INT8/FP16), and hardware-aware neural architecture search as the most effective optimization strategies. Novel findings include memory-bandwidth bottleneck analysis revealing 15-40M parameter models achieve optimal hardware utilization (60-75% efficiency), quantization sweet spots for different model types, and comprehensive energy efficiency profiling across edge platforms. We establish real-time performance boundaries and provide a practical 6-step deployment pipeline achieving 8-12x size reduction with less than 2% accuracy degradation.
Abstract (translated)
在资源受限的边缘设备上部署基于Transformer的模型，对于实现实时人工智能应用构成了一个关键挑战。这篇全面的综述研究了专门为边缘部署设计的轻量级Transformer架构，并分析了最近关于模型压缩、量化、剪枝和知识蒸馏技术的进步。我们系统地回顾了一些著名的轻量级变体，包括MobileBERT、TinyBERT、DistilBERT、EfficientFormer、EdgeFormer和MobileViT，并在GLUE、SQuAD、ImageNet-1K和COCO等标准数据集上提供了详细的性能基准测试。我们的分析涵盖了主要硬件平台（如NVIDIA Jetson、Qualcomm Snapdragon、Apple Neural Engine及ARM架构）、部署框架（TensorFlow Lite、ONNX Runtime、PyTorch Mobile及CoreML）以及优化策略的当前行业采纳模式。 实验结果表明，现代轻量级Transformer可以在保持75%-96%全模型精度的同时，将模型大小减少4-10倍，推理延迟降低3-9倍，从而能够在低至2-5W功耗的设备上部署。我们确定了稀疏注意机制、混合精度量化（INT8/FP16）和硬件感知神经架构搜索作为最有效的优化策略。 新颖的研究发现包括对内存带宽瓶颈分析，显示具有15-40M参数的模型能够实现最佳的硬件利用效率（60%-75%），针对不同类型模型的最佳量化点，以及跨边缘平台的全面能效配置文件。我们建立了实时性能边界，并提供了一个实用的六步部署管道，在减少8-12倍大小的同时保证了不到2%的精度退化。
URL
https://arxiv.org/abs/2601.03290
