Abstract
Recent large vision models (e.g., SAM) enjoy great potential to facilitate intelligent perception with high accuracy. Yet, the resource constraints in the IoT environment tend to limit such large vision models to be locally deployed, incurring considerable inference latency thereby making it difficult to support real-time applications, such as autonomous driving and robotics. Edge-cloud collaboration with large-small model co-inference offers a promising approach to achieving high inference accuracy and low latency. However, existing edge-cloud collaboration methods are tightly coupled with the model architecture and cannot adapt to the dynamic data drifts in heterogeneous IoT environments. To address the issues, we propose LAECIPS, a new edge-cloud collaboration framework. In LAECIPS, both the large vision model on the cloud and the lightweight model on the edge are plug-and-play. We design an edge-cloud collaboration strategy based on hard input mining, optimized for both high accuracy and low latency. We propose to update the edge model and its collaboration strategy with the cloud under the supervision of the large vision model, so as to adapt to the dynamic IoT data streams. Theoretical analysis of LAECIPS proves its feasibility. Experiments conducted in a robotic semantic segmentation system using real-world datasets show that LAECIPS outperforms its state-of-the-art competitors in accuracy, latency, and communication overhead while having better adaptability to dynamic environments.
Abstract (translated)
近年来，大型视觉模型（例如，SAM）具有很大潜力，可以通过高精度智能感知促进实时应用。然而，物联网环境中的资源限制往往限制了大型视觉模型在本地部署，从而导致相当长的推理延迟，使得支持实时应用（如自动驾驶和机器人）变得困难。边缘云协同大型小模型推理提供了一种实现高推理准确性和低延迟的有前途的方法。然而，现有的边缘云协同方法紧密耦合于模型架构，无法适应异构物联网环境中的动态数据漂移。为解决这个问题，我们提出了LAECIPS，一种新的边缘云协同框架。在LAECIPS中，云上的大视觉模型和边缘上的轻量级模型都是插件和可用的。我们基于硬输入挖掘设计了一种边缘云协同策略，既具有高准确度又具有低延迟。我们建议在大型视觉模型的监督下更新边缘模型及其协同策略，以便适应动态的物联网数据流。LAECIPS的理论分析证明了其可行性。使用真实世界数据集的机器人语义分割系统进行的实验表明，LAECIPS在准确性、延迟和通信开销方面都优于其最先进的竞争对手，同时具有更好的适应动态环境的能力。
URL
https://arxiv.org/abs/2404.10498
