Abstract
Real-time multi-label video classification on embedded devices is constrained by limited compute and energy budgets. Yet, video streams exhibit structural properties such as label sparsity, temporal continuity, and label co-occurrence that can be leveraged for more efficient inference. We introduce Polymorph, a context-aware framework that activates a minimal set of lightweight Low Rank Adapters (LoRA) per frame. Each adapter specializes in a subset of classes derived from co-occurrence patterns and is implemented as a LoRA weight over a shared backbone. At runtime, Polymorph dynamically selects and composes only the adapters needed to cover the active labels, avoiding full-model switching and weight merging. This modular strategy improves scalability while reducing latency and energy overhead. Polymorph achieves 40% lower energy consumption and improves mAP by 9 points over strong baselines on the TAO dataset. Polymorph is open source at this https URL.
Abstract (translated)
嵌入式设备上的实时多标签视频分类受到计算能力和能量预算的限制。然而，视频流具有结构化特性，如标签稀疏性、时间连续性和标签共现等，这些特性可以被利用来实现更高效的推理过程。我们介绍了Polymorph，这是一个上下文感知框架，它在每一帧激活一组轻量级的低秩适配器（LoRA）的最小集合。每个适配器专门处理从共现模式中衍生出的一部分类别，并作为共享骨干网络上的一个LoRA权重来实现。运行时，Polymorph动态选择并组合仅需要覆盖活动标签的适配器，从而避免了全模型切换和权重合并的操作。这种模块化策略提高了可扩展性，同时减少了延迟和能量开销。 在TAO数据集上，与强大的基线相比，Polymorph实现了40%更低的能量消耗，并且将mAP（平均精度）提升了9个百分点。Polymorph的开源代码可在上述链接中获取。（原链接未能提供，请访问Alibaba Cloud的官方资源或相关论文以获取最新信息。）
URL
https://arxiv.org/abs/2507.14959
