Abstract
Traditional object detection methods face performance degradation challenges in complex scenarios such as low-light conditions and heavy occlusions due to a lack of high-level semantic understanding. To address this, this paper proposes an adaptive guidance-based semantic enhancement edge-cloud collaborative object detection method leveraging Multimodal Large Language Models (MLLM), achieving an effective balance between accuracy and efficiency. Specifically, the method first employs instruction fine-tuning to enable the MLLM to generate structured scene descriptions. It then designs an adaptive mapping mechanism that dynamically converts semantic information into parameter adjustment signals for edge detectors, achieving real-time semantic enhancement. Within an edge-cloud collaborative inference framework, the system automatically selects between invoking cloud-based semantic guidance or directly outputting edge detection results based on confidence scores. Experiments demonstrate that the proposed method effectively enhances detection accuracy and efficiency in complex scenes. Specifically, it can reduce latency by over 79% and computational cost by 70% in low-light and highly occluded scenes while maintaining accuracy.
Abstract (translated)
传统的目标检测方法在低光条件和严重遮挡等复杂场景中面临性能下降的挑战，因为这些方法缺乏高层次的语义理解。为了解决这个问题，本文提出了一种基于自适应指导的语义增强边缘-云端协同目标检测方法，该方法利用多模态大型语言模型（MLLM），实现了准确性和效率之间的有效平衡。 具体而言，该方法首先采用指令微调来使 MLLM 生成结构化的场景描述。然后设计了一个动态映射机制，可以将语义信息转换为边缘检测器的参数调整信号，从而实现实时语义增强。在一个边缘-云端协同推理框架内，系统根据置信度分数自动选择是在云端调用语义指导还是直接输出边缘检测结果。 实验表明，所提出的方法在复杂场景中有效地提高了检测准确性和效率。具体来说，在低光和高度遮挡的场景下，它可以将延迟降低79%以上，并减少计算成本达70%，同时保持高准确性。
URL
https://arxiv.org/abs/2509.19875
