Abstract
Intelligent surveillance systems often handle perceptual tasks such as object detection, facial recognition, and emotion analysis independently, but they lack a unified, adaptive runtime scheduler that dynamically allocates computational resources based on contextual triggers. This limits their holistic understanding and efficiency on low-power edge devices. To address this, we present a real-time multi-modal vision framework that integrates object detection, owner-specific face recognition, and emotion detection into a unified pipeline deployed on a Raspberry Pi 5 edge platform. The core of our system is an adaptive scheduling mechanism that reduces computational load by 65\% compared to continuous processing by selectively activating modules such as, YOLOv8n for object detection, a custom FaceNet-based embedding system for facial recognition, and DeepFace's CNN for emotion classification. Experimental results demonstrate the system's efficacy, with the object detection module achieving an Average Precision (AP) of 0.861, facial recognition attaining 88\% accuracy, and emotion detection showing strong discriminatory power (AUC up to 0.97 for specific emotions), while operating at 5.6 frames per second. Our work demonstrates that context-aware scheduling is the key to unlocking complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving.
Abstract (translated)
智能监控系统通常独立处理如物体检测、面部识别和情绪分析等感知任务，但它们缺乏一个统一的自适应运行时调度器，该调度器可以根据上下文触发动态分配计算资源。这限制了其在低功耗边缘设备上的整体理解和效率。为解决这一问题，我们提出了一种实时多模态视觉框架，该框架将物体检测、基于个人的脸部识别和情绪检测整合到一个统一的管道中，并部署在Raspberry Pi 5边缘平台上。 我们系统的核心是一个自适应调度机制，通过选择性激活模块（例如用于对象检测的YOLOv8n、用于面部识别的定制FaceNet嵌入系统以及用于情绪分类的DeepFace卷积神经网络）来减少连续处理所需的计算负载达65%。实验结果显示，该系统的有效性得到了证明：物体检测模块实现了0.861的平均精度（AP），面部识别达到了88%的准确率，而情绪检测对于特定情感显示出强大的区分能力（AUC高达0.97）。同时，在每秒处理5.6帧的情况下，系统运行稳定。 我们的工作表明，基于上下文感知的调度是解锁复杂多模态AI的关键，并在成本效益高的边缘硬件上实现智能感知，使之更加普及且具有隐私保护功能。
URL
https://arxiv.org/abs/2601.11970
