Abstract
Practical video analytics systems that are deployed in bandwidth constrained environments like autonomous vehicles perform computer vision tasks such as face detection and recognition. In an end-to-end face analytics system, inputs are first compressed using popular video codecs like HEVC and then passed onto modules that perform face detection, alignment, and recognition sequentially. Typically, the modules of these systems are evaluated independently using task-specific imbalanced datasets that can misconstrue performance estimates. In this paper, we perform a thorough end-to-end evaluation of a face analytics system using a driving-specific dataset, which enables meaningful interpretations. We demonstrate how independent task evaluations, dataset imbalances, and inconsistent annotations can lead to incorrect system performance estimates. We propose strategies to create balanced evaluation subsets of our dataset and to make its annotations consistent across multiple analytics tasks and scenarios. We then evaluate the end-to-end system performance sequentially to account for task interdependencies. Our experiments show that our approach provides consistent, accurate, and interpretable estimates of the system's performance which is critical for real-world applications.
Abstract (translated)
实时的视频分析系统在像自动驾驶这样的带宽受限环境中执行计算机视觉任务，如面部检测和识别。在端到端面部分析系统中，首先使用流行的视频编码格式（如HEVC）对输入进行压缩，然后传递给依次执行面部检测、对齐和识别的模块。通常，这些系统的模块使用特定任务的不平衡数据集进行独立评估，这可能导致性能估计的误解。在本文中，我们通过使用驾驶特定数据集进行了对端到端面部分析系统的深入评估，这使得有意义的结果。我们证明了独立任务评估、数据不平衡和注释不统一可能导致系统性能估计错误。我们提出了创建数据集平衡评估子集以及在不同分析和任务场景下使其注释保持一致的策略。然后，我们按顺序评估端到端系统的性能，以考虑任务依赖关系。我们的实验结果表明，我们的方法提供了一致、准确和可解释的系统性能估计，这对实时应用程序至关重要。
URL
https://arxiv.org/abs/2310.06945
