Abstract
In Machine Learning (ML)-enabled autonomous systems (MLASs), it is essential to identify the hazard boundary of ML Components (MLCs) in the MLAS under analysis. Given that such boundary captures the conditions in terms of MLC behavior and system context that can lead to hazards, it can then be used to, for example, build a safety monitor that can take any predefined fallback mechanisms at runtime when reaching the hazard boundary. However, determining such hazard boundary for an ML component is challenging. This is due to the space combining system contexts (i.e., scenarios) and MLC behaviors (i.e., inputs and outputs) being far too large for exhaustive exploration and even to handle using conventional metaheuristics, such as genetic algorithms. Additionally, the high computational cost of simulations required to determine any MLAS safety violations makes the problem even more challenging. Furthermore, it is unrealistic to consider a region in the problem space deterministically safe or unsafe due to the uncontrollable parameters in simulations and the non-linear behaviors of ML models (e.g., deep neural networks) in the MLAS under analysis. To address the challenges, we propose MLCSHE (ML Component Safety Hazard Envelope), a novel method based on a Cooperative Co-Evolutionary Algorithm (CCEA), which aims to tackle a high-dimensional problem by decomposing it into two lower-dimensional search subproblems. Moreover, we take a probabilistic view of safe and unsafe regions and define a novel fitness function to measure the distance from the probabilistic hazard boundary and thus drive the search effectively. We evaluate the effectiveness and efficiency of MLCSHE on a complex Autonomous Vehicle (AV) case study. Our evaluation results show that MLCSHE is significantly more effective and efficient compared to a standard genetic algorithm and random search.
Abstract (translated)
在机器学习(ML)激活的自主系统(MLASs)中,必须确定分析中的MLAS中的ML组件(MLC)的危险边界。给定这种边界捕获了MLC行为和系统背景可能导致危险的情况,它可以被用来构建一个安全监控器,在运行时当达到危险边界时能够选择任何预定义的备用机制。然而,确定ML组件的危险边界是一项挑战。这是因为空间结合系统背景(即场景)和MLC行为(即输入和输出)的巨大数量,远远超过了进行 exhaustive 探索 和使用传统启发式算法(如遗传算法)的能力。此外,确定任何MLAS 安全性违反的模拟高计算成本使问题更加挑战。此外,考虑在问题空间中的确定性安全区域或不安全区域由于模拟中的不可控制参数和MLAS中ML模型(如深度神经网络)的非线性行为而不切实际。为了应对挑战,我们提出了MLCSHE(MLComponent Safety Hazard Envelope),这是一种基于合作协同进化算法(CCEA)的新方法,旨在解决高维度问题,将其分解为两个低维度搜索子问题。此外,我们采用概率观点看待安全和不安全区域,并定义一种新的 fitness 函数,以测量从概率危险边界的距离,从而有效地推动搜索。我们评估了MLCSHE在复杂的自主车辆(AV)案例研究中的效率和效果。我们的评估结果显示,与标准遗传算法和随机搜索相比,MLCSHE significantly more effective and efficient。
URL
https://arxiv.org/abs/2301.13807
