Abstract
Autonomous lane-change, a key feature of advanced driver-assistance systems, can enhance traffic efficiency and reduce the incidence of accidents. However, safe driving of autonomous vehicles remains challenging in complex environments. How to perform safe and appropriate lane change is a popular topic of research in the field of autonomous driving. Currently, few papers consider the safety of reinforcement learning in autonomous lane-change scenarios. We introduce safe hybrid-action reinforcement learning into discretionary lane change for the first time and propose Parameterized Soft Actor-Critic with PID Lagrangian (PASAC-PIDLag) algorithm. Furthermore, we conduct a comparative analysis of the Parameterized Soft Actor-Critic (PASAC), which is an unsafe version of PASAC-PIDLag. Both algorithms are employed to train the lane-change strategy of autonomous vehicles to output discrete lane-change decision and longitudinal vehicle acceleration. Our simulation results indicate that at a traffic density of 15 vehicles per kilometer (15 veh/km), the PASAC-PIDLag algorithm exhibits superior safety with a collision rate of 0%, outperforming the PASAC algorithm, which has a collision rate of 1%. The outcomes of the generalization assessments reveal that at low traffic density levels, both the PASAC-PIDLag and PASAC algorithms are proficient in attaining a 0% collision rate. Under conditions of high traffic flow density, the PASAC-PIDLag algorithm surpasses PASAC in terms of both safety and optimality.
Abstract (translated)
自动驾驶中的自动换道，是高级驾驶辅助系统的一个关键功能，可以提高交通效率并减少事故发生率。然而，在复杂环境中安全驾驶自动驾驶车辆仍然具有挑战性。如何安全地进行自动车道换道是自动驾驶领域研究的热门话题。目前，很少有论文考虑自动驾驶换道场景中的强化学习安全性。我们引入了安全混合动作强化学习方法对自主车道换道进行了首次提出，并提出了参数化软actor-critic with PID Lagrangian (PASAC-PIDLag)算法。此外，我们对参数化软actor-critic (PASAC)和不安全的PASAC-PIDLag算法进行了比较分析。两种算法都被用来自动车辆车道换道策略的训练，以输出离散车道换道决策和纵向车辆加速度。我们的仿真结果表明，在每公里15辆车的交通密度（15 veh/km）下，PASAC-PIDLag算法具有比PASAC算法更优越的安全性，其碰撞率为0%，优于具有碰撞率为1%的PASAC算法。扩展评估的结果表明，在较低的交通密度水平下，PASAC-PIDLag和PASAC算法都具有实现0%碰撞率的能力。在高度交通流量密度下，PASAC-PIDLag算法在安全性和最优性方面都超过了PASAC。
URL
https://arxiv.org/abs/2403.00446
