Paper Reading AI Learner

Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change

2024-03-01 11:03:17
Ruichen Xu, Xiao Liu, Jinming Xu, Yuan Lin

Abstract

Autonomous lane-change, a key feature of advanced driver-assistance systems, can enhance traffic efficiency and reduce the incidence of accidents. However, safe driving of autonomous vehicles remains challenging in complex environments. How to perform safe and appropriate lane change is a popular topic of research in the field of autonomous driving. Currently, few papers consider the safety of reinforcement learning in autonomous lane-change scenarios. We introduce safe hybrid-action reinforcement learning into discretionary lane change for the first time and propose Parameterized Soft Actor-Critic with PID Lagrangian (PASAC-PIDLag) algorithm. Furthermore, we conduct a comparative analysis of the Parameterized Soft Actor-Critic (PASAC), which is an unsafe version of PASAC-PIDLag. Both algorithms are employed to train the lane-change strategy of autonomous vehicles to output discrete lane-change decision and longitudinal vehicle acceleration. Our simulation results indicate that at a traffic density of 15 vehicles per kilometer (15 veh/km), the PASAC-PIDLag algorithm exhibits superior safety with a collision rate of 0%, outperforming the PASAC algorithm, which has a collision rate of 1%. The outcomes of the generalization assessments reveal that at low traffic density levels, both the PASAC-PIDLag and PASAC algorithms are proficient in attaining a 0% collision rate. Under conditions of high traffic flow density, the PASAC-PIDLag algorithm surpasses PASAC in terms of both safety and optimality.

Abstract (translated)

自动驾驶中的自动换道,是高级驾驶辅助系统的一个关键功能,可以提高交通效率并减少事故发生率。然而,在复杂环境中安全驾驶自动驾驶车辆仍然具有挑战性。如何安全地进行自动车道换道是自动驾驶领域研究的热门话题。目前,很少有论文考虑自动驾驶换道场景中的强化学习安全性。我们引入了安全混合动作强化学习方法对自主车道换道进行了首次提出,并提出了参数化软actor-critic with PID Lagrangian (PASAC-PIDLag)算法。此外,我们对参数化软actor-critic (PASAC)和不安全的PASAC-PIDLag算法进行了比较分析。两种算法都被用来自动车辆车道换道策略的训练,以输出离散车道换道决策和纵向车辆加速度。我们的仿真结果表明,在每公里15辆车的交通密度(15 veh/km)下,PASAC-PIDLag算法具有比PASAC算法更优越的安全性,其碰撞率为0%,优于具有碰撞率为1%的PASAC算法。扩展评估的结果表明,在较低的交通密度水平下,PASAC-PIDLag和PASAC算法都具有实现0%碰撞率的能力。在高度交通流量密度下,PASAC-PIDLag算法在安全性和最优性方面都超过了PASAC。

URL

https://arxiv.org/abs/2403.00446

PDF

https://arxiv.org/pdf/2403.00446.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot