Abstract
End-to-end autonomous driving (E2EAD) systems, which learn to predict future trajectories directly from sensor data, are fundamentally challenged by the inherent spatio-temporal imbalance of trajectory data. This imbalance creates a significant optimization burden, causing models to learn spurious correlations instead of causal inference, while also prioritizing uncertain, distant predictions, thereby compromising immediate safety. To address these issues, we propose ResAD, a novel Normalized Residual Trajectory Modeling framework. Instead of predicting the future trajectory directly, our approach reframes the learning task to predict the residual deviation from a deterministic inertial reference. The inertial reference serves as a counterfactual, forcing the model to move beyond simple pattern recognition and instead identify the underlying causal factors (e.g., traffic rules, obstacles) that necessitate deviations from a default, inertially-guided path. To deal with the optimization imbalance caused by uncertain, long-term horizons, ResAD further incorporates Point-wise Normalization of the predicted residual. It re-weights the optimization objective, preventing large-magnitude errors associated with distant, uncertain waypoints from dominating the learning signal. Extensive experiments validate the effectiveness of our framework. On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy with only two denoising steps, demonstrating that our approach significantly simplifies the learning task and improves model performance. The code will be released to facilitate further research.
Abstract (translated)
端到端自主驾驶（E2EAD）系统直接从传感器数据学习预测未来轨迹，但这种方法由于轨迹数据固有的时空不平衡而面临根本性挑战。这种不平衡导致了显著的优化负担，使模型学会错误的相关性而不是因果推理，并优先考虑不确定和遥远的预测，从而损害即时安全性。为解决这些问题，我们提出了ResAD，一种新颖的归一化残差轨迹建模框架。 不同于直接预测未来的轨迹，我们的方法重新定义学习任务为预测从确定性的惯性参考点到实际路径的偏差。这种惯性参考作为一个反事实情况，迫使模型超越简单的模式识别，并识别那些需要偏离默认的惯性引导路径的根本原因（例如交通规则、障碍物）。为了处理由不确定和长期视界导致的优化不平衡问题，ResAD进一步采用逐点归一化预测残差的方法。这种方法重新加权了优化目标，防止距离较远且不确定性较高的路标所带来的大误差主导学习信号。 广泛的实验验证了我们框架的有效性。在NAVSIM基准测试中，使用一个仅需两次去噪步骤的简单扩散策略，ResAD实现了88.6%的PDMS（轨迹匹配准确率）的最佳性能，这表明我们的方法极大地简化了学习任务并提高了模型表现。我们将发布代码以促进进一步的研究。
URL
https://arxiv.org/abs/2510.08562
