Abstract
Autonomous driving (AD) scenarios are inherently complex and diverse, posing significant challenges for a single deep learning model to effectively cover all possible conditions, such as varying weather, traffic densities, and road types. Large Model (LM)-Driven Mixture of Experts (MoE) paradigm offers a promising solution, where LM serves as the backbone to extract latent features while MoE serves as the downstream head to dynamically select and aggregate specialized experts to adapt to different scenarios. However, routing and aggregating in MoE face intrinsic challenges, including imprecise expert selection due to flawed routing strategy and inefficient expert aggregation leading to suboptimal prediction. To address these issues, we propose a statistic-augmented, decoupled MoE }outing and Aggregating Mechanism (MoE-RAM) driven by LM. Specifically, on the one hand, MoE-RAM enhances expert routing by incorporating statistical retrieval mechanism to match LM-extracted latent features with cached prototypical features of the most relevant experts; on the other hand, MoE-RAM adaptively reweights experts' outputs in fusion by measuring statistical distances of experts' instant features against LM-extracted latent features. Benefiting from the synergy of the statistic-augmented MoE's routing and aggregating, MoE-RAM ultimately improves the prediction performance. We take the AD semantic segmentation task as an example to assess the proposed MoE-RAM. Extensive experiments on AD datasets demonstrate the superiority of MoE-RAM compared to other MoE baselines and conventional single-model approaches.
Abstract (translated)
自动驾驶（AD）场景本质上是复杂且多样的，这给单一的深度学习模型有效覆盖所有可能条件带来了巨大挑战，这些条件包括不同的天气状况、交通密度和道路类型。大型模型（LM）驱动的专家混合体（MoE）范式提供了一个有希望的解决方案，在这种方案中，LM作为主干来提取潜在特征，而MoE则作为下游模块动态选择并聚合专门化的专家以适应不同场景。然而，MoE中的路由和聚集面对着固有的挑战，包括由于路由策略不当导致的不精确的专家选择以及低效的专家聚合从而导致次优预测的问题。 为了应对这些问题，我们提出了一种由LM驱动、统计增强且解耦的MoE路由与聚合机制（MoE-RAM）。具体来说，在一方面，通过引入统计检索机制来匹配从LM提取出的潜在特征和缓存的相关专家原型特征，MoE-RAM增强了专家选择过程；另一方面，MoE-RAM在融合过程中根据即时特征与从LM提取的潜在特征之间的统计距离自适应地重新加权各个专家的输出。得益于这种统计增强型MoE路由和聚集机制的协同作用，MoE-RAM最终提高了预测性能。 我们选取了AD语义分割任务作为示例来评估所提出的MoE-RAM。在一系列AD数据集上的广泛实验表明，相较于其他MoE基准模型及传统的单模型方法，MoE-RAM表现出显著优势。
URL
https://arxiv.org/abs/2512.06664
