Abstract
The accelerating pace of climate change introduces profound non-stationarities that challenge the ability of Machine Learning based climate emulators to generalize beyond their training distributions. While these emulators offer computationally efficient alternatives to traditional Earth System Models, their reliability remains a potential bottleneck under "no-analog" future climate states, which we define here as regimes where external forcing drives the system into conditions outside the empirical range of the historical training data. A fundamental challenge in evaluating this reliability is data contamination; because many models are trained on simulations that already encompass future scenarios, true out-of-distribution (OOD) performance is often masked. To address this, we benchmark the OOD robustness of three state-of-the-art architectures: U-Net, ConvLSTM, and the ClimaX foundation model specifically restricted to a historical-only training regime (1850-2014). We evaluate these models using two complementary strategies: (i) temporal extrapolation to the recent climate (2015-2023) and (ii) cross-scenario forcing shifts across divergent emission pathways. Our analysis within this experimental setup reveals an accuracy vs. stability trade-off: while the ClimaX foundation model achieves the lowest absolute error, it exhibits higher relative performance changes under distribution shifts, with precipitation errors increasing by up to 8.44% under extreme forcing scenarios. These findings suggest that when restricted to historical training dynamics, even high-capacity foundation models are sensitive to external forcing trajectories. Our results underscore the necessity of scenario-aware training and rigorous OOD evaluation protocols to ensure the robustness of climate emulators under a changing climate.
Abstract (translated)
气候变化加速的步伐带来了深刻的非平稳性，这挑战了基于机器学习的气候模拟器在训练分布之外进行泛化的能力。尽管这些模拟器为传统地球系统模型提供了计算高效的替代方案，但它们在“无先例”的未来气候状态下的可靠性仍是一个潜在瓶颈——本文中我们将“无先例”定义为外部强迫将系统推至历史训练数据经验范围之外的状态。评估这种可靠性的一个根本挑战在于数据污染；由于许多模型在已包含未来情景的模拟上进行训练，真实的分布外性能往往被掩盖。为解决此问题，我们评估了三种最先进架构的分布外鲁棒性：U-Net、ConvLSTM以及严格限制在仅历史训练 regime（1850-2014）下的ClimaX基础模型。我们通过两种互补策略评估这些模型：(i) 向近期气候（2015-2023）的时间外推，以及(ii) 跨 divergent 排放路径的强迫情景转换。在此实验设置内的分析揭示了精度与稳定性之间的权衡：虽然ClimaX基础模型实现了最低绝对误差，但在分布转移下表现出更高的相对性能变化，在极端强迫情景下降水误差增幅高达8.44%。这些发现表明，当限制在历史训练动态下时，即使是高容量基础模型也对外部强迫轨迹敏感。我们的结果强调了情景感知训练和严格分布外评估协议的必要性，以确保气候模拟器在变化气候下的鲁棒性。
URL
https://arxiv.org/abs/2603.23043
