Abstract
Cross-frequency transfer learning (CFTL) has emerged as a popular framework for curating large-scale time series datasets to pre-train foundation forecasting models (FFMs). Although CFTL has shown promise, current benchmarking practices fall short of accurately assessing its performance. This shortcoming stems from many factors: an over-reliance on small-scale evaluation datasets; inadequate treatment of sample size when computing summary statistics; reporting of suboptimal statistical models; and failing to account for non-negligible risks of overlap between pre-training and test datasets. To address these limitations, we introduce a unified reimplementation of widely-adopted neural forecasting networks, adapting them for the CFTL setup; we pre-train only on proprietary and synthetic data, being careful to prevent test leakage; and we evaluate on 15 large, diverse public forecast competition datasets. Our empirical analysis reveals that statistical models' accuracy is frequently underreported. Notably, we confirm that statistical models and their ensembles consistently outperform existing FFMs by more than 8.2% in sCRPS, and by more than 20% MASE, across datasets. However, we also find that synthetic dataset pre-training does improve the accuracy of a FFM by 7% percent.
Abstract (translated)
跨频率迁移学习（CFTL）已成为一种流行框架，用于对大规模时间序列数据集进行预处理以训练基础预测模型（FFMs）。尽管CFTL显示出潜力，但当前的基准测试方法未能准确评估其性能。这一不足源于多个因素：过度依赖小规模评估数据集；在计算汇总统计数据时忽视样本量的影响；报告次优统计模型的表现；以及未能考虑到预训练和测试数据集之间重叠的风险。为了解决这些问题，我们引入了一种广泛采用的神经预测网络的统一重新实现方法，并针对CFTL设置进行了调整；仅使用专有及合成数据进行预训练，同时小心避免测试泄漏；并在15个大型且多样的公共预测竞赛数据集中进行评估。我们的实证分析揭示了统计模型的准确性经常被低估。值得注意的是，我们确认统计模型及其集成始终优于现有FFMs，在sCRPS指标上超过8.2%，在MASE（平均绝对规模误差）指标上超过20%。然而，我们也发现使用合成数据集进行预训练可以将FFM的准确率提高7%。
URL
https://arxiv.org/abs/2509.19465
