Abstract
Recent evaluations of time-series foundation models (TSFMs) have emphasized synthetic benchmarks, leaving real-world generalization less thoroughly examined. This work proposes a novel benchmarking approach that bridges synthetic and realistic data by extracting temporal signals from real-world video using optical flow and curating datasets reflecting everyday temporal dynamics. Building upon this pipeline, we introduce REAL-V-TSFM, a novel dataset designed to capture rich and diverse time series derived from real-world videos. Experimental results on three state-of-the-art of TSFMs under zero-shot forecasting shows that, despite strong performance on conventional benchmarks, these models predominantly exhibit performance degradation on the proposed dataset, indicating limited generalizability in these foundation models. These findings highlight the urgent need for data-centric benchmarking and diverse model structure to advance TSFMs toward genuine universality, while further validating the effectiveness of our video-based time series data extraction pipeline.
Abstract (translated)
最近对时间序列基础模型（TSFM）的评估主要集中在合成基准上，而现实世界中的泛化性能则较少被深入研究。本文提出了一种新的基准测试方法，该方法通过使用光学流从真实世界的视频中提取时间信号，并整理出反映日常动态变化的数据集来连接合成数据和实际数据之间的鸿沟。在此基础上，我们引入了REAL-V-TSFM，这是一个新型数据集，旨在捕捉来自真实世界视频的丰富多样的时间序列。 在三个最先进的TSFM模型上进行的零样本预测实验表明，尽管这些模型在传统的基准测试中表现出色，但在我们的提议数据集中却普遍表现下降，这表明现有的基础模型在泛化能力方面存在局限性。这一发现强调了数据为中心的基准测试方法以及多样化模型结构对于推动时间序列基础模型向真正普适性的发展的迫切需求，并进一步验证了我们基于视频的时间序列数据提取流水线的有效性。
URL
https://arxiv.org/abs/2509.26347
