Abstract
The success of large language models (LLMs) for time series has been demonstrated in previous work. Utilizing a symbolic time series representation, one can efficiently bridge the gap between LLMs and time series. However, the remaining challenge is to exploit the semantic information hidden in time series by using symbols or existing tokens of LLMs, while aligning the embedding space of LLMs according to the hidden information of time series. The symbolic time series approximation (STSA) method called adaptive Brownian bridge-based symbolic aggregation (ABBA) shows outstanding efficacy in preserving salient time series features by modeling time series patterns in terms of amplitude and period while using existing tokens of LLMs. In this paper, we introduce a method, called LLM-ABBA, that integrates ABBA into large language models for various downstream time series tasks. By symbolizing time series, LLM-ABBA compares favorably to the recent state-of-the-art (SOTA) in UCR and three medical time series classification tasks. Meanwhile, a fixed-polygonal chain trick in ABBA is introduced to \kc{avoid obvious drifting} during prediction tasks by significantly mitigating the effects of cumulative error arising from misused symbols during the transition from symbols to numerical values. In time series regression tasks, LLM-ABBA achieves the new SOTA on Time Series Extrinsic Regression (TSER) benchmarks. LLM-ABBA also shows competitive prediction capability compared to recent SOTA time series prediction results. We believe this framework can also seamlessly extend to other time series tasks.
Abstract (translated)
大型语言模型（LLMs）在时间序列上的成功已在先前的研究中得到验证。利用符号化的时间序列表示方法，可以有效地弥合LLMs与时间序列之间的差距。然而，当前面临的挑战在于如何通过使用符号或现有LLM令牌来挖掘时间序列中的语义信息，并根据时间序列的隐藏信息对LLM的嵌入空间进行调整。一种名为自适应布朗桥基础符号聚合（ABBA）的时间序列近似（STSA）方法，在保持显著特征的同时，通过对幅度和周期建模来保留时间序列模式，同时使用现有LLMs令牌展现出了出色的效能。本文介绍了一种称为LLM-ABBA的方法，它将ABBA集成到大型语言模型中以处理各种下游时间序列任务。通过符号化时间序列，LLM-ABBA在UCR和三个医疗时间序列分类任务中优于近期的最先进（SOTA）方法。同时，在ABBA中引入了一个固定多边形链技巧，用于显著减轻从符号到数值转换过程中因误用符号而产生的累积误差影响，从而避免预测任务中的明显漂移现象。在时间序列回归任务上，LLM-ABBA在时间序列外在回归（TSER）基准测试中达到了新的SOTA标准。此外，与近期的最先进时间序列预测结果相比，LLM-ABBA展示了竞争力较强的预测能力。我们相信此框架可以无缝扩展到其他时间序列任务之中。
URL
https://arxiv.org/abs/2411.18506
