Abstract
Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large language models (LLMs). Various strategies like byte-level representations or calendar tokens have been proposed. However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns. This paper presents the first empirical study of temporal tokenization for event sequences, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar tokens, classic uniform binning, and adaptive residual scalar quantization. We evaluate these strategies by fine-tuning LLMs on real-world datasets that exemplify these diverse distributions. Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, with log-based strategies excelling on skewed distributions and human-centric formats proving robust for mixed modalities.
Abstract (translated)
在使用大型语言模型（LLM）建模时间事件序列时，表示连续时间是一个关键且研究不足的挑战。已经提出了各种策略，如字节级表示或日历令牌等方法。然而，在考虑到实际世界事件数据的各种统计分布（从平滑对数正态到离散、尖峰模式）的情况下，最优方法仍不清楚。本文首次针对事件序列的时间标记化进行了实证研究，比较了不同的编码策略：简单的数字字符串、高精度字节级表示、人类语义日历令牌、经典的均匀分箱以及自适应残差标量量化。我们通过在代表这些多样化分布的真实世界数据集上微调LLM来评估这些策略。我们的分析表明，没有任何单一策略能够普遍优于其他方法；相反，预测性能高度依赖于标记器与数据统计特性的匹配程度，基于对数的方法在偏斜分布中表现出色，而以人为中心的格式则证明了其在混合模式下的稳健性。
URL
https://arxiv.org/abs/2512.13618
