Abstract
Generative modeling of high-frequency limit order book (LOB) dynamics is a critical yet unsolved challenge in quantitative finance, essential for robust market simulation and strategy backtesting. Existing approaches are often constrained by simplifying stochastic assumptions or, in the case of modern deep learning models like Transformers, rely on tokenization schemes that affect the high-precision, numerical nature of financial data through discretization and binning. To address these limitations, we introduce ByteGen, a novel generative model that operates directly on the raw byte streams of LOB events. Our approach treats the problem as an autoregressive next-byte prediction task, for which we design a compact and efficient 32-byte packed binary format to represent market messages without information loss. The core novelty of our work is the complete elimination of feature engineering and tokenization, enabling the model to learn market dynamics from its most fundamental representation. We achieve this by adapting the H-Net architecture, a hybrid Mamba-Transformer model that uses a dynamic chunking mechanism to discover the inherent structure of market messages without predefined rules. Our primary contributions are: 1) the first end-to-end, byte-level framework for LOB modeling; 2) an efficient packed data representation; and 3) a comprehensive evaluation on high-frequency data. Trained on over 34 million events from CME Bitcoin futures, ByteGen successfully reproduces key stylized facts of financial markets, generating realistic price distributions, heavy-tailed returns, and bursty event timing. Our findings demonstrate that learning directly from byte space is a promising and highly flexible paradigm for modeling complex financial systems, achieving competitive performance on standard market quality metrics without the biases of tokenization.
Abstract (translated)
高频限价订单簿（LOB）动态的生成建模是定量金融中的一个关键但尚未解决的挑战，对于稳健的市场模拟和策略回测至关重要。现有的方法通常受限于简化随机假设或依赖于现代深度学习模型（如Transformer）使用的分词方案，这些方案通过离散化和分类影响了金融市场数据高精度数值特性。为了解决这些问题，我们引入了一个新颖的生成模型ByteGen，该模型直接在LOB事件的原始字节流上操作。我们的方法将问题视为一个自回归的下一个字节预测任务，并为此设计了一种紧凑且高效的32字节打包二进制格式来表示市场消息而不丢失信息。我们工作的核心创新在于完全消除了特征工程和分词，使模型能够从最基本的形式中学习市场动态。通过适应H-Net架构（一种混合Mamba-Transformer模型），该模型采用了一种动态切片机制，在没有预定义规则的情况下发现市场的内在结构，从而实现了这一点。 我们的主要贡献包括：1）第一个端到端、字节级的LOB建模框架；2）一种高效的打包数据表示方式；3）在高频数据上的全面评估。ByteGen使用来自CME比特币期货的超过3400万个事件进行训练，并成功再现了金融市场的关键统计特征，生成了现实的价格分布、尾部重的回报以及突发性事件时间间隔。 我们的发现表明，直接从字节空间学习是一种有前景且高度灵活的方法来建模复杂的金融市场系统，在标准市场质量指标上实现了与分词方法相比无偏差的竞争性能。
URL
https://arxiv.org/abs/2508.02247
