Paper: arXiv 2411.16585

Abstract

This work presents a generative pre-trained transformer (GPT) designed for modeling financial time series. The GPT functions as an order generation engine within a discrete event simulator, enabling realistic replication of limit order book dynamics. Our model leverages recent advancements in large language models to produce long sequences of order messages in a steaming manner. Our results demonstrate that the model successfully reproduces key features of order flow data, even when the initial order flow prompt is no longer present within the model’s context window. Moreover, evaluations reveal that the model captures several statistical properties, or ‘stylized facts’, characteristic of real financial markets and broader macro-scale data distributions. Collectively, this work marks a significant step toward creating high-fidelity, interactive market simulations.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced transformer architectures with sophisticated tokenization and training strategies, indicating high mathematical complexity. It also demonstrates empirical rigor through the use of real Nasdaq ITCH data, explicit train/validation/test splits, and evaluation against stylized facts of market microstructure.

Research Flowchart

  flowchart TD
  A["Research Goal:\nCreate GPT for financial time series modeling"] --> B["Methodology:\nOrder generation engine in discrete event simulator"]
  B --> C["Data:\nLimit Order Book<br>historical data"]
  B --> D["Computational Process:\nStreaming Transformer<br>Generative Pre-training"]
  C --> D
  D --> E["Key Findings:\n1. Reproduces LOB dynamics\n2. Captures stylized facts\n3. Works beyond context window"]

Cited References (175)

  1. Challenges, tasks, and opportunities in modeling agent-based complex systems (2021) — link
  2. The $ 109T Global Stock Market in 2023 (2023)
  3. What Percentage of Americans Own Stock? (2023)
  4. Common risk factors in the returns on stocks and bonds (1993) — link
  5. Brownian Motion in the Stock Market (1959) — link
  6. Proof That Properly Anticipated Prices Fluctuate Randomly (1965)
  7. A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993) — link
  8. A General Approach to Testing Volatility Models in Time Series (2017) — link
and 167 more
  1. An artificial stock market (1999) — link
  2. Artificial economic life: a simple model of a stockmarket (1994) — link
  3. Emergence of cooperation and organization in an evolutionary game (1997) — link
  4. On the minority game: Analytical and numerical studies (1998) — link
  5. Scaling and criticality in a stochastic multi-agent model of a financial market (1999) — link
  6. Can a zero-intelligence plus model explain the stylized facts of financial time series data? (2012)
  7. The Genoa Artificial Stock Market: Microstructure and Simulations (2003) — link
  8. Financial Market Simulation (2004) — link
  9. Simulating Security Markets in Dynamic and Equilibrium Modes (2010) — link
  10. Heterogeneous information-based artificial stock market (2010) — link
  11. Deep Reinforcement Learning in Agent Based Financial Market Simulation (2020) — link
  12. Market Making via Reinforcement Learning (2018)
  13. A calibration procedure for analyzing stock price dynamics in an agent-based framework (2015) — link
  14. A comparison of economic agent-based model calibration methods (2020) — link
  15. Quant GANs: deep generation of financial time series (2020) — link
  16. Modeling financial time-series with generative adversarial networks (2019) — link
  17. LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study (2023) — link
  18. Generating synthetic data in finance: opportunities, challenges and pitfalls (2020) — link
  19. Deep Learning for Limit Order Books (2016) — link
  20. State Dependent Parallel Neural Hawkes Process for Limit Order Book Event Stream Prediction and Simulation (2022) — link
  21. Learning internal representations by error propagation (1986)
  22. Generating Realistic Stock Market Order Streams (2020) — link
  23. Towards realistic market simulations: a generative adversarial networks approach (2021) — link
  24. Tail-GAN: Nonparametric Scenario Generation for Tail Risk Estimation (2022) — link
  25. Fin-GAN: forecasting and classifying financial time series via generative adversarial networks (2024) — link
  26. On the convergence and mode collapse of GAN (2018) — link
  27. Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy (2021) — link
  28. Temporal Data Meets LLM – Explainable Financial Time Series Forecasting (2023) — link
  29. BloombergGPT: A Large Language Model for Finance (2023) — link
  30. Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019 (2019) — link
  31. Foundation Models for Time Series Analysis: A Tutorial and Survey (2024) — link
  32. Conditional Generative Adversarial Nets (2014) — link
  33. ImageNet: A large-scale hierarchical image database (2009) — link
  34. Microsoft COCO: Common Objects in Context (2014) — link
  35. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] (2012) — link
  36. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018) — link
  37. Measuring Massive Multitask Language Understanding (2020) — link
  38. Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (2024) — link
  39. Revisiting Stylized Facts for Modern Stock Markets (2023) — link
  40. Long Memory in Stock-Market Trading Volume (2000) — link
  41. A Review of Recent Artificial Market Simulation Studies for Financial Market Regulations And/Or Rules (2016) — link
  42. High‐Frequency Trading around Large Institutional Orders (2019) — link
  43. Stochastic Behavioral Asset-Pricing Models and the Stylized Facts (2009) — link
  44. High frequency market microstructure (2015) — link
  45. Characteristic times in stock market indices (1999) — link
  46. Generalized autoregressive conditional heteroskedasticity (1986) — link
  47. bashtage/arch: Release 7.0.0 (2024) — link
  48. Efficient Capital Markets: A Review of Theory and Empirical Work (1970) — link
  49. The Long Memory of the Efficient Market (2004) — link
  50. Statistical properties of share volume traded in financial markets (2000) — link
  51. Price fluctuations from the order book perspective—empirical facts and a simple model (2001) — link
  52. Power-Law Distributions in Empirical Data (2009) — link
  53. More statistical properties of order books and price impact (2003) — link
  54. An Empirical Analysis of the Limit Order Book and the Order Flow in the Paris Bourse (1995) — link
  55. Multifractal analysis of financial markets: a review (2019) — link
  56. Physics and financial economics (1776 \textendash 2014): puzzles, Ising and agent-based models (2014) — link
  57. Empirical properties of asset returns: stylized facts and statistical issues (2001) — link
  58. Econophysics review: I. Empirical facts (2011) — link
  59. Limit order books (2013) — link
  60. Get real: realism metrics for robust limit order book market simulations (2020) — link
  61. Testing for Common Trends (1988) — link
  62. Statistical properties of stock order books: empirical results and models (2002) — link
  63. Radical Complexity (2021) — link
  64. Agent-based economic models and econometrics (2012) — link
  65. Time series properties of an artificial stock market (1999) — link
  66. The Predictive Power of Zero Intelligence in Financial Markets (2003) — link
  67. A multi-assets artificial stock market with zero-intelligence traders (2011) — link
  68. Agent-based simulation of a financial market (2001) — link
  69. Improving Generalization in Reinforcement Learning \textendash Based Trading by Using a Generative Adversarial Market Model (2021) — link
  70. Towards a fully rl-based market simulator (2021) — link
  71. ABIDES: Towards High-Fidelity Multi-Agent Market Simulation (2020) — link
  72. On the Design of Agent-Based Artificial Stock Markets (2013) — link
  73. Stock Price Formation: Precepts from a Multi-Agent Reinforcement Learning Model (2022) — link
  74. Simulation and estimation of an agent-based market-model with a matching engine (2021) — link
  75. Agent-Based Model Calibration using Machine Learning Surrogates (2017) — link
  76. Combining search strategies to improve performance in the calibration of economic ABMs (2023) — link
  77. Is the Juice Worth the Squeeze? Machine Learning (ML) In and For Agent-Based Modelling (ABM) (2020) — link
  78. Agent-Based Modeling for Complex Financial Systems (2018) — link
  79. How markets slowly digest changes in supply and demand (2008) — link
  80. Understanding flash crash contagion and systemic risk: A micro \textendash macro agent-based approach (2019) — link
  81. Reinforcement Learning for Market Making in a Multi-agent Dealer Market (2019) — link
  82. The Flash Crash: High-Frequency Trading in an Electronic Market (2017) — link
  83. Deep Q-learning market makers in a multi-agent simulated stock market (2021) — link
  84. An agent based model of the E-Mini SP500 applied to flash crash analysis (2012) — link
  85. Static and dynamic factors in an information-based multi-asset artificial stock market (2018) — link
  86. Multi-asset financial bubbles in an agent-based model with noise traders’ herding described by an n-vector Ising model (2021) — link
  87. DeepABM: Scalable, efficient and differentiable agent-based simulations via graph neural networks (2021) — link
  88. Th ' e orie de la sp ' e culation (1900) — link
  89. " U ber die von der molekularkinetischen Theorie der W" a rme geforderte Bewegung von in ruhenden Fl" u ssigkeiten suspendierten Teilchen (1905) — link
  90. Portfolio Selection (1952) — link
  91. The Pricing of Options and Corporate Liabilities (1973) — link
  92. An Intertemporal Capital Asset Pricing Model (1973) — link
  93. Capital asset prices: a theory of marketequilibrium under conditions of risk (1964) — link
  94. The Variation of Certain Speculative Prices (1963) — link
  95. ANALYSIS OF LONG-RUN DEPENDENCE IN ECONOMICS-R/S TECHNIQUE (1971)
  96. Agent based modeling in economics and finance: past, present, and future (2022)
  97. Foundations of complexity economics (2021) — link
  98. Theory of games and economic behavior (1947)
  99. The Art of Computer Programming, The: Seminumerical Algorithms (1997)
  100. Recursive least squares with forgetting for online estimation of vehicle mass and road grade: theory and experiments (2005) — link
  101. Some Implementation Aspects of Sliding Window Least Squares Algorithms (2000) — link
  102. Random matrix approach to cross correlations in financial data (2002) — link
  103. Distribution of eigenvalues for a class of real symmetric matrices. (1971)
  104. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020) — link
  105. Econophysics: Empirical facts and agent-based models (2009) — link
  106. Scaling of the distribution of fluctuations of financial market indices (1999) — link
  107. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation (1982) — link
  108. Long range dependence in financial markets — link
  109. Gain/loss asymmetry in time series of individual stock prices and its relationship to the leverage effect (2009) — link
  110. Option pricing: A simplified approach (1979) — link
  111. GARCH 101: The Use of ARCH / GARCH Models in Applied Econometrics (2001) — link
  112. ARIMA Models (1990) — link
  113. Vector Autoregressions (2001) — link
  114. Noise Dressing of Financial Correlation Matrices (1999) — link
  115. RANDOM MATRIX THEORY AND FINANCIAL CORRELATIONS (2000) — link
  116. Financial Applications of Random Matrix Theory: a short review (2009) — link
  117. Proximal Policy Optimization Algorithms (2017) — link
  118. Playing Atari with Deep Reinforcement Learning (2013) — link
  119. High-Frequency Data and Models (2005) — link
  120. The expected value of the adjusted rescaled Hurst range of independent normal summands (1976) — link
  121. Julia: A fresh approach to numerical computing (2017) — link
  122. JuMP: A Modeling Language for Mathematical Optimization (2017) — link
  123. Market Microstructure Theory (1997)
  124. A generative model of a limit order book using recurrent neural networks (2023) — link
  125. Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network (2023) — link
  126. Attention is all you need (2017)
  127. martinobdl/ITCH: ITCH50Converter (2021) — link
  128. LOBSTER: Limit Order Book Reconstruction System (2011) — link
  129. JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading (2023) — link
  130. Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness (2023) — link
  131. Limit Order Flow, Market Impact and Optimal Order Sizes: Evidence from NASDAQ TotalView-ITCH Data (2011) — link
  132. Tokenization as the initial phase in NLP (1992) — link
  133. A Survey of Large Language Models (2023) — link
  134. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) — link
  135. Random search for hyper-parameter optimization (2012)
  136. Root Mean Square Layer Normalization (2019) — link
  137. RoFormer: Enhanced Transformer with Rotary Position Embedding (2021) — link
  138. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints (2023) — link
  139. Efficiently Scaling Transformer Inference (2022) — link
  140. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2022) — link
  141. Extending Context Window of Large Language Models via Positional Interpolation (2023) — link
  142. Longformer: The Long-Document Transformer (2020) — link
  143. Efficient Streaming Language Models with Attention Sinks (2023) — link
  144. Hierarchical Neural Story Generation (2018) — link
  145. The Curious Case of Neural Text Degeneration (2020) — link
  146. ABIDES-gym: gym environments for multi-agent discrete event simulation and application to financial markets (2021) — link
  147. Limit Order Book Simulations: A Review (2024) — link
  148. Analyzing and modeling 1+1d markets (2001) — link
  149. Correlations in economic time series (1997) — link
  150. Scaling in Stock Market Data: Stable Laws and Beyond (1997) — link
  151. powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions (2014) — link
  152. A Survey of Quantization Methods for Efficient Neural Network Inference (2021) — link
  153. Fast Inference from Transformers via Speculative Decoding (2022) — link
  154. LoRA: Low-Rank Adaptation of Large Language Models (2021) — link
  155. QLoRA: Efficient Finetuning of Quantized LLMs (2023) — link
  156. Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2023) — link
  157. Jamba: A Hybrid Transformer-Mamba Language Model (2024) — link
  158. Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling (2024) — link
  159. Adam: A Method for Stochastic Optimization (2014) — link
  160. Generative adversarial nets (2014)
  161. Generative Adversarial Networks in finance: an overview (2021) — link
  162. Scalable Agent-Based Modeling for Complex Financial Market Simulations — → 2312.14903
  163. Long-Term Storage Capacity of Reservoirs (1951) — link
  164. Mosaic organization of DNA nucleotides (1994) — link
  165. Detrended Fluctuation Analysis: A Scale-Free View on Neuronal Oscillations (2012) — link
  166. Long-range correlation properties of coding and noncoding DNA sequences: GenBank analysis (1995) — link
  167. PyTorch: an imperative style, high-performance deep learning library (2019)

1 of 175 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.