Paper: arXiv 2409.07486

Abstract

Generative models aim to simulate realistic effects of various actions across different contexts, from text generation to visual effects. Despite significant efforts to build real-world simulators, the application of generative models to virtual worlds, like financial markets, remains under-explored. In financial markets, generative models can simulate complex market effects of participants with various behaviors, enabling interaction under different market conditions, and training strategies without financial risk. This simulation relies on the finest structured data in financial market like orders thus building the finest realistic simulation. We propose Large Market Model (LMM), an order-level generative foundation model, for financial market simulation, akin to language modeling in the digital world. Our financial Market Simulation engine (MarS), powered by LMM, addresses the domain-specific need for realistic, interactive and controllable order generation. Key observations include LMM’s strong scalability across data size and model complexity, and MarS’s robust and practicable realism in controlled generation with market impact. We showcase MarS as a forecast tool, detection system, analysis platform, and agent training environment, thus demonstrating MarS’s “paradigm shift” potential for a variety of financial applications. We release the code of MarS at https://github.com/microsoft/MarS/.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents complex generative modeling with transformer architectures and tokenization for order-level data, while also providing a released GitHub codebase and multi-faceted empirical demonstrations (forecasting, detection, analysis, and agent training) indicating strong backtest-ready implementation.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Develop realistic financial market simulator<br>without financial risk"] --> B["Methodology<br>Large Market Model LMM<br>Order-level generative foundation model"]
  
  B --> C{"Data Sources"}
  C --> D["Limit Order Book Data<br>Highest granularity order data"]
  C --> E["Market Context Signals<br>Price, volume, volatility"]
  
  C --> F["Computational Process"]
  
  D & E --> F
  F --> G["MarS Engine<br>Controllable order generation<br>Market impact modeling"]
  
  G --> H["Key Outcomes<br>1. Strong scalability across data/model<br>2. Realistic controlled generation<br>3. Multi-use: Forecast, Detect, Analyze, Train<br>4. Paradigm shift for financial apps"]

Cited References (91)

  1. torchdiffeq (2018) — link
  2. Detecting abnormal bid-ask spread: a comparison of event study methods (2000)
  3. Transient linear price impact and Fredholm integral equations (2012)
  4. No-dynamic-arbitrage and market impact (2010)
  5. Market impacts and the life cycle of investors orders (2015)
  6. A fully consistent, minimal model for non-linear market impact (2015)
  7. Optimal execution with non-linear transient market impact (2017)
  8. Exponential resilience and decay of market impact (2011)
and 83 more
  1. Beyond the square root: Evidence for logarithmic dependence of market impact on size and participation rate (2015)
  2. Llama 2: Open foundation and fine-tuned chat models (2023) — link
  3. The llama 3 herd of models (2024) — link
  4. Mistral 7B (2023) — link
  5. Gpt-4 technical report (2023) — link
  6. Phi-3 technical report: A highly capable language model locally on your phone (2024) — link
  7. Video generation models as world simulators (2024) — link
  8. Sora: A review on background, technology, limitations, and opportunities of large vision models (2024) — link
  9. Sequential modeling enables scalable learning for large vision models (2024)
  10. Video as the New Language for Real-World Decision Making (2024) — link
  11. Learning Interactive Real-World Simulators (2023) — link
  12. ABIDES: Towards high-fidelity multi-agent market simulation (2020)
  13. Get real: Realism metrics for robust limit order book market simulations (2020)
  14. The impact of heterogeneous trading rules on the limit order book and order flows (2009)
  15. ABIDES-gym: gym environments for multi-agent discrete event simulation and application to financial markets (2021)
  16. Towards realistic market simulations: a generative adversarial networks approach (2021)
  17. Modeling financial time-series with generative adversarial networks (2019)
  18. Generative adversarial networks (2020)
  19. Learning to simulate realistic limit order book markets from data as a world agent (2022)
  20. Conditional generators for limit order book environments: Explainability, challenges, and robustness (2023)
  21. Generating realistic stock market order streams (2020)
  22. Limit Order Book Simulations: A Review — → 2402.17359
  23. Generative ai for end-to-end limit order book modelling: A token-level autoregressive generative model of message flow using a deep state space network (2023)
  24. A generative model of a limit order book using recurrent neural networks (2023)
  25. Deep state space models for time series forecasting (2018)
  26. Benchmark dataset for mid‐price forecasting of limit order book data with machine learning methods (2018) — link
  27. DeepLOB: Deep Convolutional Neural Networks for Limit Order Books (2019) — link
  28. On the opportunities and risks of foundation models (2021) — link
  29. Language models are few-shot learners (2020) — link
  30. High-Resolution Image Synthesis with Latent Diffusion Models (2021) — link
  31. Zero-shot text-to-image generation (2021)
  32. Retrieval-Augmented Diffusion Models (2022) — link
  33. Visual Instruction Tuning (2023)
  34. Kosmos-2: Grounding Multimodal Large Language Models to the World (2023)
  35. Language Is Not All You Need: Aligning Perception with Language Models (2023)
  36. Is sora a world simulator? a comprehensive survey on general world models and beyond (2024) — link
  37. Bloomberggpt: A large language model for finance (2023) — link
  38. Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance (2024)
  39. Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
  40. Fingpt: Open-source financial large language models — → 2306.06031
  41. Fingpt: Instruction tuning benchmark for open-source large language models in financial datasets — → 2310.04793
  42. Instruct-fingpt: Financial sentiment analysis by instruction tuning of general-purpose large language models — → 2306.12659
  43. Fin-LLAMA: Efficient Finetuning of Quantized LLMs for Finance (2023)
  44. Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters (2023)
  45. Bbt-fin: Comprehensive construction of chinese financial domain pre-trained language model, corpus and benchmark (2023) — link
  46. Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications — → 2408.11878
  47. Investlm: A large language model for investment using financial domain instruction tuning — → 2309.13064
  48. Fintral: A family of gpt-4 level multimodal financial large language models (2024) — link
  49. Foundation models for generalist medical artificial intelligence (2023)
  50. Lawyer llama technical report (2023) — link
  51. Study of Stylized Facts in Stock Market Data (2023) — link
  52. Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks (2023) — link
  53. Market impact and trading profile of hidden orders in stock markets (2009) — link
  54. Master curve for price-impact function (2003) — link
  55. Direct Estimation of Equity Market Impact (2005)
  56. Market impacts and the life cycle of investors orders (2014) — link
  57. A fully consistent, minimal model for non-linear market impact (2015) — link
  58. Reinforcement Learning: An Introduction (2018)
  59. Augmented language models: a survey (2023) — link
  60. Learning Universal Policies via Text-Guided Video Generation (2023) — link
  61. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents (2022)
  62. Synthetic data generation with large language models for text classification: Potential and limitations (2023) — link
  63. Limit order books (2013)
  64. Scaling laws for neural language models (2020) — link
  65. Scaling vision transformers (2022)
  66. The mechanics and regulation of market abuse: a legal and economic analysis (2005)
  67. Market manipulation: A survey (2012)
  68. An empirical analysis of illegal insider trading (1992)
  69. MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading (2024)
  70. Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools — → 2311.10801
  71. Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
  72. Hats: A hierarchical graph attention network for stock movement prediction (2019) — link
  73. Stock trend prediction with multi-granularity data: A contrastive learning approach with adaptive fusion (2021)
  74. Pysindy: a python package for the sparse identification of nonlinear dynamics from data (2020) — link
  75. Taming transformers for high-resolution image synthesis (2021)
  76. High-resolution image synthesis with latent diffusion models (2022)
  77. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs (2021) — link
  78. Zero: Memory optimizations toward training trillion parameter models (2020)
  79. Decoupled weight decay regularization (2017) — link
  80. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  81. Revisiting Stylized Facts for Modern Stock Markets (2023) — link
  82. Modeling financial time-series with generative adversarial networks (2019) — link
  83. Volatilities of different time resolutions — Analyzing the dynamics of market components (1997) — link

7 of 91 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.