Paper: arXiv 2311.11913

Abstract

The ability to construct a realistic simulator of financial exchanges, including reproducing the dynamics of the limit order book, can give insight into many counterfactual scenarios, such as a flash crash, a margin call, or changes in macroeconomic outlook. In recent years, agent-based models have been developed that reproduce many features of an exchange, as summarised by a set of stylised facts and statistics. However, the ability to calibrate simulators to a specific period of trading remains an open challenge. In this work, we develop a novel approach to the calibration of market simulators by leveraging recent advances in deep learning, specifically using neural density estimators and embedding networks. We demonstrate that our approach is able to correctly identify high probability parameter sets, both when applied to synthetic and historical data, and without reliance on manually selected or weighted ensembles of stylised facts.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper involves advanced mathematical concepts like Bayesian inference, neural density estimators, and simulation-based inference with complex LaTeX notation, indicating high mathematical density. It demonstrates calibration on both synthetic and historical data with specific models (ZI and Chiarella), showing strong implementation and data requirements but stops short of full backtesting with trading strategies.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Calibrate market simulators without<br>manual stylised fact weighting"] --> B["Data & Inputs<br>Simulated & Historical<br>Limit Order Book Data"]
  B --> C["Core Methodology<br>Deep Calibration with Neural Density<br>Estimators & Embedding Networks"]
  C --> D["Computational Process<br>Learn density of parameters<br>conditioned on observed data"]
  D --> E["Key Findings<br>Accurate identification of<br>high probability parameter sets<br>Validated on synthetic & historical data"]

Cited References (149)

  1. The frontier of simulation-based inference (2020)
  2. Ganspace: Discovering interpretable gan controls (2020)
  3. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  4. Deeplob: Deep convolutional neural networks for limit order books (2019)
  5. A data-driven market simulator for small data environments (2020) — link
  6. Limit order books (2013)
  7. Abides: Towards high-fidelity market simulation for ai research (2019) — link
  8. Financial time series analysis and forecasting with Hilbert–Huang transform feature generation and machine learning (2021)
and 141 more
  1. Agent-based modeling in economics and finance: past, present, and future (2022)
  2. Towards closing the sim-to-real gap in collaborative multi-robot deep reinforcement learning (2020)
  3. Towards realistic market simulations: a generative adversarial networks approach (2021)
  4. Closing the simulation-to-reality gap using generative neural networks: Training object detectors for soccer robotics in simulation as a case study (2020)
  5. Agent-based models of financial markets (2007)
  6. Public regulation of the securities markets (1963)
  7. The microstructure of the “flash crash”: flow toxicity, liquidity crashes, and the probability of informed trading (2011)
  8. Validation of agent-based models in economics and finance (2019)
  9. Econophysics: financial time series from a statistical physics point of view (2000)
  10. Evaluating and explaining generative adversarial networks for continual learning under concept drift (2021)
  11. ABIDES-gym: gym environments for multi-agent discrete event simulation and application to financial markets (2021)
  12. Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology — → 2303.00080
  13. Understanding intra-day price formation process by agent-based financial market simulation: calibrating the extended chiarella model (2022) — link
  14. Efficient Calibration of Multi-Agent Simulation Models from Output Series with Bayesian Optimization (2022)
  15. Black-box model risk in finance (2021) — link
  16. Normalizing flows: An introduction and review of current methods (2020)
  17. Neural density estimation and likelihood-free inference (2019) — link
  18. The disposition effect and momentum (2002)
  19. Learning who is in the market from time series: market participant discovery through adversarial calibration of multi-agent simulators (2021) — link
  20. Graph-informed simulation-based inference for models of active matter (2023) — link
  21. Training deep neural density estimators to identify mechanistic models of neural dynamics (2020)
  22. Simulation-based inference in particle physics (2021)
  23. Co-existence of trend and value in financial markets: Estimating an extended Chiarella model (2020)
  24. Get real: Realism metrics for robust limit order book market simulations (2020)
  25. Stylized facts of return series, robust estimates and three popular models of volatility (2011)
  26. The dynamics of speculative behaviour (1992)
  27. Measuring high-frequency causality between returns, realized volatility, and implied volatility (2012)
  28. More stylized facts of financial markets: leverage effect and downside correlations (2001)
  29. Multi-agent-based order book model of financial markets (2006)
  30. Automatic posterior transformation for likelihood-free inference (2019)
  31. Fast $\varepsilon$-free inference of simulation models with bayesian conditional density estimation (2016)
  32. Neural spline flows (2019)
  33. Likelihood-free mcmc with amortized approximate ratio estimators (2020)
  34. Masked autoregressive flow for density estimation (2017)
  35. A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful (2022) — link
  36. Multivariate Realized Volatility Forecasting with Graph Neural Network (2022)
  37. Investigating the Impact of Model Misspecification in Neural Simulation-based Inference (2022) — link
  38. Learning Robust Statistics for Simulation-based Inference under Model Misspecification (2023) — link
  39. Black-box Bayesian inference for economic agent-based models (2022) — link
  40. Calibrating agent-based models to microdata with graph neural networks (2022) — link
  41. The patent holder’s dilemma: Buy, sell, or troll? (2007) — link
  42. Deciding equivalances among conjunctive aggregate queries (2007) — link
  43. Special issue: Digital Libraries (1996)
  44. Understanding Policy-Based Networking (2001)
  45. First-Order Dynamic Logic (1979) — link
  46. The title of book one (2007) — link
  47. The title of book two (2008) — link
  48. Achieving application requirements (1990) — link
  49. Statecarts in use: structured analysis and object-orientation (1998) — link
  50. The Art of Computer Programming, Vol. 1: Fundamental Algorithms (3rd. ed.) (1997)
  51. The Art of Computer Programming (1998)
  52. An experiment in bibliographic mark-up: Parsing metadata for XML export (2010) — link
  53. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2007)
  54. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2008)
  55. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2009)
  56. Predicate Path expressions (1979) — link
  57. LOGICS of Programs: AXIOMATICS and DESCRIPTIVE POWER (1978)
  58. Optimal Motion Control of a Ground Vehicle (2003)
  59. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  60. Introduction to Bayesian Statistics (2001) — link
  61. CLIFFORD: a Maple 11 Package for Clifford Algebra Computations, version 11 (2007) — link
  62. Stats and Analysis (2006) — link
  63. A more perfect union (2008) — link
  64. The fountain of youth (2009)
  65. Solder man (2003) — link
  66. Interview with Bill Kinder: January 13, 2005 (2005) — link
  67. The Enabling of Digital Libraries (2008)
  68. (old) Finding minimum congestion spanning trees (2000) — link
  69. (new) Finding minimum congestion spanning trees (2000) — link
  70. (old) Distributed data source verification in wireless sensor networks (2009) — link
  71. (new) Distributed data source verification in wireless sensor networks (2009) — link
  72. Portalis: using competitive online interactions to support aid initiatives for the homeless (2008) — link
  73. Visual Basic 6: Design, Specification, and Objects with Other (1999)
  74. The Latex Web Companion: Integrating TEX, HTML, and XML (1999)
  75. Vertex Types in Book-Embeddings (1987)
  76. Vertex Types in Book-Embeddings (1987)
  77. CHI ‘08: CHI ‘08 extended abstracts on Human factors in computing systems (2008)
  78. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  79. IEEE TCSC Executive Committee (2004) — link
  80. Distributed systems (2nd Ed.) (1993)
  81. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  82. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  83. Seminumerical Algorithms (1981)
  84. E-commerce and cultural values (2001) — link
  85. Chapter 9 (2002) — link
  86. The implementation of electronic commerce in SMEs in Singapore (Incoll) (2003) — link
  87. E-commerce and cultural values - (InBook-num-in-chap) (2004) — link
  88. E-commerce and cultural values (Inbook-text-in-chap) (2005) — link
  89. E-commerce and cultural values (Inbook-num chap) (2006) — link
  90. A library-based synthesis methodology for reversible logic (2010)
  91. Synthesis of Reversible Circuit Using Cycle-Based Approach (2010)
  92. Algorithmic Enumeration of Ideal Classes for Quaternion Orders (2010) — link
  93. Chapter II: Notes on data structuring (1972) — link
  94. Transcript of question and answer session (1981) — link
  95. Go to statement considered harmful (1979) — link
  96. Three-dimensional virtual acoustic displays (1992) — link
  97. Managerial expert systems and organizational change: some critical research issues (1987) — link
  98. Simplified Structured COBOL with Microsoft/MicroFocus COBOL (1990)
  99. A multi-radio unification protocol for IEEE 802.11 wireless networks (2004)
  100. Wireless Sensor Networks: A Survey (2002)
  101. A Survey on Wireless Multimedia Sensor Networks (2007)
  102. SSCH : Slotted Seeded Channel Hopping for Capacity Improvement in IEEE 802.11 Ad-Hoc Wireless Networks (2004)
  103. XBOW Sensor Motes Specifications (2008)
  104. Overview of Sensor Networks (2004)
  105. CodeBlue : Sensor Networks for Medical Care (2008)
  106. Investigating Network Architectures for Body Sensor Networks (2007) — link
  107. Channel-Hopping Multiple Access (2000)
  108. Body Sensor Networks (2008)
  109. Mapping Powerlists onto Hypercubes (1994)
  110. Automatic Parallelization for Distributed-Memory Multiprocessing Systems (1989)
  111. User recovery and reversal in interactive systems (1984)
  112. Generalizing specifications for uniformly implemented loops (1985)
  113. Towards monolingual programming environments (1985)
  114. The \TeX book (1984)
  115. Distributed algorithms for finding centers and medians in networks (1984)
  116. \it \LaTeX : A Document Preparation System (1986)
  117. Program transformations in a denotational setting (1985)
  118. Seminumerical Algorithms (1981)
  119. A high-level approach to computer document formatting (1980)
  120. A multifrequency MAC specially designed for wireless sensor network applications (2010) — link
  121. Institutional members of the \TeX Users Group (2017) — link
  122. acmart— Class for typesetting publications of ACM (2017) — link
  123. The \textsl doclicense package (2022) — link
  124. Reasoning About Naming Systems (1993) — link
  125. Babel, a Multilingual Style-Option System for Use with LaTeX’s Standard Document Styles (1991)
  126. A Methodology for Implementing Highly Concurrent Data Objects (1993) — link
  127. Publication quality tables in \LaTeX (2005)
  128. Using the amsthm Package (2015)
  129. R: A Language and Environment for Statistical Computing (2019) — link
  130. UMass Citation Field Extraction Dataset (2013) — link
  131. Citation concept analysis (CCA) —A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S.~Kuhn and Karl R.~Popper (2019) — link
  132. Learning Soft Linear Constraints with Application to Citation Field Extraction (2014) — link
  133. Maintaining Discrete Probability Distributions Optimally (1993)
  134. Scilab — link
  135. Scilab — link
  136. The Computational Geometry Algorithms Library — link
  137. The Computational Geometry Algorithms Library — link
  138. The Parmap library — link
  139. The Parmap library — link
  140. BabyJ : from Object Based to Class Based Programming via Types (2003)
  141. tag-sound

1 of 149 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.