Modelling crypto markets by multi-agent reinforcement learning

Paper: arXiv 2402.10803

Abstract

Building on a previous foundation work (Lussange et al. 2020), this study introduces a multi-agent reinforcement learning (MARL) model simulating crypto markets, which is calibrated to the Binance’s daily closing prices of $153$ cryptocurrencies that were continuously traded between 2018 and 2022. Unlike previous agent-based models (ABM) or multi-agent systems (MAS) which relied on zero-intelligence agents or single autonomous agent methodologies, our approach relies on endowing agents with reinforcement learning (RL) techniques in order to model crypto markets. This integration is designed to emulate, with a bottom-up approach to complexity inference, both individual and collective agents, ensuring robustness in the recent volatile conditions of such markets and during the COVID-19 era. A key feature of our model also lies in the fact that its autonomous agents perform asset price valuation based on two sources of information: the market prices themselves, and the approximation of the crypto assets fundamental values beyond what those market prices are. Our MAS calibration against real market data allows for an accurate emulation of crypto markets microstructure and probing key market behaviors, in both the bearish and bullish regimes of that particular time period.

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced multi-agent reinforcement learning (MARL) and complex agent-based modeling to simulate crypto market microstructure, indicating high mathematical density. It is empirically rigorous, being directly calibrated on real-world Binance data for 153 cryptocurrencies over a 5-year period (2018-2022), covering both bearish and bullish regimes.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Model crypto markets with<br>Multi-Agent Reinforcement Learning"] --> B{"Key Methodology"}
  B --> B1["Data: 153 Cryptocurrencies<br>Binance Daily Closing Prices<br>2018-2022"]
  B --> B2["Agents: RL-equipped<br>vs. Zero-Intelligence/Single Agent"]
  B --> B3["Information Sources<br>Market Prices & Fundamental Values"]
  B --> B4["Calibration<br>Emulate Market Microstructure"]
  B --> B5["Regime Analysis<br>Bearish vs. Bullish<br>COVID-19 Era"]
  B --> C["Computational Process"]
  C --> C1["Bottom-up Complexity Inference"]
  C --> C2["Multi-Agent Simulation<br>Interaction of Autonomous Agents"]
  C --> C3["Robustness Check<br>Market Volatility"]
  C --> D{"Outcomes"}
  D --> D1["Accurate Emulation of Crypto Markets"]
  D --> D2["Insights into Market Microstructure"]
  D --> D3["Behavioral Probing across Regimes"]
  D --> D4["Validation of RL approach<br>vs. Traditional ABM/MAS"]

Cited References (177)

1. Security tokens: architecture, smart contract applications and illustrations using SAFE (2020)
2. Market microstructure and order book dynamics in cryptocurrency exchanges (2018)
3. Order flow analysis of cryptocurrency markets (2019)
4. Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia (2023) — link
5. Using an artificial financial market for studying a cryptocurrency market (2017)
6. Highly accurate protein structure prediction with AlphaFold (2021)
7. Toward Reverse Engineering to Economic Analysis: An Overview of Tools and Methodology (2022)
8. Reverse engineering financial markets with majority and minority games using genetic algorithms (2013)

0 of 177 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.