Paper: arXiv 2511.02136

Authors: Valentin Mohl, Sascha Frey, Reuben Leyland, Kang Li, George Nigmatulin, Mihai Cucuringu, Stefan Zohren, Jakob Foerster, Anisoara Calinescu

Abstract

Agent-based modelling (ABM) approaches for high-frequency financial markets are difficult to calibrate and validate, partly due to the large parameter space created by defining fixed agent policies. Multi-agent reinforcement learning (MARL) enables more realistic agent behaviour and reduces the number of free parameters, but the heavy computational cost has so far limited research efforts. To address this, we introduce JaxMARL-HFT (JAX-based Multi-Agent Reinforcement Learning for High-Frequency Trading), the first GPU-accelerated open-source multi-agent reinforcement learning environment for high-frequency trading (HFT) on market-by-order (MBO) data. Extending the JaxMARL framework and building on the JAX-LOB implementation, JaxMARL-HFT is designed to handle a heterogeneous set of agents, enabling diverse observation/action spaces and reward functions. It is designed flexibly, so it can also be used for single-agent RL, or extended to act as an ABM with fixed-policy agents. Leveraging JAX enables up to a 240x reduction in end-to-end training time, compared with state-of-the-art reference implementations on the same hardware. This significant speed-up makes it feasible to exploit the large, granular datasets available in high-frequency trading, and to perform the extensive hyperparameter sweeps required for robust and efficient MARL research in trading. We demonstrate the use of JaxMARL-HFT with independent Proximal Policy Optimization (IPPO) for a two-player environment, with an order execution and a market making agent, using one year of LOB data (400 million orders), and show that these agents learn to outperform standard benchmarks. The code for the JaxMARL-HFT framework is available on GitHub.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper focuses heavily on implementing a practical GPU-accelerated framework (JAX) and scaling experiments with real-world data (400M orders), but lacks advanced mathematical derivations or novel theory.

Research Flowchart

  flowchart TD
  A["Research Goal: Develop GPU-accelerated<br>Multi-Agent RL for HFT"] --> B["JaxMARL-HFT Framework Development"]
  B --> C["Input: 1 Year MBO Data<br>400M Orders"]
  C --> D{"Computational Process:<br>GPU-Accelerated Training"}
  D --> E["240x Speed-Up vs SOTA<br>Enables Large-Scale Sweeps"]
  E --> F["Agents: Market Making<br>& Order Execution"]
  F --> G["Method: IPPO Training"]
  G --> H["Key Findings:<br>Agents Outperform Benchmarks<br>Open Source GitHub Code"]

Cited References (41)

  1. GPUD rive: Data-driven, multi-agent driving simulation at 1 million FPS (2025) — link
  2. An Extensible, Data-Oriented Architecture for High-Performance, Many-World Simulation (2023)
  3. Trades, quotes and prices: financial markets under the microscope (2018)
  4. Robust Autonomy Emerges from Self-Play (2025) — link
  5. Limit order books (2013)
  6. Mbt-gym: Reinforcement learning for model-based limit order book trading (2023)
  7. Deep Q-learning market makers in a multi-agent simulated stock market (2022) — link
  8. LOBSTER : Limit order book reconstruction system (2011)
and 33 more
  1. Modelling crypto markets by multi-agent reinforcement learning — → 2402.10803
  2. Generative AI for end-to-end limit order book modelling: A token-level autoregressive generative model of message flow using a deep state space network (2023)
  3. Double deep q-learning for optimal execution (2021)
  4. A reinforcement learning approach to optimal execution (2022)
  5. Reinforcement learning approaches to optimal market making (2021)
  6. Market making via reinforcement learning (2018)
  7. Deep reinforcement learning for active high frequency trading (2021) — link
  8. An application of deep reinforcement learning to algorithmic trading (2021)
  9. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  10. Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality (1993)
  11. Get real: realism metrics for robust limit order book market simulations (2021) — link
  12. An agent-based modeling approach to study price impact (2012) — link
  13. The predictive power of zero intelligence in financial markets (2005)
  14. Artificial economic life: a simple model of a stockmarket (1994)
  15. JaxMARL : Multi-agent RL Environments and Algorithms in JAX (2024)
  16. JaxMARL: Multi-agent RL environments and algorithms in JAX (2024)
  17. ABIDES -gym: gym environments for multi-agent discrete event simulation and application to financial markets (2022)
  18. A stochastic partial differential equation model for limit order book dynamics (2021)
  19. Reinforcement learning in agent-based market simulation: Unveiling realistic stylized facts and behavior (2024)
  20. Many learning agents interacting with an agent-based market model — → 2303.07393
  21. Modelling stock markets by multi -agent reinforcement learning (2021) — link
  22. An agent - based market simulator for back - testing deep reinforcement learning based trade execution strategies (2021)
  23. JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading (2023)
  24. ABIDES : Towards high - fidelity multi - agent market simulation (2020)
  25. A financial market simulation environment for trading agents using deep reinforcement learning (2024)
  26. Towards multi-agent reinforcement learning-driven over-the-counter market simulations (2024) — link
  27. Robust market making via adversarial reinforcement learning (2021)
  28. Multi-agent reinforcement learning in a realistic limit order book market simulation (2020) — link
  29. Towards a fully RL -based market simulator (2022)
  30. Tianshou: A highly modularized deep reinforcement learning library (2022)
  31. JAX : composable transformations of P ython+ N um P y programs (2018) — link
  32. Fast agent-based simulation framework with applications to reinforcement learning and the study of trading latency effects (2022)
  33. High-frequency trading in a limit order book (2008)

2 of 41 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.