Paper: arXiv 2610.16524

Authors: Fernando Acero, Philipp Plank, Yufei Zhang

Abstract

Market making under realistic order-flow dynamics is challenging due to the clustered, self-exciting nature of order arrivals, which are naturally modeled by non-Markovian Hawkes processes. We propose a model-free reinforcement learning (RL) framework for market making with general Hawkes order-flow dynamics. We first establish non-asymptotic performance bounds that quantify the loss incurred by approximating the infinite-dimensional Hawkes state using a finite collection of exponentially weighted features. Guided by this analysis, we introduce the Hawkes-Structured Network (HSN), a novel neural architecture that encodes historical order flow through a learnable mixture-of-exponentials filter. HSN is trained end-to-end using model-free RL, jointly optimizing the feature representation and quoting policy directly from reward feedback. Experiments on single- and multi-asset market-making problems with mixture-of-exponentials, power-law, and regime-switching kernels demonstrate that HSN outperforms general-purpose neural architectures, achieving higher trading rewards while implicitly recovering the underlying order-flow intensities.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a novel neural architecture (HSN) for market making under complex Hawkes processes, backed by theoretical performance bounds. It combines advanced mathematical modeling of order flow with a robust, model-free RL framework. The empirical results demonstrate strong performance against baselines across various kernel types.

Research Flowchart

  flowchart TD
    A[Research Goal: Market Making under Realistic Order-Flow] --> B{Key Methodology: Model-Free RL with Hawkes Processes};
    B --> C[Data/Inputs: Historical Order Flow Dynamics];
    C --> D{Computational Process: Hawkes-Structured Network (HSN) Training};
    D --> E[Key Findings: HSN Outperforms Baselines, Recovers Intensities];

Cited References (61)

  1. Markovian approximations of stochastic Volterra equations with the fractional kernel (2023)
  2. Logarithmic regret in the ergodic Avellaneda–Stoikov market making model — → 2409.02025
  3. Deep q-learning market makers in a multi-agent simulated stock market (2021)
  4. Recent advances in reinforcement learning in finance (2023)
  5. Deep reinforcement learning for market making under a Hawkes process-based limit order book model (2022)
  6. Reinforcement learning for market making in a multi-agent dealer market (2019) — link
  7. Limit order book dynamics and order size modelling using compound Hawkes process — → 2312.08927
  8. Modelling security market events in continuous time: Intensity based, multivariate point process models (2007)
and 53 more
  1. Quantifying reflexivity in financial markets: Toward a prediction of flash crashes (2012)
  2. FX market making with internal liquidity — → 2512.04603
  3. Closed-form approximations in multi-asset market making (2021)
  4. Market making with fads, informed, and uninformed traders — → 2501.03658
  5. AlphaPortfolio: Direct construction through deep reinforcement learning and interpretable AI (2021)
  6. A universal end-to-end approach to portfolio optimization via deep learning (2021) — link
  7. A machine learning approach to risk based asset allocation in portfolio optimization (2025)
  8. Proximal policy optimization algorithms (2017) — link
  9. Approximation of stochastic V olterra equations with kernels of completely monotone type (2024)
  10. Bernstein Functions: Theory and Applications (2012)
  11. Stability of nonlinear H awkes processes (1996)
  12. An expansion formula for H awkes processes and application to cyber-insurance derivatives (2023)
  13. Exponential moments for H awkes processes under minimal assumptions (2024)
  14. Brownian Motion: An Introduction to Stochastic Processes (2014)
  15. Robust market making via adversarial reinforcement learning (2020) — link
  16. Deep reinforcement learning for market making in corporate bonds: Beating the curse of dimensionality (2019)
  17. Optimal market making (2017)
  18. Optimal market-making with H awkes process: A M arkovian approximation approach via M ercer’s expansion (2026)
  19. An impulse control approach to market making in a Hawkes LOB market — → 2510.26438
  20. Buy low, sell high: A high frequency trading perspective (2014)
  21. High-frequency trading in a limit order book (2008)
  22. Dealing with the inventory risk: A solution to the market making problem (2013)
  23. Modelling microstructure noise with mutually exciting point processes (2013)
  24. Hawkes processes in finance (2015)
  25. Hawkes model for price and trades high-frequency dynamics (2014)
  26. Critical reflexivity in financial markets: A H awkes process analysis (2013)
  27. State-dependent Hawkes processes and their application to limit order book modelling (2021)
  28. Reinforcement learning for market making in a multi-agent dealer market (2019)
  29. Market making via reinforcement learning (2018)
  30. Approximately optimal approximate reinforcement learning (2002)
  31. Performance bounds in $L_p$-norm for approximate value iteration (2007)
  32. An upper bound on the loss from approximate optimal-value functions (1994)
  33. High-dimensional continuous control using generalized advantage estimation (2016)
  34. Learning phrase representations using RNN encoder–decoder for statistical machine translation (2014)
  35. Long short-term memory (1997)
  36. Attention is all you need (2017)
  37. Adam: A method for stochastic optimization (2015)
  38. An Introduction to the Theory of Point Processes, Volume I : Elementary Theory and Methods (2003)
  39. On Lewis’ simulation method for point processes (1981)
  40. Apparent criticality and calibration issues in the Hawkes self-excited point process model: Application to high-frequency financial data (2015)
  41. The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process (2017)
  42. Recurrent Marked Temporal Point Processes: Embedding Event History to Vector (2016)
  43. Transformer Hawkes Process (2020)
  44. Algorithmic and High-Frequency Trading (2015)
  45. Reinforcement Learning and Optimal Control (2019)
  46. Approximation of $1/x$ by exponential sums in $[1, \infty)$ (2005)
  47. Approximation by exponential sums revisited (2010)
  48. Generalized F eller processes and M arkovian lifts of stochastic V olterra processes: The affine case (2020)
  49. Affine V olterra processes (2019)
  50. A general characterization of one factor affine term structure models (2001)
  51. Solving Least Squares Problems (1995)
  52. A data-driven deep learning approach for options market making (2023)
  53. Rough volatility, path-dependent PDEs and weak rates of convergence — → 2304.03042

6 of 61 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.