Paper: arXiv 2409.02025

Abstract

We analyse the regret arising from learning the price sensitivity parameter $κ$ of liquidity takers in the ergodic version of the Avellaneda-Stoikov market making model. We show that a learning algorithm based on a maximum-likelihood estimator for the parameter achieves the regret upper bound of order $\ln^2 T$ in expectation. To obtain the result we need two key ingredients. The first is the twice differentiability of the ergodic constant under the misspecified parameter in the Hamilton-Jacobi-Bellman (HJB) equation with respect to $κ$, which leads to a second–order performance gap. The second is the learning rate of the regularised maximum-likelihood estimator which is obtained from concentration inequalities for Bernoulli signals. Numerical experiments confirm the convergence and the robustness of the proposed algorithm.

Complexity vs Empirical Score

  • Math Complexity: 9.2/10
  • Empirical Rigor: 3.5/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper is mathematically dense, featuring ergodic stochastic control, Hamilton-Jacobi-Bellman equations, and logarithmic regret analysis in continuous time. However, empirical validation is limited to numerical experiments without specific backtest metrics, code, or real-market datasets.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Regret analysis for learning κ<br>in ergodic Avellaneda-Stoikov model"] --> B["Methodology"]
  
  B --> C["Key Ingredients & Analysis"]
  C --> D["1. HJB Differentiability:<br>Twice differentiable ergodic constant<br>wrt κ (misspecified parameter)"]
  C --> E["2. Learning Theory:<br>Regularised Maximum-Likelihood Estimator<br>for Bernoulli signals"]
  
  B --> F["Computational Process:<br>Simulation of ergodic market<br>with adaptive spreads based on learned κ"]
  
  D & E & F --> G["Main Theoretical Result:<br>Regret upper bound O(ln²T) in expectation"]
  
  H["Numerical Experiments"] --> I["Key Findings:<br>- Convergence of regret<br>- Robustness of algorithm<br>- Validation of theory"]
  
  G --> I

Cited References (64)

  1. Optimal Scheduling of Entropy Regularizer for Continuous-Time Linear-Quadratic Reinforcement Learning (2024)
  2. Naive exploration is optimal for online LQR (2020)
  3. Explore more and improve regret in linear quadratic regulators (2020) — link
  4. Optimal adaptive control of linear-quadratic-Gaussian systems (1983)
  5. Policy gradient methods for the noisy linear quadratic regulator over a finite horizon (2021)
  6. On adaptive linear–quadratic regulators (2020)
  7. Regret bounds for robust adaptive control of the linear quadratic regulator (2018)
  8. Learning Linear-Quadratic Regulators Efficiently with only $\sqrt T $ Regret (2019)
and 56 more
  1. Logarithmic regret for learning linear quadratic regulators efficiently (2020)
  2. Adaptive linear quadratic gaussian control: the cost-biased approach revisited (1998)
  3. Improved regret bounds for thompson sampling in linear quadratic control problems (2018)
  4. Automated market makers designs beyond constant functions (2023)
  5. Regret bounds for the adaptive control of linear quadratic systems (2011)
  6. Logarithmic regret for online control (2019)
  7. Logarithmic online regret bounds for undiscounted reinforcement learning (2006)
  8. Ergodic problem for the H amilton– J acobi– B ellman equation. I. Existence of the ergodic attractor (1997)
  9. Ergodic control of McKean–Vlasov SDEs and associated Bellman equation (2023)
  10. On ergodic stochastic control (1998)
  11. Asymptotic solutions of Hamilton-Jacobi equations in Euclidean n space (2006)
  12. Long time asymptotics for fully nonlinear Bellman equations: a backward SDE approach (2016)
  13. Ergodic control of diffusion processes (2012)
  14. Optimal control on graphs: existence, uniqueness, and long-term behavior (2020)
  15. Numerical methods for stochastic control problems in continuous time (2001)
  16. Numerical Method for Ergodic Optimal Control Problems of Switching Stochastic Differential Equations with Reflection (2020)
  17. Algorithmic and high-frequency trading (2015)
  18. The Financial Mathematics of Market Liquidity: From optimal execution to market making (2016)
  19. Optimal transport for applied mathematicians (2015)
  20. Optimal market making (2017)
  21. Dealing with the inventory risk: a solution to the market making problem (2013)
  22. High-frequency market-making with inventory constraints and directional bets (2012) — link
  23. High-frequency trading in a limit order book (2008)
  24. Vega Protocol (2019)
  25. Margins and Credit Risk on Vega (2019)
  26. Optimal dealer pricing under transactions and return uncertainty (1981)
  27. High-frequency market-making with inventory constraints and directional bets (2013)
  28. Algorithmic trading with model uncertainty (2017)
  29. Risk metrics and fine tuning of high-frequency trading strategies (2015)
  30. Ambiguity aversion in algorithmic and high frequency trading (2014)
  31. Market making and portfolio liquidation under uncertainty (2014)
  32. Space-time periodic solutions and long-time behavior of solutions to quasi-linear parabolic equations (2001)
  33. Large time asymptotic problems for optimal stochastic control with superlinear cost (2012)
  34. Large time behavior for some nonlinear degenerate parabolic equations (2014)
  35. Matrix analysis (2012)
  36. Derivatives of functions of eigenvalues and eigenvectors for symmetric matrices (2016)
  37. The symmetric eigenvalue problem (1998)
  38. Absolute continuity of eigenvectors of time-varying operators (1974)
  39. Probability: theory and examples (2019)
  40. Geometric bounds for eigenvalues of Markov chains (1991)
  41. Asymptotic properties of non-linear least squares estimators (1969)
  42. Survey of measurable selection theorems: an update (2006)
  43. Linear algebra and its applications (2007)
  44. Differentiable perturbation of unbounded operators (2003)
  45. Choosing roots of polynomials smoothly (1998)
  46. Logarithmic regret bounds for continuous-time average-reward M arkov decision processes (2024) — link
  47. Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex models (2021) — link
  48. Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon (2022)
  49. Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls (2023)
  50. Near-optimal regret bounds for reinforcement learning (2008)
  51. Stochastic differential equations: an introduction with applications (2013)
  52. Exploration-exploitation in MDP s with options (2017)
  53. Statistical inference (2024)
  54. Adventures in stochastic processes (1992)
  55. Convergence of M arkov Processes (2021)
  56. Continuous-time Markov chains: An applications-oriented approach (2012)

0 of 64 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.