Paper: arXiv 2512.09224

Authors: Yuling Max Chen, Bin Li, David Saunders

Abstract

Revisiting the continuous-time Mean-Variance (MV) Portfolio Optimization problem, we model the market dynamics with a jump-diffusion process and apply Reinforcement Learning (RL) techniques to facilitate informed exploration within the control space. We recognize the time-inconsistency of the MV problem and adopt the time-inconsistent control (TIC) approach to analytically solve for an exploratory equilibrium investment policy, which is a Gaussian distribution centered on the equilibrium control of the classical MV problem. Our approach accounts for time-inconsistent preferences and actions, and our equilibrium policy is the best option an investor can take at any given time during the investment period. Moreover, we leverage the martingale properties of the equilibrium policy, design a RL model, and propose an Actor-Critic RL algorithm. All of our RL model parameters converge to the corresponding true values in a simulation study. Our numerical study on 24 years of real market data shows that the proposed RL model is profitable in 13 out of 14 tests, demonstrating its practical applicability in real world investment.

Complexity vs Empirical Score

  • Math Complexity: 9.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper features dense advanced mathematics, including stochastic control theory, jump-diffusion processes, and equilibrium solutions, while also demonstrating strong empirical backing through a simulation study and backtesting on 24 years of real market data.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Optimize MV Portfolio with Jumps & RL"] --> B
  subgraph B ["Methodology"]
      B1["Model: Jump-Diffusion<br>Market Dynamics"] --> B2["Solve: Time-Inconsistent<br>Equilibrium Policy"]
      B2 --> B3["Design: Actor-Critic<br>RL Algorithm"]
  end
  B --> C{"Data & Computation"}
  C --> D["Simulation Study:<br>Convergence of RL Parameters"]
  C --> E["Real Market Data<br>24 Years Equity Data"]
  E --> F["Performance:<br>13/14 Tests Profitable"]
  D --> F
  F --> G["Outcome:<br>Verified Equilibrium Policy &<br>Practical Investment Tool"]

Cited References (47)

  1. Continuous-time mean–variance portfolio selection: A reinforcement learning framework (2020)
  2. Learning equilibrium mean-variance strategy (2023)
  3. Reinforcement Learning for Jump-Diffusions — → 2405.16449
  4. Reinforcement learning in continuous time and space: A stochastic control approach (2020)
  5. The reinforcement learning kelly strategy (2022)
  6. Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics (2025) — link
  7. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms (2022)
  8. q-Learning in continuous time (2023)
and 39 more
  1. Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach (2022) — link
  2. Option pricing when underlying stock returns are discontinuous (1976)
  3. The crash of ‘87: was it expected? T he evidence from options markets (1991)
  4. The surprise element: jumps in interest rates (2002)
  5. Portfolio selection (1952)
  6. Portfolio optimization in practice (1992)
  7. Continuous-time mean-variance portfolio selection: A stochastic LQ framework (2000)
  8. Asset and liability management under a continuous-time mean–variance optimization framework (2006)
  9. Continuous-time portfolio selection with liability: Mean–variance model and stochastic LQ approach (2008)
  10. Portfolio selection problems with markowitz’s mean–variance framework: a review of literature (2018)
  11. A comprehensive review of deterministic models and applications for mean-variance portfolio optimization (2019)
  12. Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching (2024) — link
  13. Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market (2024)
  14. A stochastic reinforcement learning algorithm for learning real-valued functions (1990)
  15. Optimal stopping, free boundary, and American option in a jump-diffusion model (1997)
  16. Optimizing venture capital investments in a jump diffusion model (2008)
  17. Optimal stopping and perpetual options for L 'e vy processes (2002)
  18. Jump diffusion option valuation in discrete time (1993)
  19. A jump-diffusion model for option pricing under fuzzy environments (2009)
  20. Numerical valuation of European and American options under K ou’s jump-diffusion model (2008)
  21. Numerical solution of two asset jump diffusion models for option valuation (2008)
  22. On the numerical evaluation of option prices in jump diffusion processes (2007)
  23. Jump-diffusion processes: Volatility smile fitting and numerical methods for option pricing (2000)
  24. Calibration and hedging under jump diffusion (2006)
  25. Optimal hedged portfolios: the case of jump-diffusion risks (1993)
  26. Approximate hedging of options under jump-diffusion processes (2015)
  27. Bayesian reinforcement learning (2012)
  28. Bayesian reinforcement learning: A survey (2015)
  29. Stochastic differential games for optimal investment problems in a Markov regime-switching jump-diffusion market (2022)
  30. Optimal portfolio and consumption for a Markovian regime-switching jump-diffusion process (2021)
  31. Portfolio optimization for jump-diffusion risky assets with regime switching: A time-consistent approach. (2022)
  32. Dynamic control of the investment portfolio in the jump-diffusion financial market with regime switching (2005)
  33. Using simulation studies to evaluate statistical methods (2019)
  34. Time-inconsistent control theory with finance applications (2021)
  35. Applied stochastic control of jump diffusions (2005)
  36. Filtering methods (2011)
  37. Financial modelling with jump processes (2003)
  38. A simplex method for function minimization (1965)
  39. Investment Science (2013)

1 of 47 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.