Paper: arXiv 2512.10913

Authors: Mohammad Rezoanul Hoque, Md Meftahul Ferdaus, M. Kabir Hassan

Abstract

Reinforcement learning (RL) is an innovative approach to financial decision making, offering specialized solutions to complex investment problems where traditional methods fail. This review analyzes 167 articles from 2017–2025, focusing on market making, portfolio optimization, and algorithmic trading. It identifies key performance issues and challenges in RL for finance. Generally, RL offers advantages over traditional methods, particularly in market making. This study proposes a unified framework to address common concerns such as explainability, robustness, and deployment feasibility. Empirical evidence with synthetic data suggests that implementation quality and domain knowledge often outweigh algorithmic complexity. The study highlights the need for interpretable RL architectures for regulatory compliance, enhanced robustness in nonstationary environments, and standardized benchmarking protocols. Organizations should focus less on algorithm sophistication and more on market microstructure, regulatory constraints, and risk management in decision-making.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper reviews complex RL algorithms and theoretical finance concepts, indicating substantial mathematical density, but its empirical rigor is limited as it analyzes 167 articles using synthetic data rather than presenting new backtests or real-world datasets.

Research Flowchart

  flowchart TD
  A["Research Goal: Review RL in Finance<br/>(2017-2025, n=167)"] --> B{"Systematic Literature Review"}
  B --> C["Topics Analyzed<br/>Market Making | Portfolio Optimization | Algorithmic Trading"]
  C --> D["Computational Synthesis<br/>Performance Metrics vs. Challenges"]
  D --> E["Key Findings & Outcomes"]
  E --> F["Framework & Recommendations<br/>Focus: Microstructure & Risk over Complexity"]

Cited References (124)

  1. A Review of Reinforcement Learning in Financial Markets (2024)
  2. How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda (2022)
  3. Cryptocurrency trading strategy based on deep reinforcement learning (2024)
  4. Reinforcement learning for market making: A theoretical framework — → 2410.14504
  5. Deep reinforcement learning for portfolio optimization with ESG constraints (2024)
  6. Risk-aware reinforcement learning for financial decision making (2023)
  7. Multi-agent reinforcement learning for competitive trading (2023)
  8. Optimal execution with reinforcement learning under market impact (2024)
and 116 more
  1. Explainable reinforcement learning for financial applications (2023)
  2. Regulatory challenges for AI in financial markets (2024)
  3. Safe exploration in financial reinforcement learning (2023)
  4. Quantum reinforcement learning for portfolio optimization (2024)
  5. ESG-aware reinforcement learning for sustainable investing (2023)
  6. Central bank digital currencies and reinforcement learning (2024)
  7. Blockchain-based reinforcement learning for decentralized finance (2023)
  8. Edge computing for high-frequency trading with reinforcement learning (2024)
  9. Validation frameworks for reinforcement learning in finance (2023)
  10. Standardization needs for financial AI systems (2024)
  11. International coordination of AI regulation in finance (2023)
  12. Industry best practices for reinforcement learning deployment (2024)
  13. Machine Learning (1997)
  14. Foundations of Machine Learning (2018)
  15. Reinforcement Learning: An Introduction (2018)
  16. Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case (1969)
  17. Portfolio Selection (1952)
  18. Options, Futures, and Other Derivatives (2017)
  19. The Pricing of Options and Corporate Liabilities (1973)
  20. Analysis of Financial Time Series (2010)
  21. Dynamic Programming (1957)
  22. Dynamic Programming and Optimal Control (2012)
  23. Q-learning (1992)
  24. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning (1992)
  25. Actor-Critic Algorithms (2000)
  26. Human-level Control through Deep Reinforcement Learning (2015)
  27. Proximal Policy Optimization Algorithms (2017) — link
  28. Continuous Control with Deep Reinforcement Learning (2015) — link
  29. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018)
  30. A Comprehensive Survey on Safe Reinforcement Learning (2015)
  31. Challenges of Real-World Reinforcement Learning (2019) — link
  32. Deep Learning (2016)
  33. Learning to trade via direct reinforcement (2001)
  34. Forecasting stock prices from the limit order book using convolutional neural networks (2017)
  35. A review of the past and future of electronic trading and its impact on market structure (2013)
  36. Towards automated trading based on reinforcement learning (2018)
  37. Deep learning for finance: deep portfolios (2017)
  38. … and the cross-section of expected returns (2016)
  39. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  40. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown (2017)
  41. Towards a rigorous science of interpretable machine learning (2017) — link
  42. High-frequency trading (2017)
  43. Explainable reinforcement learning: a survey (2020)
  44. Algorithmic and high-frequency trading (2015)
  45. Optimal execution of portfolio transactions (2001)
  46. Market making via reinforcement learning (2018)
  47. Quantitative risk management: concepts, techniques and tools-revised edition (2015)
  48. Modern perspectives on reinforcement learning in finance (2019)
  49. Machine learning for risk management (2020)
  50. The probability of backtest overfitting (2014)
  51. Advances in financial machine learning (2018)
  52. High-frequency trading: a practical guide to algorithmic strategies and trading systems (2013)
  53. Quantitative equity investing: techniques and strategies (2010)
  54. Analysis of financial time series (2005)
  55. Survivorship bias in performance studies (1992)
  56. A sparse model of demand for news: Evidence from the options market (2019)
  57. The black swan: The impact of the highly improbable (2007)
  58. A theory of learning from different domains (2010)
  59. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  60. Dueling network architectures for deep reinforcement learning (2016)
  61. Rainbow: Combining improvements in deep reinforcement learning (2018)
  62. A distributional perspective on reinforcement learning (2017)
  63. Trust region policy optimization (2015)
  64. Asynchronous methods for deep reinforcement learning (2016)
  65. Addressing function approximation error in actor-critic methods (2018)
  66. Deep reinforcement learning in a handful of trials using probabilistic dynamics models (2018)
  67. Neural network dynamics for model-based control (2018)
  68. Dyna, an integrated architecture for learning, planning, and reacting (1991)
  69. Multi-agent actor-critic for mixed cooperative-competitive environments (2017)
  70. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning (2018)
  71. Counterfactual multi-agent policy gradients (2018)
  72. Learning multi-level hierarchies with hindsight (2019)
  73. FeUdal networks for hierarchical reinforcement learning (2017)
  74. The option-critic architecture (2017)
  75. Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading (2020)
  76. Deep learning for portfolio optimization (2020)
  77. Reinforcement learning in economics and finance (2021)
  78. Deep robust reinforcement learning for practical algorithmic trading (2019)
  79. Bridging the gap between Markowitz planning and deep reinforcement learning (2021)
  80. Deep reinforcement learning for multi-period portfolio optimization (2021)
  81. An application of deep reinforcement learning to algorithmic trading (2021)
  82. CEFi vs. DeFi–comparing centralized to decentralized finance (2021) — link
  83. QLBS: Q-learner in the Black-Scholes (-Merton) worlds (2020)
  84. Reinforcement learning applied to Forex trading (2018)
  85. Modelling stock markets by multi-agent reinforcement learning (2021)
  86. A multi-agent and machine learning based online adaptive energy management system for plug-in hybrid electric vehicles (2020)
  87. Multi-agent deep reinforcement learning for portfolio management (2021)
  88. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  89. Recent advances in reinforcement learning in finance (2023)
  90. Machine learning for trading (2017)
  91. Deep robust reinforcement learning for practical algorithmic trading (2019)
  92. Deep hedging (2019)
  93. Modelling stock markets by multi-agent reinforcement learning (2021)
  94. Recent advances in reinforcement learning in finance (2023)
  95. Reinforcement learning applied to Forex trading (2018)
  96. Using deep learning to detect price change indications in financial markets (2017)
  97. Network structure and systemic risk in banking systems (2010)
  98. Handbook of financial risk management (2020)
  99. Digital finance and FinTech: current research and future research directions (2017)
  100. Deep learning with long short-term memory networks for financial market predictions (2018)
  101. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019 (2020)
  102. Stock price prediction via discovering multi-frequency trading patterns (2017)
  103. Deep transformer models for time series forecasting: The influenza prevalence case (2020) — link
  104. Time-series generative adversarial networks (2019)
  105. Temporal relational ranking for stock prediction (2019)
  106. Learning to reinforcement learn (2016) — link
  107. Empirical market microstructure: The institutions, economics, and econometrics of securities trading (2007)
  108. Optimal control of execution costs (1998)
  109. Value at risk: the new benchmark for managing financial risk (2007)
  110. Optimal execution with nonlinear impact functions and trading-enhanced risk (2003)
  111. Limit order books (2013)
  112. The implementation shortfall: Paper versus reality (1988)
  113. Asset Pricing and Machine Learning: A critical review (2022) — link
  114. Behavioral finance: Evolution from the classical theory and remarks (2024) — link
  115. The evolution of stock market efficiency over time: A survey of the empirical literature (2011) — link
  116. From Econometrics to Machine Learning: Transforming Empirical Asset Pricing (2025)

1 of 124 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.