Paper: arXiv 2511.12120

Authors: Hongyang Yang, Xiao-Yang Liu, Shan Zhong, Anwar Walid

Abstract

Stock trading strategies play a critical role in investment. However, it is challenging to design a profitable strategy in a complex and dynamic stock market. In this paper, we propose an ensemble strategy that employs deep reinforcement schemes to learn a stock trading strategy by maximizing investment return. We train a deep reinforcement learning agent and obtain an ensemble trading strategy using three actor-critic based algorithms: Proximal Policy Optimization (PPO), Advantage Actor Critic (A2C), and Deep Deterministic Policy Gradient (DDPG). The ensemble strategy inherits and integrates the best features of the three algorithms, thereby robustly adjusting to different market situations. In order to avoid the large memory consumption in training networks with continuous action space, we employ a load-on-demand technique for processing very large data. We test our algorithms on the 30 Dow Jones stocks that have adequate liquidity. The performance of the trading agent with different reinforcement learning algorithms is evaluated and compared with both the Dow Jones Industrial Average index and the traditional min-variance portfolio allocation strategy. The proposed deep ensemble strategy is shown to outperform the three individual algorithms and two baselines in terms of the risk-adjusted return measured by the Sharpe ratio. This work is fully open-sourced at \href{“https://github.com/AI4Finance-Foundation/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020"}{“GitHub”}.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper uses advanced deep reinforcement learning algorithms (PPO, A2C, DDPG) and MDP formulation, indicating high mathematical complexity, and is backtest-ready with specific data (30 Dow Jones stocks), open-source code, and risk-adjusted metrics like Sharpe ratio.

Research Flowchart

  flowchart TD
  A["Research Goal: Create Profitable<br>Stock Trading Strategy"] --> B["Data: 30 Dow Jones Stocks<br>Load-on-Demand Processing"]
  B --> C["Method: Deep Reinforcement Learning<br>Ensemble of 3 Actor-Critic Algorithms"]
  C --> D{"Computational Process: Train Agents"}
  D --> E["PPO"]
  D --> F["A2C"]
  D --> G["DDPG"]
  E & F & G --> H["Key Outcome: Ensemble Strategy<br>Best Features Integrated"]
  H --> I["Result: Outperformed<br>Baselines & Individual Agents<br>(Highest Sharpe Ratio)"]

Cited References (77)

  1. Value Investing: A Look at the Benjain Graham Approach (1996)
  2. SCREENING FOR STOCKS USING THE JOHN TEMPLETON VALUE APPROACH (2001)
  3. The Use of the P/E Ratio to Stock Valuation (2012)
  4. Ls-decomposition for robust recovery of sensory big data (2017)
  5. Parallel Architectures for Improving the Performance of a GA Based Trading System (2012)
  6. The implied cost of capital: A new approach (2011)
  7. Security Analysis, Fifth Edition (1988)
  8. The Intelligent Investor: a Book of Practical Counsel (1959)
and 69 more
  1. Regression-Based Earnings Forecasts (2013)
  2. Deep Learning for Multivariate Financial Time Series (2015)
  3. A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
  4. Investigation Into The Effectiveness Of Long Short Term Memory Networks For Stock Price Prediction (2016) — link
  5. Improving Factor-Based Quantitative Investing by Forecasting Company Fundamentals (2017) — link
  6. The Elements of Statistical Learning, 2nd Edition (2009)
  7. R: A Language and Environment for Statistical Computing.
  8. An Introduction to Statistical Learning with Applications in R (2013)
  9. Least Angle Regression
  10. Classification and Regression by randomForest (2002)
  11. Generalized Boosted Models: A guide to the gbm package (2007)
  12. Mean-Variance Investing (August 10, 2012)
  13. MATLAB Financial Toolbox: Portfolio Object (2017)
  14. Digesting Anomalies: An Investment Approach (2014)
  15. Fourth-order tensors with multidimensional discrete transforms (2017) — link
  16. Our codes
  17. Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach (2010)
  18. Fuzzy adaptive decision-making for boundedly rational traders in speculative stock markets (2010)
  19. Testing different reinforcement learning configurations for financial trading: introduction and applications (2012)
  20. Dynamic programming and optimal control (1995)
  21. Large-scale study of curiosity-driven learning (2018)
  22. Reinforcement learning for control: Performance, stability, and deep approximators (2018)
  23. Application of deep reinforcement learning on automated stock trading (2019)
  24. Revisiting the performance of MACD and RSI oscillators (2014)
  25. Reinforcement learning in stock trading (2020)
  26. Deep direct reinforcement learning for financial signal representation and trading (2016)
  27. OpenAI Baselines (2017)
  28. An empirical investigation of the challenges of real-world reinforcement learning (2020)
  29. Practical machine learning approach to capture the scholar data driven alpha in AI industry (2019)
  30. Reinforcement learning in financial markets - a survey (2018)
  31. Performance of the average directional index as a market timing tool for the most actively traded USD based currency pairs (2018)
  32. Stable baselines (2018)
  33. Improving financial trading decisions using deep Q-learning: predicting the number of shares, action strategies, and transfer learning (2018)
  34. Cryptocurrency portfolio management with deep reinforcement learning (2017)
  35. An intelligent hybrid trading system for discovering trading rules for the futures market using rough sets and genetic algorithms (2017)
  36. Actor-critic algorithms (2001)
  37. Skulls, financial turbulence, and risk management (2010)
  38. Learning to Trade with Deep Actor Critic Methods (2018)
  39. Adversarial deep reinforcement learning in portfolio management (2018)
  40. Continuous control with deep reinforcement learning (2015)
  41. Deep reinforcement learning for intelligent transportation systems (2018)
  42. Transform-based multilinear dynamical system for tensor time series analysis (2018)
  43. Commodity Channel index: evaluation of trading rule of agricultural Commodities (2016)
  44. Portfolio selection (1952)
  45. Asynchronous methods for deep reinforcement learning (2016)
  46. Learning to trade via direct reinforcement (2001)
  47. Optimal asset allocation using adaptive dynamic programming (1996)
  48. Enhancing Q-learning for optimal asset allocation. (1997)
  49. Trust region policy optimization (2015)
  50. Proximal policy optimization algorithms (2017) — link
  51. Standard & Poor’s Compustat (2015)
  52. The Sharpe ratio (1994)
  53. Reinforcement learning: an introduction (1998)
  54. Policy gradient methods for reinforcement learning with function approximation (2000)
  55. Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation (2018)
  56. Training agent for first-person shooter game with actor-critic curriculum learning (2017)
  57. A practical machine learning approach for dynamic stock recommendation — → 2511.12129
  58. Trading strategies to exploit blog and news sentiment. (2010)
  59. Online portfolio selection strategy based on combining experts’ advice (2016)
  60. Deep reinforcement learning for trading (2019)
  61. Quantifying ESG alpha using scholar big data: An automated machine learning approach (2020)
  62. Multi-agent deep reinforcement learning for liquidation strategy analysis (2019)
  63. DP-LSTM: Differential privacy-inspired LSTM for stock prediction using financial news (2019)
  64. Expected Returns: An Investor’s Guide to Harvesting Market Rewards (2012)
  65. Practical deep reinforcement learning approach for stock trading (2018)
  66. Human-level control through deep reinforcement learning (2015) — link
  67. Deterministic Policy Gradient Algorithms (2014)
  68. Financial Trading as a Game: A Deep Reinforcement Learning Approach (2018)
  69. Options futures and other derivatives (2003)

1 of 77 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.