Paper: arXiv 2311.05743

Abstract

This study enhances a Deep Q-Network (DQN) trading model by incorporating advanced techniques like Prioritized Experience Replay, Regularized Q-Learning, Noisy Networks, Dueling, and Double DQN. Extensive tests on assets like BTC/USD and AAPL demonstrate superior performance compared to the original model, with marked increases in returns and Sharpe Ratio, indicating improved risk-adjusted rewards. Notably, convolutional neural network (CNN) architectures, both 1D and 2D, significantly boost returns, suggesting their effectiveness in market trend analysis. Across instruments, these enhancements have yielded stable and high gains, eclipsing the baseline and highlighting the potential of CNNs in trading systems. The study suggests that applying sophisticated deep learning within reinforcement learning can greatly enhance automated trading, urging further exploration into advanced methods for broader financial applicability. The findings advocate for the continued evolution of AI in finance.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 9.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced deep reinforcement learning techniques (Prioritized Experience Replay, Dueling/Double DQN, Noisy Networks, CNN architectures) with explicit mathematical formulations, earning a high complexity score. It demonstrates high empirical rigor through extensive backtesting on real financial data (BTC/USD and AAPL), reporting specific metrics like returns and Sharpe Ratios, and comparing performance against baselines.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Enhance DQN Trading Model"] --> B["Methodology:<br>Advanced Techniques"]
  B --> C["Data & Inputs:<br>Historical Market Data<br>BTC/USD & AAPL"]
  C --> D["Computational Process:<br>CNN Architectures<br>1D & 2D Integration"]
  D --> E["Key Findings & Outcomes:<br>Superior Performance<br>Higher Returns & Sharpe Ratio"]
  
  subgraph B
      B1["Prioritized Experience Replay"]
      B2["Regularized Q-Learning"]
      B3["Noisy Networks & Dueling"]
      B4["Double DQN"]
  end

Cited References (25)

  1. Quantitative Trading: How to Build Your Own Algorithmic Trading Business (2021)
  2. High Frequency Trading (2015)
  3. Deep Learning for Portfolio Optimization (2020) — link
  4. Learning Financial Asset-Specific Trading Rules via Deep Reinforcement Learning (2022)
  5. Using Genetic Algorithms to Find Technical Trading Rules (1999)
  6. Generating Trading Rules on the Stock Markets with Genetic Programming (2004)
  7. Mining Associative Classification Rules with Stock Trading Data–A GA-Based Method (2010)
  8. A Reinforcement Learning Based Encoder-Decoder Framework for Learning Stock Trading Rules (2021)
and 17 more
  1. High-dimensional stock portfolio trading with deep reinforcement learning (2022)
  2. Deep reinforcement learning with double q-learning (2016)
  3. Dueling network architectures for deep reinforcement learning (2016)
  4. Noisy networks for exploration (2017) — link
  5. Prioritized experience replay (2015) — link
  6. Human-level control through deep reinforcement learning (2015)
  7. Modeling purposeful adaptive behavior with the principle of maximum causal entropy (2010)
  8. first_page settings Order Article Reprints Open AccessArticle Pore-Level Multiphase Simulations of Realistic Distillation Membranes for Water Desalination (2022)
  9. Perspectives on predictive power of multimodal deep learning: Surprises and future directions (2018)
  10. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care (2018)
  11. Financial trading as a game: A deep reinforcement learning approach (2018) — link
  12. An application of deep reinforcement learning to algorithmic trading (2021)
  13. Application of deep reinforcement learning in stock trading strategies and stock forecasting (2020)
  14. Regularization in reinforcement learning (2011)
  15. Practical deep reinforcement learning approach for stock trading (2018) — link
  16. Adaptive stock trading strategies with deep reinforcement learning methods (2020)
  17. Deep Q-trading. cslt. riit. tsinghua. edu. cn (2017)

0 of 25 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.