Paper: arXiv 2401.06179

Abstract

The published MLP-based DRL in finance has difficulties in learning the dynamics of the environment when the action scale increases. If the buying and selling increase to one thousand shares, the MLP agent will not be able to effectively adapt to the environment. To address this, we designed a CNN agent that concatenates the data from the last ninety days of the daily feature vector to create the CNN input matrix. Our extensive experiments demonstrate that the MLP-based agent experiences a loss corresponding to the initial environment setup, while our designed CNN remains stable, effectively learns the environment, and leads to an increase in rewards.

Complexity vs Empirical Score

  • Math Complexity: 3.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper relies on standard deep learning architectures (MLP/CNN) and basic RL formulations (MDP) with minimal complex derivations, but it details a full trading environment, specific datasets, and reports performance metrics, indicating a high degree of empirical testing.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Address Action Scaling<br>in Finance DRL"] --> B["Input Data:<br>90-day Market Features"]
  B --> C["Methodology:<br>CNN vs MLP Agents"]
  C --> D["Computation:<br>Deep Reinforcement Learning<br>with Scaled Actions"]
  D --> E{"Comparison"}
  E --> F["Outcome: MLP Agent<br>Loss & Instability"]
  E --> G["Outcome: CNN Agent<br>Stability & Reward Increase"]

Cited References (17)

  1. Proximal policy optimization algorithms (2017) — link
  2. Asynchronous methods for deep reinforcement learning (2016)
  3. Imagenet classification with deep convolutional neural networks (2012)
  4. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  5. Deep reinforcement learning in quantitative algorithmic trading: A review (2021) — link
  6. Wharton — link
  7. Performance functions and reinforcement learning for trading systems and portfolios (1998)
  8. Cryptocurrency portfolio management with deep reinforcement learning (2017)
and 9 more
  1. Reinforcement learning: An introduction (2018)
  2. CNNPred: CNN-based stock market prediction using several data sources (2018) — link
  3. Time series analysis: forecasting and control (2015)
  4. Forecasting seasonals and trends by exponentially weighted moving averages (2004)
  5. Evaluating multiple classifiers for stock price direction prediction (2015)
  6. Machine learning for quantitative finance: fast derivative pricing, hedging and fitting (2018)
  7. Financial time series forecasting using support vector machines (2003)
  8. Deep learning with long short-term memory networks for financial market predictions (2018)
  9. CNN-DRL with Shuffled Features in Finance — → 2402.03338

1 of 17 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.