Paper: arXiv 2406.08013

Abstract

Deep reinforcement learning (DRL) is a well-suited approach to financial decision-making, where an agent makes decisions based on its trading strategy developed from market observations. Existing DRL intraday trading strategies mainly use price-based features to construct the state space. They neglect the contextual information related to the position of the strategy, which is an important aspect given the sequential nature of intraday trading. In this study, we propose a novel DRL model for intraday trading that introduces positional features encapsulating the contextual information into its sparse state space. The model is evaluated over an extended period of almost a decade and across various assets including commodities and foreign exchange securities, taking transaction costs into account. The results show a notable performance in terms of profitability and risk-adjusted metrics. The feature importance results show that each feature incorporating contextual information contributes to the overall performance of the model. Additionally, through an exploration of the agent’s intraday trading activity, we unveil patterns that substantiate the effectiveness of our proposed model.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced deep reinforcement learning techniques with specific architectural choices (like sparse state spaces) and discusses sequential decision-making theory, indicating high mathematical complexity. It demonstrates high empirical rigor by evaluating the model over a decade across multiple asset classes, accounting for transaction costs, and performing feature importance analysis.

Research Flowchart

  flowchart TD
  A["Research Goal: Enhance DRL for Intraday Trading"] --> B["Methodology: DRL Model with Positional State"]
  B --> C["Data: Multi-Asset History<br/>with Transaction Costs"]
  C --> D["Process: Sparse State Space Construction<br/>(Price + Position Context)"]
  D --> E["Process: Deep Reinforcement Learning<br/>Agent Training & Decision Making"]
  E --> F["Key Findings: High Profitability &<br/>Risk-Adjusted Returns"]
  F --> G["Outcome: Validated Positional<br/>Feature Importance & Trading Patterns"]

Cited References (51)

  1. The index of general nonlinear D A E S (1995)
  2. Clinical implications of dysregulated cytokine production (2000) — link
  3. Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations (1995)
  4. Algorithms for C omputer A lgebra (1992)
  5. Conductive P olymers (1981)
  6. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae (1978)
  7. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon (2002)
  8. Mixing induced CP asymmetries in inclusive B decays (1997) — link
and 43 more
  1. deep SIP : deep learning of S upernova I a P arameters (2020) — link
  2. Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing (2019) — link
  3. Reinforcement learning: An introduction (2018)
  4. Inside the black box: A simple guide to quantitative and high frequency trading (2013)
  5. Learning dexterous in-hand manipulation (2020)
  6. Mastering the game of go without human knowledge (2017)
  7. Human-level control through deep reinforcement learning (2015)
  8. Deep reinforcement learning for autonomous driving: A survey (2021)
  9. Deep learning (2016)
  10. Efficient capital markets: A review of theory and empirical work (1970)
  11. Time series momentum (2012) — link
  12. Cross-sectional and time-series determinants of momentum returns (2002)
  13. Mean reversion in stock prices: Evidence and implications (1988) — link
  14. Intraday market predictability: A machine learning approach (2023) — link
  15. Intraday Stock Predictability Everywhere (2023)
  16. Intraday return predictability in the cryptocurrency markets: Momentum, reversal, or both (2022) — link
  17. Intraday patterns in the cross-section of stock returns (2010)
  18. Deep direct reinforcement learning for financial signal representation and trading (2016) — link
  19. Learning to trade via direct reinforcement (2001) — link
  20. Financial trading as a game: A deep reinforcement learning approach (2018)
  21. Effect of labeling algorithms on financial performance metrics (2022)
  22. Universal features of price formation in financial markets: perspectives from deep learning (2019) — link
  23. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  24. A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
  25. Deep robust reinforcement learning for practical algorithmic trading (2019) — link
  26. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  27. Imitation Learning for Financial Applications (2023)
  28. Proximal policy optimization algorithms (2017)
  29. Deep reinforcement learning for trading (2020) — link
  30. Enhancing time-series momentum strategies using deep neural networks (2019) — link
  31. Statistical modeling of high-frequency financial data (2011) — link
  32. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  33. High-dimensional continuous control using generalized advantage estimation (2015)
  34. Continuous futures contracts methodology for backtesting (2020)
  35. Pseudomathematics and financial charlatanism: The effects of backtest over fitting on out-of-sample performance (2014) — link
  36. Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book (2023) — link
  37. Adam: A method for stochastic optimization (2014)
  38. Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
  39. Does algorithmic trading improve liquidity? (2011) — link
  40. Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs (2018) — link
  41. Forecasting East Asian indices futures via a novel hybrid of wavelet-PCA denoising and artificial neural network models (2016) — link
  42. Bitcoin transaction strategy construction based on deep reinforcement learning (2021) — link
  43. Evolving deep gated recurrent unit using improved marine predator algorithm for profit prediction based on financial accounting information system (2023) — link

0 of 51 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.