Paper: arXiv 2410.23294

Abstract

In recent years, the popularity of artificial intelligence has surged due to its widespread application in various fields. The financial sector has harnessed its advantages for multiple purposes, including the development of automated trading systems designed to interact autonomously with markets to pursue different aims. In this work, we focus on the possibility of recognizing and leveraging intraday price patterns in the Foreign Exchange market, known for its extensive liquidity and flexibility. Our approach involves the implementation of a Reinforcement Learning algorithm called Fitted Natural Actor-Critic. This algorithm allows the training of an agent capable of effectively trading by means of continuous actions, which enable the possibility of executing orders with variable trading sizes. This feature is instrumental to realistically model transaction costs, as they typically depend on the order size. Furthermore, it facilitates the integration of risk-averse approaches to induce the agent to adopt more conservative behavior. The proposed approaches have been empirically validated on EUR-USD historical data.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs a sophisticated Reinforcement Learning algorithm (Fitted Natural Actor-Critic) involving continuous action spaces, risk aversion modeling, and natural gradient theory, indicating high mathematical complexity. It is validated on real historical EUR-USD data with realistic transaction cost modeling, though it lacks specific code or detailed performance metrics in the excerpt, suggesting moderate empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Identify & leverage intraday FX patterns<br>to optimize automated trading"] --> B["Methodology: Fitted Natural Actor-Critic<br>Reinforcement Learning with continuous actions"]
  B --> C["Data & Inputs<br>EUR-USD historical data"]
  C --> D["Key Features Implemented"]
  D --> E["Computational Process<br>Agent training & execution"]
  subgraph D ["Key Features Implemented"]
      F["Size-dependent fees<br>Realistic transaction costs"]
      G["Risk-aversion modeling<br>Conservative trading behavior"]
  end
  E --> H["Key Findings & Outcomes"]
  H --> I["Effective trading strategy<br>trained on intraday patterns"]
  H --> J["Realistic cost modeling<br>via variable order sizes"]
  H --> K["Risk-averse agent behavior<br>validated empirically"]

Cited References (36)

  1. Policy gradient methods for reinforcement learning with function approximation (1999)
  2. Reinforcement learning: An introduction (2018)
  3. Soft actor-critic algorithms and applications (2018) — link
  4. Continuous deep q-learning with model-based acceleration (2016)
  5. Learning to trade via direct reinforcement (2001)
  6. Fitted natural actor-critic: A new algorithm for continuous state-action MDPs (2008)
  7. Natural actor-critic (2008)
  8. Information-geometric optimization algorithms: A unifying picture via invariance principles (2017)
and 28 more
  1. Playing atari with deep reinforcement learning (2013) — link
  2. Deep reinforcement learning for active high frequency trading (2021) — link
  3. Markov decision processes: discrete stochastic dynamic programming (2014)
  4. Risk-averse trust region optimization for reward-volatility reduction (2021)
  5. Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning (2020) — link
  6. Robust forex trading with deep q network (dqn) (2019)
  7. Trust region policy optimization (2015)
  8. A deep reinforcement learning approach for automated cryptocurrency trading (2019)
  9. Model learning for robot control: a survey (2011)
  10. Optimizing market making using multi-agent reinforcement learning (2018) — link
  11. Automated trading systems statistical and machine learning methods and hardware implementation: a survey (2019)
  12. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown (2017)
  13. Foreign exchange trading: A risk-averse batch reinforcement learning approach (2020)
  14. Tree-based batch mode reinforcement learning (2005)
  15. Learning FX trading strategies with FQI and persistent actions (2021)
  16. HistData.com
  17. Xgboost: A scalable tree boosting system (2016)
  18. The truncated normal distribution (2014)
  19. A comprehensive survey on safe reinforcement learning (2015)
  20. Risk-Averse Optimization of Reward-based Coherent Risk Measures (2023)
  21. Markov decision processes with average-value-at-risk criteria (2011)
  22. Addressing Non-Stationarity in FX Trading with Online Model Selection of Offline RL Experts (2022)
  23. Reinforcement learning applied to Forex trading (2018)
  24. Financial trading as a game: A deep reinforcement learning approach (2018) — link
  25. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  26. Proximal policy optimization algorithms (2017) — link
  27. Continuous control with deep reinforcement learning (2015) — link
  28. Skulls, financial turbulence, and risk management (2010)

0 of 36 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.