Paper: arXiv 2504.05521

Abstract

Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing them as sequential decision-making problems. However, most previous work assesses the performance of only one or two DRL algorithms, making an objective comparison across algorithms difficult. In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a novel application to the task of dynamic hedging. In our experiments, we use the Black-Scholes delta hedge as a baseline and simulate the dataset using a GJR-GARCH(1,1) model. Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi-quadratic penalty. Moreover, MCPG is the only algorithm to outperform the Black-Scholes delta hedge baseline with the allotted computational budget, possibly due to the sparsity of rewards in our environment.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts like GJR-GARCH modeling, risk measures (RSQP), and Bellman equations, while providing rigorous empirical evaluation with code and dataset availability but limited to simulated data without real-world trading implementation.

Research Flowchart

  flowchart TD
  A["Research Goal: Compare performance of eight DRL algorithms in dynamic hedging"] --> B["Data: Simulated dataset using GJR-GARCH(1,1) model"]
  B --> C["Methodology: Apply eight DRL algorithms to hedging problem<br/>(MCPG, PPO, 4x DQL, 2x DDPG)"]
  C --> D["Baseline: Black-Scholes delta hedge"]
  C --> E["Computational Process: Train & evaluate algorithms using Root Semi-Quadratic Penalty metric"]
  D --> E
  E --> F["Key Findings:<br/>1. MCPG & PPO perform best<br/>2. MCPG is the only algorithm to outperform baseline<br/>3. Success linked to sparse reward environment"]
  F --> G["Outcome: Validated MCPG effectiveness for dynamic hedging"]

Cited References (79)

  1. Optimal execution of portfolio transactions (2001)
  2. Distributed Distributional Deterministic Policy Gradients (2018) — link
  3. QLBS : Q -learner in the B lack- S choles (- M erton) worlds (2020)
  4. Advanced Option Pricing and Hedging with Q -Learning: Performance Evaluation of the QLBS Algorithm (2025)
  5. Deterministic policy gradient algorithms (2014)
  6. A Distributional Perspective on Reinforcement Learning (2017) — link
  7. Deep Hedging (2019)
  8. The pricing of options and corporate liabilities (1973)
and 71 more
  1. Gamma and vega hedging using deep distributional reinforcement learning (2023)
  2. Deep Hedging of Derivatives Using Reinforcement Learning (2020)
  3. Deep Hedging of Long-Term Financial Derivatives (2021)
  4. Deep equal risk pricing of financial derivatives with non-translation invariant risk measures (2023)
  5. Equal risk pricing of derivatives with deep hedging (2021)
  6. Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments (2024)
  7. Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning (2023)
  8. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  9. DEEP EXECUTION - VALUE AND POLICY BASED REINFORCEMENT LEARNING FOR TRADING AND BEATING MARKET BENCHMARKS (2019) — link
  10. Deep Reinforcement Learning for Option Replication and Hedging (2020)
  11. Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret (2020)
  12. Addressing Function Approximation Error in Actor-Critic Methods (2018)
  13. On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks (1993) — link
  14. Deep Learning (2016)
  15. QLBS: Q-Learner in the B lack- S choles(- M erton) Worlds (2020) — link
  16. Recent Advances in Reinforcement Learning in Finance (2023) — link
  17. A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle (1989)
  18. A reinforcement learning extension to the A lmgren- C hriss framework for optimal trade execution (2014) — link
  19. A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993)
  20. No-Transacion Band Network: A Neural Network Architecture for Efficient Deep Hedging (2023)
  21. Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019) — link
  22. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  23. Cryptocurrency portfolio management with deep reinforcement learning. (2017)
  24. Portfolio Management using Reinforcement Learning (2016)
  25. A new interpretation of information rate (1956)
  26. Online portfolio selection: A survey (2014)
  27. OLPS : A Toolbox for On-Line Portfolio Selection (2016)
  28. Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
  29. Continuous control with deep reinforcement learning (2016)
  30. Optimal Liquidation with Deep Reinforcement Learning (2019)
  31. An End-to-End Optimal Trade Execution Framework based on Proximal Policy Optimization (2020) — link
  32. Practical Deep Reinforcement Learning Approach for Stock Trading (2018)
  33. Computational Finance: A Scientific Perspective (2000) — link
  34. Portfolio Selection (1952)
  35. The Optimization of a Quadratic Function Subject to Linear Constraints (1956)
  36. Portfolio Selection: Efficient Diversification of Investments (1959)
  37. Deep reinforcement learning for option pricing and hedging under dynamic expectile risk measures (2023) — link
  38. Convergence of Q -learning: a simple proof (2001)
  39. On estimating the expected return on the market: An exploratory investigation (1980)
  40. Option pricing when underlying stock returns are discontinuous (1976) — link
  41. Empirical deep hedging (2023) — link
  42. Playing A tari with Deep Reinforcement Learning (2013) — link
  43. Human-level control through deep reinforcement learning (2015)
  44. Deep Hedging with Market Impact — → 2402.13326
  45. Deep Reinforcement Learning Algorithms for Financial Decision-Making (2024)
  46. Kelly C riterion for Multivariate Portfolios: A Model-Free Approach (2014)
  47. Double Deep Q -learning for Optimal Execution (2021)
  48. OpenAI Spinning Up: Deep Deterministic Policy Gradient (2024)
  49. OpenAI Spinning Up: Proximal Policy Optimization (2024)
  50. OpenAI Spinning Up: Trust Region Policy Optimization (2024)
  51. An intelligent financial portfolio trading strategy using deep Q-learning (2020) — link
  52. Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey (2019) — link
  53. Trust Region Policy Optimization (2015) — link
  54. Proximal Policy Optimization Algorithms (2017) — link
  55. Hedging Beyond the Mean: A Distributional Reinforcement Learning Perspective for Hedging Portfolios with Structured Products (2024) — link
  56. Stochastic Calculus for Finance II : Continuous-Time Models (2004)
  57. Deterministic policy gradient algorithms (2014)
  58. Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization (2023) — link
  59. A parsimonious neural network approach to solve portfolio optimization problems without using dynamic programming (2023) — link
  60. Reinforcement Learning: An Introduction (2018 )
  61. Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999) — link
  62. Deep hedging under rough volatility (2021)
  63. The K elly C riterion in Blackjack, Sports Betting and the Stock Market (2006)
  64. Deep Reinforcement Learning with Double Q -learning (2015) — link
  65. Option hedging with risk averse reinforcement learning (2021) — link
  66. Dueling network architectures for deep reinforcement learning (2016)
  67. An Asymptotic Analysis of an Optimal Hedging Model for Option Pricing with Transaction Costs (2002) — link
  68. Robust Risk-Aware Option Hedging (2023) — link
  69. Optimal Trade Execution Based on Deep Deterministic Policy Gradient (2020)
  70. Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization (2019) — link
  71. Deep Reinforcement Learning for Trading (2020)

1 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.