Paper: arXiv 2504.11116

Abstract

We introduce the Pontryagin-Guided Direct Policy Optimization (PG-DPO) framework for high-dimensional continuous-time portfolio choice. Our approach combines Pontryagin’s Maximum Principle (PMP) with backpropagation through time (BPTT) to directly inform neural network-based policy learning, enabling accurate recovery of both myopic and intertemporal hedging demands–an aspect often missed by existing methods. Building on this, we develop the Projected PG-DPO (P-PGDPO) variant, which achieves nearoptimal policies with substantially improved efficiency. P-PGDPO leverages rapidly stabilizing costate estimates from BPTT and analytically projects them onto PMP’s first-order conditions, reducing training overhead while improving precision. Numerical experiments show that PG-DPO matches or exceeds the accuracy of Deep BSDE, while P-PGDPO delivers significantly higher precision and scalability. By explicitly incorporating time-to-maturity, our framework naturally applies to finite-horizon problems and captures horizon-dependent effects, with the long-horizon case emerging as a stationary special case.

Complexity vs Empirical Score

  • Math Complexity: 9.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper relies heavily on advanced continuous-time mathematics like Pontryagin’s Maximum Principle and partial differential equations, warranting a high math score. It provides empirical comparisons and demonstrates the method on high-dimensional problems, though it lacks direct backtest code or live trading data, resulting in a moderate empirical score.

Research Flowchart

  flowchart TD
  A["Research Goal<br>High-dimensional<br>Continuous-time Portfolio Choice"] --> B["Methodology<br>Pontryagin-Guided Direct Policy Optimization<br>(PG-DPO)"]
  B --> C["Key Process<br>Combines Pontryagin's Maximum Principle (PMP)<br>with Backpropagation Through Time (BPTT)"]
  C --> D["Variant<br>Projected PG-DPO (P-PGDPO)<br>Projects costate estimates onto PMP conditions"]
  C --> E["Data/Inputs<br>Continuous-time multi-asset market data<br>Finite/Long-horizon constraints"]
  D --> F["Computation<br>Neural Network Policy Training<br>with reduced overhead & improved precision"]
  E --> F
  F --> G["Outcomes<br>1. Matches/Exceeds Deep BSDE accuracy<br>2. Recovers myopic & hedging demands<br>3. Scalable to high dimensions<br>4. Handles finite-horizon problems naturally"]

Cited References (137)

  1. Theory of asset pricing (2008)
  2. Methods of Mathematical Finance (1998)
  3. Partial differential equations (2022)
  4. Portfolio Selection, the journal of finance. 7 (1) (1952)
  5. A Monte Carlo method for optimal portfolios (2003)
  6. Lifetime portfolio selection by dynamic stochastic programming (1969)
  7. Strategic Asset Allocation (1997)
  8. Optimum Consumption and Portfolio Rules in a Continuous-time Model (1971)
and 129 more
  1. Nonnegative wealth, absence of arbitrage, and feasible consumption plans (1988)
  2. Brownian motion and stochastic calculus (1991)
  3. Stochastic differential equations: an introduction with applications (2013)
  4. Deep Learning for Continuous-time Stochastic Control with Jumps — → 2505.15602
  5. Functional analysis, calculus of variations and optimal control (2013)
  6. The sensitivity of tests of the intertemporal allocation of consumption to near-rational alternatives (1988)
  7. Portfolios for long-term investors (2022)
  8. Perturbation analysis of optimization problems (2013)
  9. Deep splitting method for parabolic PDEs (2021)
  10. DGM: A deep learning algorithm for solving partial differential equations (2018)
  11. Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms (2022)
  12. Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach (2022)
  13. q-Learning in Continuous Time (2023)
  14. Continuous-time mean-variance portfolio selection: A reinforcement learning framework (2020)
  15. Mean–Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study — → 2412.16175
  16. Capital market equilibrium with transaction costs (1986)
  17. Learning equilibrium mean-variance strategy (2023)
  18. Reinforcement learning in continuous time and space: A stochastic control approach (2020)
  19. Pontryagin-Guided Deep Learning for Large-Scale Constrained Dynamic Portfolio Choice (2025)
  20. Factor Learning Portfolio Optimization Informed by Continuous-Time Finance Models (2023)
  21. Monotone (nonlinear) operators in Hilbert space (1962)
  22. Policy Optimization for Continuous Reinforcement Learning (2023)
  23. Parabolic and elliptic systems with VMO coefficients (2009)
  24. Stochastic Approximation and Recursive Algorithms and Applications (2003)
  25. Stochastic approximation: a dynamical systems viewpoint (2008)
  26. Stochastic Flows and Stochastic Differential Equations (1997)
  27. Foundations of Modern Probability (1997)
  28. Measure Theory, Vol. I and II (2007)
  29. Stochastic differential utility (1992)
  30. Linear and quasi-linear equations of parabolic type (1968)
  31. Stochastic Stability of Differential Equations (2012)
  32. Deep learning algorithm to solve portfolio management with proportional transaction cost (2019)
  33. Deep Learning Approximation for Stochastic Control Problems (2016) — link
  34. Deep stochastic optimization in finance (2023)
  35. Deep empirical risk minimization in finance: Looking into the future (2023)
  36. Pontryagin differentiable programming: An end-to-end learning and control framework (2020)
  37. Deep hedging (2019)
  38. Deep optimal stopping (2019)
  39. Deep learning for constrained utility maximisation (2022)
  40. Continuous-time stochastic control and optimization with financial applications (2009)
  41. Solving high-dimensional partial differential equations using deep learning (2018)
  42. Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations (2019)
  43. A regression-based Monte Carlo method to solve backward stochastic differential equations (2005)
  44. Controlled Markov processes and viscosity solutions (2006)
  45. Handbook on systemic risk (2013)
  46. Learning Merton’s Strategies in an Incomplete Market: Recursive Entropy Regularization and Biased Gaussian Exploration — → 2312.11797
  47. Taming the factor zoo: A test of new factors (2020)
  48. Presidential address: Discount rates (2011)
  49. Empirical asset pricing via machine learning (2020)
  50. Autoencoder asset pricing models (2021)
  51. Predicting excess stock returns out of sample: Can anything beat the historical average? (2008)
  52. Optimal number of stock holdings in mutual fund portfolios based on market performance (2005)
  53. Thousands of alpha tests (2021)
  54. Adapted solution of a backward stochastic differential equation (1990)
  55. An intertemporal capital asset pricing model (1973)
  56. Mathematical theory of optimal processes (2018)
  57. Portfolio optimization with feedback strategies based on artificial neural networks — → 2411.09899
  58. Backpropagation through time: what it does and how to do it (1990)
  59. A learning algorithm for continually running fully recurrent neural networks (1989)
  60. Gradient flows: in metric spaces and in the space of probability measures (2008)
  61. $\ $Euclidean, metric, and Wasserstein$\ $ gradient flows: an overview (2017)
  62. Numerical solution of stochastic differential equations with jumps in finance (2010)
  63. A stochastic approximation method (1951)
  64. Approximation by superpositions of a sigmoidal function (1989)
  65. Approximation capabilities of multilayer feedforward networks (1991)
  66. $\ $Euclidean, metric, and Wasserstein$\ $ gradient flows: an overview (2017)
  67. Nonlinear Systems (2002)
  68. Sur les op 'e rations dans les ensembles abstraits et leur application aux 'e quations int 'e grales (1922)
  69. Fixed point theory (2003)
  70. Forward-backward stochastic differential equations and their applications (1999)
  71. Stochastic controls: Hamiltonian systems and HJB equations (2012)
  72. Adapted solution of a backward stochastic differential equation (1990)
  73. A proposal on machine learning via dynamical systems (2017)
  74. DNN expression rate analysis of high-dimensional PDEs: application to option pricing (2022)
  75. Deep backward schemes for high-dimensional nonlinear PDEs (2020)
  76. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations (2019)
  77. Perspectives in flow control and optimization (2002)
  78. Optimization with PDE constraints (2008)
  79. A reduction of imitation learning and structured prediction to no-regret online learning (2011)
  80. Hindsight experience replay (2017)
  81. Multitask learning (1997)
  82. The loss surfaces of multilayer networks (2015)
  83. Nonlinear monotone operators and convex sets in Banach spaces (1965)
  84. Deep learning (2016)
  85. Neuro-dynamic programming: an overview (1995)
  86. Improved Deep Learning Methods for Large-Scale Dynamic Portfolio Choice — → 2501.12600
  87. Continuous-time Portfolio Optimization via Model-based Reinforcement Learning (2024)
  88. Solving consumption and portfolio choice problems: The state variable decomposition method (2010)
  89. Correlation risk and optimal portfolio choice (2010)
  90. Dynamic programming (1966)
  91. A simulation approach to dynamic portfolio choice with an application to learning about return predictability (2005)
  92. Portfolio choice and equity characteristics: Characterizing the hedging demands induced by return predictability (2001)
  93. Predictability and transaction costs: The impact on rebalancing rules and behavior (2000)
  94. Transaction costs and predictability: Some utility cost calculations (1999)
  95. Optimal value and growth tilts in long-horizon portfolios (2011)
  96. Dynamic consumption and portfolio choice with stochastic volatility in incomplete markets (2005)
  97. A multivariate model of strategic asset allocation (2003)
  98. Who should buy long-term bonds? (2001)
  99. Lectures on elliptic and parabolic equations in Sobolev spaces (2024)
  100. Second-order equations with nonnegative characteristic form (2012)
  101. Hypoelliptic Second Order Differential Equations (1967)
  102. Consumption and portfolio decisions when expected returns are time varying (1999)
  103. Sobolev spaces (2003)
  104. Dynamic nonmyopic portfolio behavior (1996)
  105. Portfolio selection in stochastic environments (2007)
  106. Ordinary Differential Equations: Adiwes International Series in Mathematics (2014)
  107. Lifetime portfolio selection under uncertainty: The continuous-time case (1969)
  108. Lifetime portfolio selection by dynamic stochastic programming (1975)
  109. A Pontryagin-Guided Neural Policy Optimization Framework for Merton’s Portfolio Problem — → 2412.13101
  110. Minima of functions of several variables with inequalities as side constraints (1939)
  111. Nonlinear programming (2013)
  112. Numerical optimization (1999)
  113. Nonlinear programming: sequential unconstrained minimization techniques (1990)
  114. Convex optimization (2004)
  115. Optimal portfolio and consumption decisions for a “small investor” on a finite horizon (1987)
  116. Hedging and portfolio optimization under transaction costs: A martingale approach 1 2 (1996)
  117. Optimal investment and consumption with transaction costs (1994)
  118. Dusenberry’s ratcheting of consumption: optimal dynamic consumption and investment given intolerance for any decline in standard of living (1995)
  119. Habit formation: A resolution of the equity premium puzzle (1990)
  120. Intertemporally dependent preferences and the volatility of consumption and wealth (1989)
  121. An intertemporal asset pricing model with stochastic consumption and investment opportunities (1979)
  122. Machine learning for continuous-time finance (2024)
  123. A deep branching solver for fully nonlinear partial differential equations (2024)
  124. Second order parabolic differential equations (1996)
  125. Monte Carlo methods in financial engineering (2004)
  126. Residual policy learning (2018) — link
  127. Residual reinforcement learning for robot control (2019)
  128. AlphaPortfolio: Direct construction through deep reinforcement learning and interpretable AI (2022)
  129. Rational Expectations, Reinforcement Learning, and Market Efficiency (2025)

6 of 137 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.