Onflow: an online portfolio allocation algorithm

Paper: arXiv 2312.05169

Abstract

We introduce Onflow, a reinforcement learning technique that enables online optimization of portfolio allocation policies based on gradient flows. We devise dynamic allocations of an investment portfolio to maximize its expected log return while taking into account transaction fees. The portfolio allocation is parameterized through a softmax function, and at each time step, the gradient flow method leads to an ordinary differential equation whose solutions correspond to the updated allocations. This algorithm belongs to the large class of stochastic optimization procedures; we measure its efficiency by comparing our results to the mathematical theoretical values in a log-normal framework and to standard benchmarks from the ‘old NYSE’ dataset. For log-normal assets, the strategy learned by Onflow, with transaction costs at zero, mimics Markowitz’s optimal portfolio and thus the best possible asset allocation strategy. Numerical experiments from the ‘old NYSE’ dataset show that Onflow leads to dynamic asset allocation strategies whose performances are: a) comparable to benchmark strategies such as Cover’s Universal Portfolio or Helmbold et al. “multiplicative updates” approach when transaction costs are zero, and b) better than previous procedures when transaction costs are high. Onflow can even remain efficient in regimes where other dynamical allocation techniques do not work anymore. Therefore, as far as tested, Onflow appears to be a promising dynamic portfolio management strategy based on observed prices only and without any assumption on the laws of distributions of the underlying assets’ returns. In particular it could avoid model risk when building a trading strategy.

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a sophisticated reinforcement learning algorithm based on gradient flows and stochastic optimization with theoretical analysis, indicating high mathematical complexity. It is tested on both theoretical (log-normal framework) and empirical (NYSE) data, with results compared to established benchmarks, though the summary does not provide detailed implementation code or full backtest metrics.

Research Flowchart

  flowchart TD
  Goal["<b>Research Goal</b><br>Optimize online portfolio allocation<br>with transaction costs"] --> Method["<b>Methodology</b><br>Onflow: Gradient Flow Reinforcement Learning<br>Softmax Parameterization & ODE Updates"]
  Method --> Inputs["<b>Data & Inputs</b><br>1. Log-Normal Assets (Theory)<br>2. 'old NYSE' Dataset (Empirical)"]
  Inputs --> Comp["<b>Computational Process</b><br>Stochastic Optimization via Gradient Flows<br>Maximize Expected Log Return<br>Minimize Transaction Fees"]
  Comp --> Results["<b>Key Findings & Outcomes</b><br>1. Matches Markowitz Optimal (Zero Cost)<br>2. Outperforms Benchmarks (High Cost)<br>3. Robust & Model-Free"]

Cited References (48)

1. Universal Portfolios (1991) — link
2. Note on the Derivatives with Respect to a Parameter of the Solutions of a System of Differential Equations (1919)
3. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python (2020) — link
4. A Deep Deterministic Policy Gradient-based Strategy for Stocks Portfolio Management (2021) — link
5. Reinforcement Learning for Quantitative Trading (2023) — link
6. On adaptive control processes (1959) — link
7. Dynamic time warping (2007)
8. Does reinforcement learning outperform deep learning and traditional portfolio optimization models in frontier and developed financial markets? (2023) — link

0 of 48 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.