Alpha^2: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning

Paper: arXiv 2406.16505

Abstract

Alphas are pivotal in providing signals for quantitative trading. The industry highly values the discovery of formulaic alphas for their interpretability and ease of analysis, compared with the expressive yet overfitting-prone black-box alphas. In this work, we focus on discovering formulaic alphas. Prior studies on automatically generating a collection of formulaic alphas were mostly based on genetic programming (GP), which is known to suffer from the problems of being sensitive to the initial population, converting to local optima, and slow computation speed. Recent efforts employing deep reinforcement learning (DRL) for alpha discovery have not fully addressed key practical considerations such as alpha correlations and validity, which are crucial for their effectiveness. In this work, we propose a novel framework for alpha discovery using DRL by formulating the alpha discovery process as program construction. Our agent, $\text{“Alpha”}^2$, assembles an alpha program optimized for an evaluation metric. A search algorithm guided by DRL navigates through the search space based on value estimates for potential alpha outcomes. The evaluation metric encourages both the performance and the diversity of alphas for a better final trading strategy. Our formulation of searching alphas also brings the advantage of pre-calculation dimensional analysis, ensuring the logical soundness of alphas, and pruning the vast search space to a large extent. Empirical experiments on real-world stock markets demonstrates $\text{“Alpha”}^2$’s capability to identify a diverse set of logical and effective alphas, which significantly improves the performance of the final trading strategy. The code of our method is available at https://github.com/x35f/alpha2.

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces advanced mathematical concepts like program construction and dimensional analysis for alpha discovery, while also demonstrating high empirical rigor through real-world stock market experiments and a publicly available codebase for backtesting.

Research Flowchart

  flowchart TD
  Goal["Research Goal<br>Discover Interpretability &<br>Effective Formulaic Alphas"] --> Input["Input Data<br>Historical Market Data"]
  Input --> Method["Methodology: DRL Agent Alpha²<br>Formulates Alpha Discovery as Program Construction"]
  Method --> Search["Search Process<br>DRL-Guided Search Space Navigation<br>+ Dimensional Analysis & Pruning"]
  Search --> Eval["Evaluation<br>Metrics for Performance & Diversity"]
  Eval --> Outcomes["Outcomes<br>Logical, Diverse Alphas<br>+ Improved Trading Strategy"]
  Outcomes --> Comparison["Result: Outperforms<br>Genetic Programming Baselines"]
  
  style Goal fill:#f9f,stroke:#333,stroke-width:2px
  style Outcomes fill:#bbf,stroke:#333,stroke-width:2px

Cited References (22)

1. Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning — → 2306.12964
2. Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning — → 2306.12964
3. Auto A lpha: An efficient hierarchical evolutionary algorithm for mining alpha factors in quantitative investment (2020) — link
4. 101 formulaic alphas (2016)
5. Faster sorting algorithms discovered using deep reinforcement learning (2023)
6. Mastering the game of G o without human knowledge (2017) — link
7. Mastering the game of go without human knowledge (2017)
8. Mastering the game of Go with deep neural networks and tree search (2016) — link

2 of 22 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.