Paper: arXiv 2509.01393

Authors: Qizhao Chen, Hiroaki Kawashima

Abstract

This paper proposes a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies. Formulaic alphas are mathematically defined trading signals derived from price, volume, sentiment, and other data. Although recent studies have shown that LLMs can generate diverse and effective alphas, a critical challenge lies in how to adaptively integrate them under varying market conditions. To address this gap, we leverage the deepseek-r1-distill-llama-70b model to generate fifty alphas for five major stocks: Apple, HSBC, Pepsi, Toyota, and Tencent, and then use PPO to adjust their weights in real time. Experimental results demonstrate that the PPO-optimized strategy achieves strong returns and high Sharpe ratios across most stocks, outperforming both an equal-weighted alpha portfolio and traditional benchmarks such as the Nikkei 225, S&P 500, and Hang Seng Index. The findings highlight the importance of reinforcement learning in the allocation of alpha weights and show the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced reinforcement learning (PPO) and deals with mathematical formulation of alpha signals, indicating moderate to high math complexity, while the experimental setup with specific stock tickers, Sharpe ratios, and comparisons to benchmarks demonstrates strong empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: <br>Adaptive Alpha Weighting"] --> B["Data: 50 LLM-Generated Alphas<br>5 Stocks: Apple, HSBC, Pepsi, Toyota, Tencent"]
  B --> C["Core Methodology:<br>Proximal Policy Optimization PPO"]
  C --> D{"Compute Portfolio Weighted by<br>PPO-Optimized Alpha Weights"}
  D --> E["Execution: <br>Trading Strategy Backtest"]
  E --> F["Outcome: <br>Higher Returns & Sharpe Ratios"]
  F --> G["Comparison: <br>Outperforms Equal-Weight & Benchmarks S&P 500, Nikkei 225"]

Cited References (55)

  1. Technical Analysis Indicators in Stock Market Using Machine Learning: A Comparative Analysis (2021) — link
  2. Key technical indicators for stock market prediction (2025) — link
  3. Machine learning approaches in stock market prediction: A systematic literature review (2023) — link
  4. Interpretability of Machine Learning: Recent Advances and Future Prospects (2023) — link
  5. Automate Strategy Finding with LLM in Quant investment (2024) — link
  6. Stock Price Prediction Using LLM-Based Sentiment Analysis (2024) — link
  7. Common risk factors in the returns on stocks and bonds (1993) — link
  8. Short and long term stock trend prediction using decision tree (2017) — link
and 47 more
  1. Stocks market prediction using Support Vector Machine (2013) — link
  2. Stock Price Prediction Model Based on RBF-SVM Algorithm (2020) — link
  3. Research on stock forecasting based on random forest (2022) — link
  4. Stock Price Prediction Using CNN and LSTM-Based Deep Learning Models (2020) — link
  5. Stock Price Prediction using LSTM (2023) — link
  6. A CNN-LSTM-Based Model to Forecast Stock Prices (2020) — link
  7. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
  8. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2022) — link
  9. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024) — link
  10. MASTER: Market-Guided Stock Transformer for Stock Price Forecasting (2023) — link
  11. FinGPT: Open-Source Financial Large Language Models (2023) — link
  12. PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
  13. Automate Strategy Finding with LLM in Quant Investment (2025) — link
  14. GPT-Signal: Generative AI for Semi-automated Feature Engineering in the Alpha Research Process (2024) — link
  15. Proximal Policy Optimization Algorithms (2017) — link
  16. 101 Formulaic Alphas (2016) — link
  17. The unreasonable effectiveness of deep learning in artificial intelligence (2020) — link
  18. The alpha life cycle of quantitative strategy (2017) — link
  19. Stock Price Change Prediction Using Prompt-Based LLM s with RL -Enhanced Post-Hoc Adjustments (2025) — link
  20. A Deep Reinforcement Learning Framework for Dynamic Portfolio Optimization: Evidence from China’s Stock Market (2025) — link
  21. Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization (2024) — link
  22. Adaptive Investment Strategies: Deep Reinforcement Learning Approaches for Portfolio Optimization (2024) — link
  23. Distilling the Knowledge in a Neural Network (2015) — link
  24. Contrarian Investment Strategies: The Next Generation (1998)
  25. From local explanations to global understanding with explainable AI for trees (2020)
  26. A novel sentiment correlation-based method with dual transformer model for stock price prediction (2025) — link
  27. Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha (2025) — link
  28. An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
  29. Comparative Analysis of A3C and PPO Algorithms in Reinforcement Learning: A Survey on General Environments (2024) — link
  30. Proximal policy optimization via enhanced exploration efficiency (2022) — link
  31. Reinforcement Learning in a Virtual World: A Study of PPO and SAC within Unity ML-Agents (2025) — link
  32. Stock Price Prediction with LLM -Guided Market Movement Signals and Transformer Model (2025) — link
  33. Image-Driven Stock Price Prediction with LLaMA : A Prompt-Based Approach (2025) — link
  34. Explore Anomaly-Aware Transformers for Robust Financial Time Series Forecasting (2025) — link
  35. A Framework for Measuring How News Topics Drive Stock Movement (2025) — link
  36. Offline Reinforcement Learning for Automated Stock Trading (2023) — link
  37. Enhancing Deep Reinforcement Learning for Stock Trading: A Reward Shaping Approach via Expert Feedback (2025) — link
  38. Stock Market Trading via Actor–Critic Reinforcement Learning and Adaptable Data Structure (2025) — link
  39. A Novel RMS-Driven Deep Reinforcement Learning for Optimized Portfolio Management in Stock Trading (2025) — link
  40. Stock Closing Price Prediction using Machine Learning Techniques (2020) — link
  41. Multi-Feature Supervised Reinforcement Learning for Stock Trading (2023) — link
  42. Single stock trading with deep reinforcement learning: A comparative study (2022) — link
  43. Asynchronous Methods for Deep Reinforcement Learning (2016) — link
  44. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
  45. Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
  46. Optimal ratio for data splitting (2022) — link
  47. Explore the Use of Prompt-Based LLM for Credit Risk Classification (2025) — link

0 of 55 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.