Paper: arXiv 2507.18560

Authors: Benjamin Coriat, Eric Benhamou

Abstract

This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators. Our three-tier architecture employs base RL agents to process hybrid data, meta-agents to aggregate their decisions, and a super-agent to merge decisions based on market data and sentiment analysis. Evaluated on data from 2018 to 2024, after training on 2000-2017, the framework achieves a 26% annualized return and a Sharpe ratio of 1.2, outperforming equal-weighted and S&P 500 benchmarks. Key contributions include scalable cross-modal integration, a hierarchical RL structure for enhanced stability, and open-source reproducibility.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts in deep reinforcement learning, hierarchical architectures, and Markowitz portfolio theory, while also presenting a backtest-ready framework with specific performance metrics (26% annualized return, 1.2 Sharpe), detailed data pipelines, and open-source reproducibility.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Integrate LLM Sentiment & DRL<br>for Portfolio Optimization"] --> B["Data Inputs:<br>Financial News & Market Data<br>(2000-2024)"]
  B --> C["HARLF Architecture"]
  C --> D["1. Base RL Agents:<br>Process hybrid data"]
  D --> E["2. Meta-Agents:<br>Aggregate decisions"]
  E --> F["3. Super-Agent:<br>Final sentiment-weighted action"]
  F --> G["Backtesting:<br>2018-2024 Period"]
  G --> H["Outcomes:<br>26% Annualized Return<br>Sharpe Ratio 1.2"]

Cited References (45)

  1. Structure and Interpretation of Computer Programs (1985)
  2. Visual Information Extraction with Lixto (2001)
  3. An overview of the KL-ONE knowledge representation system (1985)
  4. Complexity results for nonmonotonic logics (1992)
  5. Hypertree Decompositions and Tractable Queries (2002)
  6. Foundations of a functional approach to knowledge representation (1984)
  7. A logic of implicit and explicit belief (1984)
  8. On the compilability and expressive power of propositional planning formalisms (2000)
and 37 more
  1. IJCAI Camera Ready Submission
  2. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
  3. Deciphering the Liquidity and Credit Crunch 2007–2008 (2009) — link
  4. Generalized Mean-Variance Frameworks for Portfolio Optimization: A Comprehensive Literature Review (2009) — link
  5. Deep Direct Reinforcement Learning for Financial Signal Representation and Trading (2016) — link
  6. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  7. Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
  8. FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) — link
  9. Portfolio Selection (1952) — link
  10. Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007) — link
  11. BloombergGPT: A Large Language Model for Finance (2023) — link
  12. Proximal Policy Optimization Algorithms (2017) — link
  13. Continuous Control with Deep Reinforcement Learning (2015) — link
  14. Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
  15. Financial News-Driven LLM Reinforcement Learning for Portfolio Management (2024) — link
  16. Stable-Baselines3: Reliable Reinforcement Learning Implementations (2021) — link
  17. Optimizing Performance: How Compact Models Match or Exceed GPT’s Classification Capabilities through Fine-Tuning (2024) — link
  18. Sentiment-Driven Reinforcement Learning for Portfolio Optimization (2023) — link
  19. A Systematic Approach to Portfolio Optimization: A Comparative Study of Reinforcement Learning Agents, Market Signals, and Investment Horizons (2024)
  20. News-Driven Reinforcement Learning for Algorithmic Trading (2018) — link
  21. Large Language Models in Finance: A Survey (2024) — link
  22. Finance Wizard at the FinLLM Challenge Task: Financial Text Summarization (2024) — link
  23. L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification and Summarization (2024) — link
  24. CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications (2024) — link
  25. Twitter Mood Predicts the Stock Market (2011) — link
  26. OpenAI Gym (2016) — link
  27. Wisdom of Crowds: The Value of Stock Opinions Transmitted through Social Media (2019) — link
  28. Sentiment Analysis and its Impact on Financial Markets: A Comprehensive Review (2021) — link
  29. Human-level Control through Deep Reinforcement Learning (2015) — link
  30. Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (1999) — link
  31. Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016)
  32. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
  33. Large Language Models for Contextual Financial Sentiment Analysis (2024) — link
  34. Deep Reinforcement Learning for Financial Trading (2019) — link
  35. Financial Sentiment Analysis on News and Reports Using Large Language Models (2024) — link
  36. Enhancing Financial Sentiment Analysis via Retrieval-Augmented Large Language Models (2023) — link
  37. ‘Finance Wizard’at the FinLLM Challenge Task: Financial Text Summarization (2024) — link

0 of 45 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.