Multimodal Deep Reinforcement Learning for Portfolio Optimization

Paper: arXiv 2412.17293

Abstract

We propose a reinforcement learning (RL) framework that leverages multimodal data including historical stock prices, sentiment analysis, and topic embeddings from news articles, to optimize trading strategies for SP100 stocks. Building upon recent advancements in financial reinforcement learning, we aim to enhance the state space representation by integrating financial sentiment data from SEC filings and news headlines and refining the reward function to better align with portfolio performance metrics. Our methodology includes deep reinforcement learning with state tensors comprising price data, sentiment scores, and news embeddings, processed through advanced feature extraction models like CNNs and RNNs. By benchmarking against traditional portfolio optimization techniques and advanced strategies, we demonstrate the efficacy of our approach in delivering superior portfolio performance. Empirical results showcase the potential of our agent to outperform standard benchmarks, especially when utilizing combined data sources under profit-based reward functions.

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs only partially modelled; see penalty flags)
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts including Markov Decision Processes, deep neural networks, and tensor-based feature extraction, while also detailing comprehensive data sourcing, preprocessing pipelines, and empirical benchmarking against traditional methods.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Optimize SP100 Trading via RL<br>with Multimodal Data"] --> B["Data Inputs & Processing"]
  subgraph B ["Inputs"]
      B1["Price History"]
      B2["News Sentiment"]
      B3["SEC Filings"]
  end
  B --> C["Deep RL Agent Construction"]
  C --> D["Feature Extraction:<br>CNN/RNN on State Tensors"]
  D --> E["Policy Learning:<br>Profit-based Reward Function"]
  E --> F["Execution & Backtesting"]
  F --> G["Key Findings:<br>Outperforms Benchmarks<br>Synergy of Data Sources"]

Cited References (15)

1. MAPS : Multi -agent Reinforcement Learning -based Portfolio Management System (2020) — link
2. Stock portfolio selection using learning-to-rank algorithms with news sentiment (2017) — link
3. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
4. Reinforcement- Learning based Portfolio Management with Augmented Asset Movement Prediction States (2020) — link
5. Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory (2023) — link
6. Deep Reinforcement Learning for Optimal Portfolio Allocation : A Comparative Study with Mean - Variance Optimization
7. FinBERT - FOMC : Fine - Tuned FinBERT Model with Sentiment Focus Method for Enhancing Sentiment Analysis of FOMC Minutes (2023) — link
8. Reinforcement Learning for Trading Systems and Portfolios: Immediate vs Future Rewards — link

0 of 15 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.