Paper: arXiv 2509.05080

Authors: Yang Chen, Yueheng Jiang, Zhaozhao Ma, Yuchen Cao, Jacky Keung, Kun Kuang, Leilei Gan, Yiquan Wu, Fei Wu

Abstract

The inherent non-stationarity of financial markets and the complexity of multi-modal information pose significant challenges to existing quantitative trading models. Traditional methods relying on fixed structures and unimodal data struggle to adapt to market regime shifts, while large language model (LLM)-driven solutions - despite their multi-modal comprehension - suffer from static strategies and homogeneous expert designs, lacking dynamic adjustment and fine-grained decision mechanisms. To address these limitations, we propose MM-DREX: a Multimodal-driven, Dynamically-Routed EXpert framework based on large language models. MM-DREX explicitly decouples market state perception from strategy execution to enable adaptive sequential decision-making in non-stationary environments. Specifically, it (1) introduces a vision-language model (VLM)-powered dynamic router that jointly analyzes candlestick chart patterns and long-term temporal features to allocate real-time expert weights; (2) designs four heterogeneous trading experts (trend, reversal, breakout, positioning) generating specialized fine-grained sub-strategies; and (3) proposes an SFT-RL hybrid training paradigm to synergistically optimize the router’s market classification capability and experts’ risk-adjusted decision-making. Extensive experiments on multi-modal datasets spanning stocks, futures, and cryptocurrencies demonstrate that MM-DREX significantly outperforms 15 baselines (including state-of-the-art financial LLMs and deep reinforcement learning models) across key metrics: total return, Sharpe ratio, and maximum drawdown, validating its robustness and generalization. Additionally, an interpretability module traces routing logic and expert behavior in real time, providing an audit trail for strategy transparency.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces advanced concepts like POMDP formulation and hybrid SFT-RL training, but the dense mathematics is contained within a high-level framework rather than extensive derivations. It demonstrates strong empirical rigor with a multi-modal dataset, comparisons against 15 baselines, and cross-market validation, indicating backtest-ready implementation.

Research Flowchart

  flowchart TD
  A["Research Goal: Overcome market non-stationarity & static LLM limitations in financial trading"] --> B["MM-DREX Framework Architecture"]
  
  B --> C["Multi-Modal Input Processing<br/>(Candlestick Charts & Temporal Data)"]
  
  C --> D["VLM-Powered Dynamic Router<br/>(Real-time expert weight allocation)"]
  
  D --> E["Heterogeneous Expert Pool<br/>1. Trend<br/>2. Reversal<br/>3. Breakout<br/>4. Positioning"]
  
  E --> F["SFT-RL Hybrid Training<br/>(Optimize router & experts jointly)"]
  
  F --> G["Real-time Interpretability<br/>(Routing logic audit trail)"]
  
  F --> H["Key Findings: Outperforms 15 baselines<br/>Higher returns, Sharpe ratio & lower drawdown<br/>across stocks, futures & crypto"]
  
  subgraph "Computational Process"
      C
      D
      E
      F
      G
  end
  
  H

Cited References (50)

  1. Deep reinforcement learning for portfolio management (2023)
  2. Reinforcement Learning in Economics and Finance (2021)
  3. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019 (2020)
  4. Trading in Fast-Changing Markets with Meta-Reinforcement Learning (2024)
  5. Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (2024) — link
  6. FinGPT: Open-Source Financial Large Language Models (2023) — link
  7. BloombergGPT: A Large Language Model for Finance (2023) — link
  8. A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist (2024)
and 42 more
  1. Tradexpert: Revolutionizing trading with mixture of expert llms (2024) — link
  2. FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design (2024)
  3. TradingAgents: Multi-agents LLM financial trading framework (2024) — link
  4. Multimodal Deep Learning for Finance: Integrating and Forecasting International Stock Markets (2019) — link
  5. An Econometric Analysis of ETF and ETF Futures in Financial and Energy Markets Using Generated Regressors (2018)
  6. COVID-19 and the march 2020 stock market crash. Evidence from S&P1500 (2021)
  7. Feverish stock price reactions to COVID-19 (2020)
  8. Cue the volatility spillover in the cryptocurrency markets during the COVID-19 pandemic: evidence from DCC-GARCH and wavelet analysis (2022)
  9. Impacts of COVID-19 on the Return and Volatility Nexus among Cryptocurrency Market (2022)
  10. Pixiu: A large language model, instruction data and evaluation benchmark for finance (2023) — link
  11. Investing in the S&P 500 index: Can anything beat the buy-and-hold strategy? (2020)
  12. Stock market efficiency and the MACD. Evidence from countries around the world (2015)
  13. Technical analysis of three stock oscillators testing MACD, RSI and KDJ rules in SH & SZ stock markets (2015)
  14. Forecasting stock market returns: New technical indicators and two-step economic constraint method (2020)
  15. A New Indicator for Describing Bull and Bear Markets (2018) — link
  16. Identifying oversold levels and developing low-frequency trading algorithms for the S&P 500: An analysis using stochastic oscillator, Williams% R, and trading volume (2024)
  17. Assessing the Impact of Technical Indicators on Machine Learning Models for Stock Price Prediction — → 2412.15448
  18. Lightgbm: A highly efficient gradient boosting decision tree (2017)
  19. Deep learning with long short‑term memory networks for financial market predictions (2018) — link
  20. Attention is all you need (2017)
  21. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  22. Proximal policy optimization algorithms (2017) — link
  23. Human-level control through deep reinforcement learning (2015)
  24. The representation landscape of few-shot learning and fine-tuning in large language models (2024)
  25. Qwen3-VL Technical Report (2025) — link
  26. LoRA: Low-Rank Adaptation of Large Language Models (2021) — link
  27. QLoRA: Efficient Finetuning of Quantized LLMs (2023)
  28. LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading (2025) — link
  29. MountainLion: A Multi-Modal LLM-Based Agent System for Interpretable and Adaptive Financial Trading — → 2507.20474
  30. AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay (2025) — link
  31. R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization — → 2505.15155
  32. Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting (2025) — link
  33. The Probability of Backtest Overfitting (2016)
  34. The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality (2014)
  35. The Evaluation and Optimization of Trading Strategies (2008)
  36. Know Your System! Turning a Trading Strategy into a Money Machine (2020)
  37. Know Your System! Turning Data Mining from Bias to Benefit through System Parameter Permutation (2014) — link
  38. A Reality Check for Data Snooping (2000)
  39. A Test for Superior Predictive Ability (2005)
  40. Practical Bayesian Optimization of Machine Learning Algorithms (2012)
  41. LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation (2026)
  42. Predicting the Output from a Complex Computer Code When Fast Approximations Are Available (2000) — link

3 of 50 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.