Paper: arXiv 2407.01577

Abstract

Algorithmic trading refers to executing buy and sell orders for specific assets based on automatically identified trading opportunities. Strategies based on reinforcement learning (RL) have demonstrated remarkable capabilities in addressing algorithmic trading problems. However, the trading patterns differ among market conditions due to shifted distribution data. Ignoring multiple patterns in the data will undermine the performance of RL. In this paper, we propose MOT,which designs multiple actors with disentangled representation learning to model the different patterns of the market. Furthermore, we incorporate the Optimal Transport (OT) algorithm to allocate samples to the appropriate actor by introducing a regularization loss term. Additionally, we propose Pretrain Module to facilitate imitation learning by aligning the outputs of actors with expert strategy and better balance the exploration and exploitation of RL. Experimental results on real futures market data demonstrate that MOT exhibits excellent profit capabilities while balancing risks. Ablation studies validate the effectiveness of the components of MOT.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts including optimal transport (OT) theory, Markov Decision Processes (MDP), and differential Sharpe ratio derivations, indicating high mathematical complexity. It demonstrates strong empirical rigor by using real futures market data, conducting ablation studies, and detailing specific implementations for backtesting, though it lacks the extreme reproducibility cues like GitHub links or exhaustive statistical metrics.

Research Flowchart

  flowchart TD
  A["Research Goal<br/>'How to address market condition shifts<br/>in RL for algorithmic trading?'"] --> B["Data Input<br/>Real futures market data"]
  B --> C["Core Methodology<br/>MOT: Mixture of Actors with Optimal Transport"]
  C --> D["Computational Process<br/>1. Disentangled RL Actors<br/>2. OT-based sample allocation<br/>3. Pretrain Module for imitation"]
  D --> E["Key Findings<br/>1. Superior profit & risk balance<br/>2. Validated components via ablation<br/>3. Robustness to market shifts"]

Cited References (68)

  1. Blackboard Systems (1986)
  2. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education (1983)
  3. Classification Problem Solving (1984)
  4. New Ways to Make Microcircuits Smaller (1980) — link
  5. New Ways to Make Microcircuits Smaller—Duplicate Entry (1980)
  6. Strategic explanations for a diagnostic consultation system (1984) — link
  7. Strategic Explanations in Consultation—Duplicate (1983)
  8. Poligon: A System for Parallel Problem Solving (1986)
and 60 more
  1. Transfer of Rule-Based Expertise through a Tutorial Dialogue (1979)
  2. The Engineering of Qualitative Models (2021)
  3. Attention Is All You Need (2017) — link
  4. Pluto: The ‘Other’ Red Planet (2015)
  5. Deep reinforcement learning in quantitative algorithmic trading: A review (2021) — link
  6. The behavior of stock-market prices (1965)
  7. Cross-sectional and time-series determinants of momentum returns (2002)
  8. Multifactor explanations of asset pricing anomalies (1996)
  9. HIST: A graph-based framework for stock trend forecasting via mining concept-oriented shared information (2021) — link
  10. Learning multiple stock trading patterns with temporal routing adaptor and optimal transport (2021)
  11. Rest: Relational event-driven stock trend forecasting (2021)
  12. Trimming the Sail: A Second-order Learning Paradigm for Stock Prediction (2020) — link
  13. Listening to chaotic whispers: A deep learning framework for news-oriented stock trend prediction (2018)
  14. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  15. Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
  16. Universal trading for order execution with oracle policy distillation (2021)
  17. Market making via reinforcement learning (2019)
  18. Returns to buying winners and selling losers: Implications for stock market efficiency (1993)
  19. Time series momentum (2012)
  20. Mean reversion in stock prices: Evidence and implications (1988)
  21. Dissecting investment strategies in the cross section and time series (2015)
  22. New concepts in technical trading systems (1978)
  23. Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs (2018)
  24. Commodity channel index: Evaluation of trading rule of agricultural commodities (2016)
  25. Profitability of the on-balance volume indicator (2009)
  26. The exponentially weighted moving average (1986)
  27. A unified theory of underreaction, momentum trading, and overreaction in asset markets (1999)
  28. Deep direct reinforcement learning for financial signal representation and trading (2016)
  29. Optimization of trading systems and portfolios (1997)
  30. An automated FX trading system using adaptive reinforcement learning (2006)
  31. Adaptive stock trading with dynamic asset allocation using reinforcement learning (2006)
  32. A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
  33. A tabular sarsa-based stock market agent (2020)
  34. Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019)
  35. Deep reinforcement learning for trading (2019)
  36. Using data augmentation based reinforcement learning for daily stock trading (2020)
  37. Mutual fund performance (1966)
  38. Reinforcement learning for trading (1998)
  39. Proximal policy optimization algorithms (2017)
  40. Empirical evaluation of gated recurrent neural networks on sequence modeling (2014)
  41. Behavioral finance (2003)
  42. Sinkhorn distances: Lightspeed computation of optimal transport (2013)
  43. Memory-based control with recurrent neural networks (2015)
  44. A hybrid approach based on neural networks and genetic algorithms for detecting temporal patterns in stock markets (2007)
  45. Categorical reparameterization with gumbel-softmax (2016)
  46. Multi-domain learning by confidence-weighted parameter combination (2010)
  47. Parameter-efficient transfer learning for NLP (2019)
  48. Multi-domain learning: when do domains matter? (2012)
  49. Learning multiple visual domains with residual adapters (2017)
  50. Efficient parametrization of multi-domain deep neural networks (2018)
  51. Recent advances in reinforcement learning in finance (2021)
  52. Deep execution-value and policy based reinforcement learning for trading and beating market benchmarks (2019)
  53. An End-to-End Optimal Trade Execution Framework based on Proximal Policy Optimization. (2020)
  54. Deep risk model: a deep learning solution for mining latent risk factors to improve covariance matrix estimation (2021)
  55. ADD: Augmented disentanglement distillation framework for improving stock trend forecasting (2020)
  56. A machine learning view on momentum and reversal trading (2018)
  57. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity (2022)
  58. Gshard: Scaling giant models with conditional computation and automatic sharding (2020)
  59. Glam: Efficient scaling of language models with mixture-of-experts (2022)
  60. Go wider instead of deeper (2022)

0 of 68 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.