Paper: arXiv 2403.06779

Abstract

This paper comprehensively reviews the application of machine learning (ML) and AI in finance, specifically in the context of asset pricing. It starts by summarizing the traditional asset pricing models and examining their limitations in capturing the complexities of financial markets. It explores how 1) ML models, including supervised, unsupervised, semi-supervised, and reinforcement learning, provide versatile frameworks to address these complexities, and 2) the incorporation of advanced ML algorithms into traditional financial models enhances return prediction and portfolio optimization. These methods can adapt to changing market dynamics by modeling structural changes and incorporating heterogeneous data sources, such as text and images. In addition, this paper explores challenges in applying ML in asset pricing, addressing the growing demand for explainability in decision-making and mitigating overfitting in complex models. This paper aims to provide insights into novel methodologies showcasing the potential of ML to reshape the future of quantitative finance.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper presents substantial mathematical density with multiple advanced equations, derivatives, and formal proofs regarding latent factor estimation and non-linear modeling. However, it is a comprehensive review article with no empirical backtests, code, or implementation details, relying instead on theoretical discussions of ML applications.

Research Flowchart

  flowchart TD
  A["Research Goal: Review ML<br>in Asset Pricing"] --> B["Data: Heterogeneous<br>Market Data"]
  B --> C["Methodology: ML &<br>Traditional Models"]
  C --> D["Computational Process:<br>Supervised/Unsupervised Learning"]
  D --> E["Outcome: Enhanced<br>Return Prediction"]
  D --> F["Outcome: Improved<br>Portfolio Optimization"]
  E & F --> G["Key Challenge:<br>Explainability & Overfitting"]

Cited References (157)

  1. Machine learning in empirical asset pricing (2019)
  2. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown (2017)
  3. Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction. (2020)
  4. Factorvae: A probabilistic dynamic factor model based on variational autoencoder for predicting cross-sectional stock returns (2022)
  5. Stock broad-index trend patterns learning via domain knowledge informed generative network — → 2302.14164
  6. Instrumented principal component analysis (2020)
  7. Deep learning-based feature engineering for stock price movement prediction (2019)
  8. Modeling corporate bond returns (2023)
and 149 more
  1. Portfolio Selection (1952)
  2. Returns to Buying Winners and Selling Losers: Implications for (1993)
  3. Mean reversion in stock prices: Evidence and implications (1988)
  4. Capital asset prices: A theory of market equilibrium under conditions of risk (1964)
  5. Characteristics are covariances: A unified model of risk and return (2019)
  6. Empirical asset pricing via machine learning (2019)
  7. Structural deep learning in conditional asset pricing (2022)
  8. Deep learning for predicting asset returns (2018) — link
  9. Deep learning in asset pricing (2023)
  10. Common risk factors in the returns on stocks and bonds (1993)
  11. Dissecting anomalies with a five-factor model (2016)
  12. The cross-section of expected stock returns (1992)
  13. Margins and hedge fund contagion (2011)
  14. Application of deep reinforcement learning in stock trading strategies and stock forecasting (2020)
  15. Cost-sensitive portfolio selection via deep reinforcement learning (2020)
  16. Reinforcement learning for quantitative trading (2023)
  17. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  18. Recent advances in reinforcement learning in finance (2023)
  19. Reinforcement learning: An introduction (2018)
  20. Proximal policy optimization algorithms (2017) — link
  21. Continuous control with deep reinforcement learning (2015) — link
  22. Asynchronous methods for deep reinforcement learning (2016)
  23. Can sentiment analysis and options volume anticipate future returns? (2017)
  24. Variance reduction applied to machine learning for pricing Bermudan/American options in high dimension (2019) — link
  25. Machine learning for pricing American options in high-dimensional Markovian and non-Markovian models (2020)
  26. Investor sentiment and machine learning: Predicting the price of China’s crude oil futures market (2022)
  27. Linguistic specificity and stock price synchronicity (2022)
  28. Missing financial data (2022)
  29. Deep parametric portfolio policies (2023)
  30. Graphmae: Self-supervised masked graph autoencoders (2022)
  31. Machine learning in the Chinese stock market (2022)
  32. Financial machine learning (2023)
  33. Missing values and the dimensionality of expected returns (2022) — link
  34. T2-gnn: Graph neural networks for graphs with incomplete features and structure via teacher-student distillation (2023)
  35. Traffic flow prediction with missing data imputed by tensor completion methods (2020)
  36. Comparison of performance of data imputation methods for numeric dataset (2019)
  37. Costco: A neural tensor completion model for sparse tensors (2019)
  38. Volatility forecasting for interbank offered rate using grey extreme learning machine: The case of China (2016)
  39. Empirical asset pricing via machine learning: evidence from the European stock market (2021)
  40. Predicting returns in the stock and bond markets (1986)
  41. Cryptocurrencies asset pricing via machine learning (2021)
  42. Short-term bitcoin market prediction via machine learning (2021)
  43. AI-enabled business-model innovation and transformation in industrial ecosystems: A framework, model and outline for further research (2021)
  44. Modeling of frequency containment reserve prices with econometrics and artificial intelligence (2020)
  45. Using univariate models for construction output forecasting: Comparing artificial intelligence and econometric techniques (2016)
  46. Machine learning and AI in marketing–Connecting computing power to human insights (2020)
  47. Predictability of stock returns: Robustness and economic significance (1995)
  48. Boundaries of predictability: Noisy predictive regressions (2000)
  49. A comprehensive look at the empirical performance of equity premium prediction (2008)
  50. Option hedging with risk averse reinforcement learning (2020)
  51. Open source cross-sectional asset pricing (2021)
  52. A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
  53. Asset pricing with omitted factors (2021)
  54. Dimension Reduction for time series with Variational AutoEncoders (2022) — link
  55. Variational autoencoder-based dimensionality reduction for high-dimensional small-sample data classification (2020)
  56. Empirical asset pricing via machine learning (2020)
  57. Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
  58. Estimating latent asset-pricing factors (2020)
  59. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  60. A factor model for option returns (2022)
  61. Asset pricing explorations for macroeconomics (1992)
  62. A parsimonious macroeconomic model for asset pricing (2009)
  63. Practical Deep Reinforcement Learning Approach for Stock Trading (2023) (2018)
  64. Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019)
  65. A tabular sarsa-based stock market agent (2020)
  66. FinRL: Deep reinforcement learning framework to automate trading in quantitative finance (2021)
  67. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  68. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  69. Comprehensive review of deep reinforcement learning methods and applications in economics (2020)
  70. Deep reinforcement learning for trading—A critical survey (2021)
  71. DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
  72. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  73. Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
  74. Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
  75. A smart trader for portfolio management based on normalizing flows (2022)
  76. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  77. Modelling stock markets by multi-agent reinforcement learning (2021)
  78. MetaTrader: An reinforcement learning approach integrating diverse policies for portfolio optimization (2022)
  79. Relation-aware transformer for portfolio policy learning (2021)
  80. The sharpe ratio (1998)
  81. Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints (2023)
  82. Deep learning for event-driven stock prediction (2015)
  83. Incorporating corporation relationship via graph convolutional neural networks for stock price prediction (2018)
  84. Hats: A hierarchical graph attention network for stock movement prediction (2019) — link
  85. Deepwalk: Online learning of social representations (2014)
  86. Line: Large-scale information network embedding (2015)
  87. node2vec: Scalable feature learning for networks (2016)
  88. Semi-supervised classification with graph convolutional networks (2016) — link
  89. Knowledge-driven stock trend prediction and explanation via temporal convolutional network (2019)
  90. Multi-scale Two-way Deep Neural Network for Stock Trend Prediction. (2020)
  91. Modeling the momentum spillover effect for stock prediction via attribute-driven graph attention networks (2021)
  92. Coupling macro-sector-micro financial indicators for learning stock representations with less uncertainty (2021)
  93. Stock trend prediction with multi-granularity data: A contrastive learning approach with adaptive fusion (2021)
  94. Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts (2021)
  95. FinGAT: Financial Graph Attention Networks for Recommending Top-$ K $ K Profitable Stocks (2021)
  96. DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting (2023)
  97. Attention based dynamic graph learning framework for asset pricing (2021)
  98. Investment behaviors can tell what inside: Exploring stock intrinsic properties for stock trend prediction (2019)
  99. Temporal relational ranking for stock prediction (2019)
  100. Machine learning and portfolio optimization (2018)
  101. A five-factor asset pricing model (2015)
  102. On persistence in mutual fund performance (1997)
  103. Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction — → 2309.00073
  104. VolTAGE: Volatility forecasting via text audio fusion with graph convolution networks for earnings calls (2020)
  105. Deep attentive learning for stock movement prediction from social media text and company correlations (2020)
  106. Time-aware graph relational attention network for stock recommendation (2020)
  107. Modeling the stock relation with graph network for overnight stock movement prediction (2021)
  108. Stock selection via spatiotemporal hypergraph attention network: A learning to rank approach (2021)
  109. Learning multiple stock trading patterns with temporal routing adaptor and optimal transport (2021)
  110. Pattern Adaptive Specialist Network for Learning Trading Patterns in Stock Market (2022)
  111. Heterogeneous Interactive Snapshot Network for Review-Enhanced Stock Profiling and Recommendation (2021)
  112. Mastering Stock Markets with Efficient Mixture of Diversified Trading Experts (2023)
  113. Contrastive learning and reward smoothing for deep portfolio management (2023)
  114. PEN: prediction-explanation network to forecast stock price movement with better explainability (2023)
  115. Prediction with time-series mixer for the S&P500 index (2023)
  116. Stock closing price prediction using machine learning techniques (2020)
  117. Stock price prediction using principal components (2020)
  118. A CNN-BiLSTM-AM method for stock price prediction (2021)
  119. Short term stock price prediction using deep learning (2017)
  120. From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting (2023)
  121. Trading via image classification (2020)
  122. Deep video prediction for time series forecasting (2021)
  123. Option pricing using machine learning (2021)
  124. Machine learning in finance: the case of deep learning for option pricing (2017)
  125. Predicting the stock price of frontier markets using machine learning and modified Black–Scholes Option pricing model (2020)
  126. Stock market price prediction using LSTM RNN (2019)
  127. Stock prediction using deep learning (2017)
  128. Stock market prediction using LSTM recurrent neural network (2020)
  129. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer (2017) — link
  130. Attention is all you need (2017)
  131. Language models are few-shot learners (2020)
  132. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  133. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2019)
  134. PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting (2023) — link
  135. Extra-market components of covariance in security returns (1974)
  136. Economic forces and the stock market (1986)
  137. Performance measurement with the arbitrage pricing theory: A new framework for analysis (1986)
  138. Intermediary asset pricing (2013)
  139. Risks for the long run: A potential resolution of asset pricing puzzles (2004)
  140. Factor models, machine learning, and asset pricing (2022)
  141. Machine Learning for Financial Risk Management: A Survey (2020)
  142. Stock price prediction using LSTM, RNN and CNN-sliding window model (2017)
  143. Taming the factor zoo: A test of new factors (2020)
  144. Autoencoder asset pricing models (2021)
  145. Latent factor model for asset pricing (2020)
  146. Are missing values important for earnings forecasts? A machine learning perspective (2022)
  147. A Fast Non-Linear Coupled Tensor Completion Algorithm for Financial Data Integration and Imputation (2023)
  148. Machine Learning for Earnings Prediction: A Nonlinear Tensor Approach for Data Integration and Completion (2022)
  149. Recovering missing firm characteristics with attention-based machine learning (2022)

2 of 157 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.