Paper: arXiv 2507.16548

Authors: Jakub Michańków, Paweł Sakowski, Robert Ślepaczuk

Abstract

The proper design and architecture of testing machine learning models, especially in their application to quantitative finance problems, is crucial. The most important aspect of this process is selecting an adequate loss function for training, validation, estimation purposes, and hyperparameter tuning. Therefore, in this research, through empirical experiments on equity and cryptocurrency assets, we apply the Mean Absolute Directional Loss (MADL) function, which is more adequate for optimizing forecast-generating models used in algorithmic investment strategies. The MADL function results are compared between Transformer and LSTM models, and we show that in almost every case, Transformer results are significantly better than those obtained with LSTM.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents advanced neural network architecture mathematics (Transformer attention mechanisms with formal equations) and demonstrates rigorous empirical testing including walk-forward procedures, extended out-of-sample periods, and risk-adjusted performance metrics across multiple asset classes.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Design ML testing for Finance"] --> B["Data Sources<br>(Equity & Crypto Assets)"]
  B --> C{"Model Training & Tuning"}
  C --> D["Transformer Model"]
  C --> E["LSTM Model"]
  D & E --> F["Apply MADL<br>Loss Function"]
  F --> G["Comparison &<br>Evaluation Results"]
  G --> H["Outcome:<br>Transformer outperforms LSTM"]

Cited References (107)

  1. Attention Is All You Need (2023) — link
  2. Are Transformers Effective for Time Series Forecasting? (2022) — link
  3. Transformers in Time Series: A Survey (2023) — link
  4. Forecasting stock prices with long-short term memory neural network based on attention mechanism (2020) — link
  5. Comparison of Time Series Forecasting Based on Statistical ARIMA Model and LSTM with Attention Mechanism (2020) — link
  6. Attention is all you need: An interpretable transformer-based asset allocation approach (2023) — link
  7. Financial Time Series Forecasting using CNN and Transformer (2023) — link
  8. Stock market index prediction using deep Transformer model (2022) — link
and 99 more
  1. Investment Strategies that Beat the Market. What Can We Squeeze from the Market? (2018) — link
  2. Predicting prices of S&P 500 index using classical methods and recurrent neural networks (2024)
  3. Long Short-Term Memory (1997) — link
  4. Mean Absolute Directional Loss as a new loss function for machine learning problems in algorithmic investment strategies — → 2309.10546
  5. The lorentz transformation and absolute time (1953) — link
  6. The theory of a general quantum system interacting with a linear dissipative system (1963) — link
  7. New evidence on the implied-realized volatility relation (2002)
  8. Forecasting Volatility in the Financial Markets (2007)
  9. The relationship between implied and realized volatility of S&P 500 index (2003) — link
  10. Implied Volatilities as Forecasts of Future Volatility, Time-Varying Risk Premia, and Returns Variability (2001)
  11. The relation between implied and realized volatility (1998)
  12. Volatility forecasting in the framework of the option expiry cycle (1999)
  13. The Relationship between Implied and Realized Volatility in the Danish Option and Equity Markets (1999)
  14. Information Content of Option Implied Volatility : Evidence from the Indian Market (2008)
  15. The relationship between implied and realized volatility: evidence from the Australian stock index option market (2009) — link
  16. Predicting Volatility in the Foreign Exchange Market (1995) — link
  17. Econometric Computing with HC and HAC Covariance Matrix Estimators (2004) — link
  18. Implementing a class of structural change tests: An econometric computing approach (2006) — link
  19. Object-oriented Computation of Sandwich Estimators (2006) — link
  20. The Informational Content of Implied Volatility (1993) — link
  21. Machine Learning Methods in Algorithmic Trading Strategy Optimization–Design and Time Efficiency (2019)
  22. Is smart beta really smart? (2014)
  23. How Smart are Smart Beta ETFs? Analysis of Relative Performance and Factor Exposure (2015)
  24. Do smart beta ETFs deliver persistent performance? (2020)
  25. Portfolio theory in terms of partial covariance (2016)
  26. Patterns in financial markets: Dynamic time warping (2012)
  27. An empirical study on the differences between synthetic and physical ETFs (2015)
  28. Tracking ability of ETFs: physical versus synthetic replication (2014)
  29. Tracking S&P 500 index funds (2001)
  30. Robo-Advisors: Machine Learning in Trend-Following ETF Investments (2020)
  31. Similarity learning for time series classification (2016) — link
  32. Guide to pragmatic replication (2019)
  33. A novel dynamic asset allocation system using Feature Saliency Hidden Markov models for smart beta investing (2021)
  34. Redefining Indexing Using Smart Beta Strategies (2017)
  35. The asset manager’s dilemma: How smart beta is disrupting the investment management industry (2016)
  36. Fat Tailed Factors (2020) — link
  37. Competition for Attention in the ETF Space (2021)
  38. Alternative Beta: Redefining Alpha and Beta
  39. Making Your Smart Beta Even Smarter (2014)
  40. Tail Risk Mitigation with Managed Volatility Strategies (2017)
  41. Ten misconceptions about smart beta (2016)
  42. TailCoR (2020)
  43. Managing Beta Exposures in Portable Alpha Strategies
  44. Tail Risk of Smart Beta Portfolios: An Extreme Value Theory Approach (2014)
  45. Predicting prices of S&P500 index using classical methods and recurrent neural networks (2020)
  46. Application of machine learning in quantitative investment strategies on global stock markets (2021)
  47. Predicting stock prices using LSTM (2017)
  48. A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
  49. Improving trading technical analysis with tensorflow long short-term memory (lstm) neural network (2019)
  50. Nvidia’s Stock Returns Prediction Using Machine Learning Techniques for Time Series Forecasting Problem (2020)
  51. Improving stock closing price prediction using recurrent neural network and technical indicators (2018)
  52. Forex exchange rate forecasting using deep recurrent neural networks (2020)
  53. Deep learning with long short-term memory networks for financial market predictions (2018)
  54. Using neural networks to enhance technical trading rule returns: A case with KLCI (2016)
  55. Deep leaming for stock market prediction using technical indicators and financial news articles (2018)
  56. Forecasting price movements using technical indicators: Investigating the impact of varying input window length (2017)
  57. All that glitters is not gold: Comparing backtest and out-of-sample performance on a large cohort of trading algorithms (2016)
  58. What to Look for in a Backtest (2013)
  59. The probability of backtest overfitting (2016)
  60. Backtest overfitting in financial markets (2016)
  61. Portfolio of Global Futures Algorithmic Trading Strategies for Best Out-of-Sample Performance (2016)
  62. Algorithmic trading: winning strategies and their rationale (2013)
  63. Quantitative trading: how to build your own algorithmic trading business (2021)
  64. Machine Learning for Algorithmic Trading: Predictive models to extract signals from market and alternative data for systematic trading strategies with Python (2020)
  65. A high-frequency algorithmic trading strategy for cryptocurrency (2020)
  66. Redes Neurais | LSTM (2019)
  67. Applying Hybrid ARIMA-SGARCH in Algorithmic Investment Strategies on S&P500 Index (2022)
  68. Applying Hurst Exponent in pair trading strategies on Nasdaq 100 index (2021) — link
  69. Deep Learning with Python, 2nd ed. (2021)
  70. Long Short - Term Memory (1997) — link
  71. Learning to Forget : Continual Prediction with LSTM (2000) — link
  72. Learning to forget: continual prediction with LSTM (1999) — link
  73. Multi Factor Stock Selection Model Based on LSTM (2018)
  74. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling (2014) — link
  75. Deep learning with long short-term memory networks for financial market predictions (2018)
  76. Deep Learning (2016)
  77. Investment Strategies that Beat the Market . What Can We Squeeze from the Market ? (2018)
  78. AT - LSTM : An Attention -based LSTM Model for Financial Time Series Prediction (2019) — link
  79. Momentum and contrarian effects on the cryptocurrency market (2019) — link
  80. Deep Learning (2016)
  81. Adam: A Method for Stochastic Optimization (2017) — link
  82. Hyperband: A Novel Bandit - Based Approach to Hyperparameter Optimization (2018) — link
  83. KerasTuner (2019)
  84. Artificial neural networks architectures for stock price prediction: Comparisons and applications (2016)
  85. Multi Factor Stock Selection Model Based on LSTM (2018) — link
  86. Deep Learning for Stock Selection Based on High Frequency Price - Volume Data (2019) — link
  87. Deep Learning in Finance (2018) — link
  88. Modified support vector machines in ÿnancial time series forecasting (2002)
  89. Dynamic financial distress prediction with concept drift based on time weighting combined with Adaboost support vector machine ensemble (2017) — link
  90. Financial time series prediction using least squares support vector machines within the evidence framework (2001) — link
  91. Using deep learning to detect price change indications in financial markets (2017) — link
  92. A New Kernel of Support Vector Regression for Forecasting High - Frequency Stock Returns (2016) — link
  93. Machine-learning classification techniques for the analysis and prediction of high-frequency stock direction - University of Iowa (2014)
  94. Stock market’s price movement prediction with LSTM neural networks (2017) — link
  95. A deep learning framework for financial time series using stacked autoencoders and long-short term memory (2017)
  96. The impact of COVID-19 on emerging stock markets (2020) — link
  97. The day of the week effect in the cryptocurrency market (2019) — link
  98. A Q-learning agent for automated trading in equity stock markets (2021) — link
  99. Technical trading rules in the cryptocurrency market (2020) — link

1 of 107 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.