Paper: arXiv 2506.03780

Authors: Hasan Fallahgoul

Abstract

Recent advances in machine learning have shown promising results for financial prediction using large, over-parameterized models. This paper provides theoretical foundations and empirical validation for understanding when and how these methods achieve predictive success. I examine two key aspects of high-dimensional learning in finance. First, I prove that within-sample standardization in Random Fourier Features implementations fundamentally alters the underlying Gaussian kernel approximation, replacing shift-invariant kernels with training-set dependent alternatives. Second, I establish information-theoretic lower bounds that identify when reliable learning is impossible no matter how sophisticated the estimator. A detailed quantitative calibration of the polynomial lower bound shows that with typical parameter choices, e.g., 12,000 features, 12 monthly observations, and R-square 2-3%, the required sample size to escape the bound exceeds 25-30 years of data–well beyond any rolling-window actually used. Thus, observed out-of-sample success must originate from lower-complexity artefacts rather than from the intended high-dimensional mechanism.

Complexity vs Empirical Score

  • Math Complexity: 9.0/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper is dense with advanced theoretical constructs including PAC-learning theory, information-theoretic bounds, random matrix theory, and detailed proofs on kernel approximation properties, placing it at the high end of mathematical complexity. While it includes quantitative calibration and numerical validation, the summary and excerpt focus on theoretical proofs and bounds rather than extensive backtesting, code, or dataset implementation, resulting in moderate empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Understanding High-Dim ML Success in Finance"] --> B{"Key Methodology<br>1. RFF Kernel Analysis<br>2. Info-Theoretic Bounds"}
  B --> C["Data/Inputs<br>12k Features, 12 Monthly Observations"]
  C --> D["Computational Process<br>Quantitative Calibration of Bounds"]
  D --> E{"Outcome: Required Sample Size"}
  E -- > 25-30 Years --> F["Key Finding<br>Success from Low-Dim Artefacts"]
  E -- <= 12 Months --> G["False Success Scenario<br>Unachievable Requirements"]
  F --> H["Conclusion<br>Targeted Low-Complexity Models"]
  G --> H

Cited References (52)

  1. Seemingly Virtuous Complexity in Return Prediction (2025)
  2. The Virtue of Complexity in Return Prediction (2024)
  3. A Comprehensive Look at the Empirical Performance of Equity Premium Prediction (2008)
  4. Random Features for Large-Scale Kernel Machines (2007)
  5. Introduction to Nonparametric Estimation (2009)
  6. Rademacher and Gaussian Complexities: Risk Bounds and Structural Results (2002)
  7. Stability and Generalization (2002)
  8. Gaussian Processes for Machine Learning (2006)
and 44 more
  1. Reconciling Modern Machine-Learning Practice and the Classical Bias–Variance Trade-off (2019)
  2. Surprises in High-Dimensional Ridgeless Least Squares Interpolation (2022)
  3. Benign Overfitting in Linear Regression (2020)
  4. On the Error of Random Fourier Features (2015)
  5. The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve (2022)
  6. High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections (2024)
  7. Early Stopping, Regularization Paths and Generalization Bounds for Ridge Regression (2014)
  8. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2009)
  9. High-Dimensional Statistics: A Non-Asymptotic Viewpoint (2019)
  10. Foundations of Machine Learning (2018)
  11. Support Vector Machines (2008)
  12. Learning Theory: An Approximation Theory Viewpoint (2007)
  13. Can Machines Learn Weak Signals? (2025)
  14. APT or “AIPT”? The Surprising Dominance of Large Factor Models (2024)
  15. On the equivalence between kernel quadrature rules and random feature expansions (2017)
  16. Neural Network Learning: Theoretical Foundations (1999) — link
  17. On the density of families of sets (1972) — link
  18. A combinatorial problem; stability and order for models and theories in infinitary languages (1972) — link
  19. On the uniform convergence of relative frequencies of events to their probabilities (1971) — link
  20. Statistical Learning Theory (1998)
  21. Learnability and the Vapnik-Chervonenkis dimension (1989) — link
  22. Understanding Machine Learning: From Theory to Algorithms (2014) — link
  23. Forecasting stock returns (2013)
  24. The Virtue of Complexity in Return Prediction (2024) — link
  25. Empirical Asset Pricing via Machine Learning (2020) — link
  26. Deep Learning in Asset Pricing (2024) — link
  27. Taming the Factor Zoo: A Test of New Factors (2020) — link
  28. Bond Risk Premiums with Machine Learning (2021) — link
  29. … and the Cross‐Section of Expected Returns (2016) — link
  30. Does Academic Research Destroy Stock Return Predictability? (2016) — link
  31. Reconciling Modern Machine‐Learning Practice and the Bias–Variance Trade‐off (2019) — link
  32. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2009)
  33. A Theory of the Learnable (1984)
  34. An Introduction to Computational Learning Theory (1994)
  35. Generalization Properties of Learning with Random Features (2017)
  36. An Introduction to Matrix Concentration Inequalities (2015) — link
  37. User-Friendly Tail Bounds for Sums of Random Matrices (2012) — link
  38. The Econometrics of Financial Markets (1997)
  39. High-Dimensional Probability: An Introduction with Applications in Data Science (2018)
  40. Simplified: A Closer Look at the Virtue of Complexity in Return Prediction (2025)
  41. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2002)
  42. A generalized representer theorem (2001)
  43. Probability inequalities for sums of bounded random variables (1963)
  44. Asset pricing with neural networks: Significance tests (2024)

0 of 52 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.