Paper: arXiv 2512.12924

Authors: Gagan Deep, Akash Deep, William Lamptey

Abstract

We develop a rigorous walk-forward validation framework for algorithmic trading designed to mitigate overfitting and lookahead bias. Our methodology combines interpretable hypothesis-driven signal generation with reinforcement learning and strict out-of-sample testing. The framework enforces strict information set discipline, employs rolling window validation across 34 independent test periods, maintains complete interpretability through natural language hypothesis explanations, and incorporates realistic transaction costs and position constraints. Validating five market microstructure patterns across 100 US equities from 2015 to 2024, the system yields modest annualized returns (0.55%, Sharpe ratio 0.33) with exceptional downside protection (maximum drawdown -2.76%) and market-neutral characteristics (beta = 0.058). Performance exhibits strong regime dependence, generating positive returns during high-volatility periods (0.60% quarterly, 2020-2024) while underperforming in stable markets (-0.16%, 2015-2019). We report statistically insignificant aggregate results (p-value 0.34) to demonstrate a reproducible, honest validation protocol that prioritizes interpretability and extends naturally to advanced hypothesis generators, including large language models. The key empirical finding reveals that daily OHLCV-based microstructure signals require elevated information arrival and trading activity to function effectively. The framework provides complete mathematical specifications and open-source implementation, establishing a template for rigorous trading system evaluation that addresses the reproducibility crisis in quantitative finance research. For researchers, practitioners, and regulators, this work demonstrates that interpretable algorithmic trading strategies can be rigorously validated without sacrificing transparency or regulatory compliance.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 9.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical frameworks for walk-forward validation and reinforcement learning, but its core contribution is a rigorous, implementation-heavy empirical protocol with extensive backtesting data, strict information discipline, and open-source code.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Develop Rigorous Validation<br>for Microstructure Trading"] --> B["Methodology: Walk-Forward Framework<br>Signal Gen & Strict OOS Testing"]
  B --> C["Data Input:<br>5 MS Patterns, 100 US Equities<br>2015-2024"]
  C --> D["Computational Process:<br>34 Rolling Windows +<br>Transaction Costs +<br>RL Optimization"]
  D --> E["Outcome 1: Modest Returns<br>0.55% Annualized, Sharpe 0.33"]
  D --> F["Outcome 2: Regime Dependence<br>+0.60% High Vol vs -0.16% Stable"]
  D --> G["Outcome 3: Stat. Insignificant<br>P-value 0.34<br>Reproducible Template"]

Cited References (39)

  1. … and the cross-section of expected returns (2016)
  2. Why most published research findings are false (2005)
  3. Does academic research destroy stock return predictability? (2016)
  4. Replicating anomalies (2020)
  5. Is there a replication crisis in finance? (2023)
  6. Design, testing, and optimization of trading systems (1992)
  7. The evaluation and optimization of trading strategies (2008)
  8. Pseudo-mathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance (2014)
and 31 more
  1. The deflated Sharpe ratio: correcting for selection bias, backtest overfitting, and non-normality (2014)
  2. The probability of backtest overfitting (2017)
  3. Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods in a synthetic controlled environment (2024)
  4. Empirical asset pricing via machine learning (2020)
  5. Deep learning with long short-term memory networks for financial market predictions (2018)
  6. A unified approach to interpreting model predictions (2017)
  7. Dissecting characteristics nonparametrically (2020)
  8. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead (2019)
  9. Interpretable machine learning for imbalanced credit scoring datasets (2021)
  10. Continuous auctions and insider trading (1985)
  11. Liquidity, information, and infrequently traded stocks (1996)
  12. Is information risk a determinant of asset returns? (2002)
  13. One security, many markets: Determining the contributions to price discovery (1995)
  14. Flow toxicity and liquidity in a high-frequency world (2012)
  15. BV-VPIN: Measuring the impact of order flow toxicity and liquidity on international equities markets (2016)
  16. Directional trading in option-to-stock volume and the predictability of future returns (2024)
  17. Learning to trade via direct reinforcement (2001)
  18. Deep direct reinforcement learning for financial signal representation and trading (2017)
  19. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  20. Data-snooping biases in tests of financial asset pricing models (1990)
  21. Data snooping, technical trading rule performance, and the bootstrap (1999)
  22. Regime-dependent performance evaluation and portfolio construction using machine learning (2022) — link
  23. " Why should I trust you?" Explaining the predictions of any classifier (2016)
  24. Interpretable machine learning (2020)
  25. Contemporary symbolic regression methods and their relative performance (2021)
  26. Hybrid AI: Combining symbolic and machine learning approaches (2022)
  27. Evaporating liquidity (2012)
  28. Algorithmic trading and information (2011)
  29. Machine learning and the futures markets: A systematic approach to hypothesis-driven artificial intelligence (2020)
  30. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI (2020)
  31. Advanced financial market forecasting: integrating Monte Carlo simulations with ensemble machine learning models (2024) — link

0 of 39 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.