Paper: arXiv 2510.14986

Authors: Yiyao Zhang, Diksha Goel, Hussain Ahmad, Claudia Szabo

Abstract

Financial markets are inherently non-stationary, with shifting volatility regimes that alter asset co-movements and return distributions. Standard portfolio optimization methods, typically built on stationarity or regime-agnostic assumptions, struggle to adapt to such changes. To address these challenges, we propose RegimeFolio, a novel regime-aware and sector-specialized framework that, unlike existing regime-agnostic models such as DeepVol and DRL optimizers, integrates explicit volatility regime segmentation with sector-specific ensemble forecasting and adaptive mean-variance allocation. This modular architecture ensures forecasts and portfolio decisions remain aligned with current market conditions, enhancing robustness and interpretability in dynamic markets. RegimeFolio combines three components: (i) an interpretable VIX-based classifier for market regime detection; (ii) regime and sector-specific ensemble learners (Random Forest, Gradient Boosting) to capture conditional return structures; and (iii) a dynamic mean-variance optimizer with shrinkage-regularized covariance estimates for regime-aware allocation. We evaluate RegimeFolio on 34 large cap U.S. equities from 2020 to 2024. The framework achieves a cumulative return of 137 percent, a Sharpe ratio of 1.17, a 12 percent lower maximum drawdown, and a 15 to 20 percent improvement in forecast accuracy compared to conventional and advanced machine learning benchmarks. These results show that explicitly modeling volatility regimes in predictive learning and portfolio allocation enhances robustness and leads to more dependable decision-making in real markets.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs only partially modelled; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper uses standard ML models (RF, GBDT) and classic portfolio optimization (mean-variance with shrinkage) rather than heavy advanced mathematics; the implementation details are straightforward. However, it presents strong empirical results with specific backtest metrics (137% return, 1.17 Sharpe) over a clear 2020-2024 dataset of 34 equities, demonstrating high backtest readiness.

Research Flowchart

  flowchart TD
  Goal["Research Goal<br>Develop a Regime-Aware ML System<br>for Sectoral Portfolio Optimization"]

  Data["Data & Inputs<br>34 Large Cap U.S. Equities<br>2020-2024<br>VIX & Sector Data"]

  Proc1["Methodology: Regime Detection<br>Interpretable VIX-based Classifier<br>Identifies Volatility Regimes"]

  Proc2["Methodology: Forecasting<br>Regime & Sector-Specific Ensemble<br>RF/GBM for Conditional Returns"]

  Proc3["Methodology: Optimization<br>Dynamic Mean-Variance<br>Shrinkage-Regularized Covariance"]

  Out["Key Outcomes & Findings<br>Cumulative Return: 137%<br>Sharpe Ratio: 1.17<br>Max Drawdown: -12%<br>Forecast Accuracy: +15-20%"]

  Goal --> Data
  Data --> Proc1
  Proc1 --> Proc2
  Proc2 --> Proc3
  Proc3 --> Out

Cited References (63)

  1. Network centrality and the cross section of stock returns (2017)
  2. What Skills Do Cyber Security Professionals Need? (2025) — link
  3. " I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data (2022) — link
  4. Towards deep learning enabled cybersecurity risk assessment for microservice architectures (2025)
  5. The future of ai: Exploring the potential of large concept models (2025) — link
  6. Resilient Auto-Scaling of Microservice Architectures with Efficient Resource Management (2025) — link
  7. Towards resource-efficient reactive and proactive auto-scaling for microservice architectures (2025)
  8. Chatnvd: Advancing cybersecurity vulnerability assessment with large language models (2024) — link
and 55 more
  1. Machine learning driven smishing detection framework for mobile security (2024) — link
  2. A survey on immersive cyber situational awareness systems (2025)
  3. Smart HPA: A resource-efficient horizontal pod auto-scaler for microservice architectures (2024)
  4. The Cross-Section of Volatility and Expected Returns (2006)
  5. Leveraging LLMs for financial news analysis and macroeconomic indicator nowcasting (2024) — link
  6. Random forests (2001)
  7. Strategic asset allocation (1997)
  8. VIX Index Historical Data (2024)
  9. DRL-PF: A Double-Scales Deep Reinforcement Learning-Based Portfolio Management Framework (2024) — link
  10. A hybrid GARCH , convolutional neural network and long short term memory methods for volatility prediction in stock market (2023)
  11. Federal Reserve Economic Data (2024)
  12. Greedy function approximation: a gradient boosting machine (2001)
  13. The fundamental law of active management (1989)
  14. Asset allocation under multivariate regime switching (2007)
  15. Size and value anomalies under regime shifts (2008)
  16. A new approach to the economic analysis of nonstationary time series and the business cycle (1989)
  17. A deep reinforcement learning framework for dynamic portfolio optimization: Evidence from China’s stock market (2024) — link
  18. Performance hypothesis testing with the Sharpe and Treynor measures (1981)
  19. Dynamic graph neural networks for enhanced volatility prediction in financial markets (2024) — link
  20. A well-conditioned estimator for large-dimensional covariance matrices (2004)
  21. GNN-Forecast : A Graph Neural Network-Based Stock Movement Prediction with Pre-training (2024) — link
  22. Automated volatility forecasting (2024) — link
  23. Ensemble learning algorithms based on easyensemble sampling for financial distress prediction (2024)
  24. Identifying bull and bear markets in stock returns (2000)
  25. Portfolio selection (1952)
  26. Portfolio optimization using a hybrid machine learning stock selection framework (2024)
  27. Machine learning, economic regimes and portfolio optimisation (2018)
  28. Interpretable VIX forecasting using Kolmogorov–Arnold networks (2025)
  29. DeepVol : Volatility forecasting from high-frequency data with dilated causal convolutions (2024)
  30. Anatomy of the VIX spike in February 2018 (2018)
  31. Mutual Fund Performance (1966)
  32. S&P U.S. Indices Methodology (2025)
  33. Network-based prediction of financial cross-sector risk spillover in China : A deep learning approach (2024)
  34. Forecasting intraday volatility and return densities with a new GARCH-f-GAN model (2023)
  35. Deep Graph-Reinforcement Learning for Portfolio Management (2024)
  36. Historical Stock Price Data (2024)
  37. Calmar ratio: A new measure of portfolio performance (1991)
  38. Volatility forecasting with machine learning and intraday commonality (2024)
  39. Econometric measures of connectedness and systemic risk in the finance and insurance sectors (2012)
  40. Fear and greed: A tale of two sentiments (2015)
  41. Markov-switching dynamic factor models in real time (2015)
  42. Asset allocation under multivariate regime switching (2008)
  43. Are stocks really so risky? A long-run perspective (2012)
  44. Empirical asset pricing via machine learning (2020)
  45. A Hidden Markov Model approach to detecting regimes in financial time series (2022)
  46. International stock return comovements (2009)
  47. The macroeconomic effects of investment-specific technology shocks (2014)
  48. Scikit-learn: Machine learning in Python (2011)
  49. A systematic review of reproducibility research in machine learning (2018)
  50. S&P 500® (2025) — link
  51. Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? (2009)
  52. Capital Asset Prices: A Theory of Market Equilibrium Under Conditions of Risk (1964)
  53. International asset allocation with regime shifts (2002)
  54. Hierarchical graph attention network for stock movement prediction (2021)
  55. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection (2003) — link

0 of 63 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.