Paper: arXiv 2601.04062

Authors: Wang Yi, Takashi Hasuike

Abstract

Improvements in return forecast accuracy do not always lead to proportional improvements in portfolio decision quality, especially under realistic trading frictions and constraints. This paper adopts the Smart Predict–then–Optimize (SPO) paradigm for portfolio optimization in real markets, which explicitly aligns the learning objective with downstream portfolio decision quality rather than pointwise prediction accuracy. Within this paradigm, predictive models are trained using an SPO-based surrogate loss that directly reflects the performance of the resulting investment decisions. To preserve interpretability and robustness, we employ linear predictors built on return-based and technical-indicator features and integrate them with portfolio optimization models that incorporate transaction costs, turnover control, and regularization. We evaluate the proposed approach on U.S. ETF data (2015–2025) using a rolling-window backtest with monthly rebalancing. Empirical results show that decision-focused training consistently improves risk-adjusted performance over predict–then–optimize baselines and classical optimization benchmarks, and yields strong robustness during adverse market regimes (e.g., the 2020 COVID-19). These findings highlight the practical value of the Smart Predict–then–Optimize paradigm for portfolio optimization in realistic and non-stationary financial environments.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematics, including convex optimization, SPO surrogate loss derivations, and robustness extensions, while demonstrating strong empirical rigor with a detailed rolling-window backtest on U.S. ETF data spanning 10 years, incorporating transaction costs and regime analysis.

Research Flowchart

  flowchart TD
  Goal["Research Goal: Does decision-focused training improve<br>portfolio performance vs. standard prediction?"]
  
  Data["Input Data: U.S. ETF Daily Returns & Technical Indicators<br>2015-2025"]
  
  Model["Methodology: Smart Predict-then-Optimize (SPO)"] -->
  Model["Linear Predictor + Portfolio Optimization<br>with Transaction Costs & Constraints"]
  
  Train["Computational Process: Rolling-Window Backtest<br>Train: SPO Surrogate Loss vs. MSE<br>Test: Monthly Rebalancing"]
  
  Results["Key Findings: Decision-Focused Training Outperforms"]
  
  Results --> Feat1["Higher Risk-Adjusted Returns<br>(Sharpe Ratio)"]
  Results --> Feat2["Superior Performance in Adverse Regimes<br>(e.g., 2020 COVID-19)"]
  Results --> Feat3["Robustness to Trading Frictions<br>(Transaction Costs)"]
  
  Goal --> Data
  Data --> Model
  Model --> Train
  Train --> Results

Cited References (33)

  1. Portfolio Selection (1952)
  2. The Sharpe Ratio (1998)
  3. Optimization of Conditional Value-at-Risk (2000)
  4. Robust Portfolio Selection Problems (2003) — link
  5. Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps (2003) — link
  6. Multi-Period Trading via Convex Optimization (2017)
  7. Learning to Trade via Direct Reinforcement (2001) — link
  8. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
and 25 more
  1. Deep Direct Reinforcement Learning for Financial Signal Representation and Trading (2017) — link
  2. High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning (2022) — link
  3. Dynamic Datasets and Market Environments for Financial Reinforcement Learning (2024)
  4. A Review of Reinforcement Learning in Financial Applications — → 2411.12746
  5. Prediction-Based Portfolio Optimization Model Using Neural Networks (2009) — link
  6. Deep Neural Networks, Gradient-Boosted Trees, Random Forests: Statistical Arbitrage on the S&P 500 (2017) — link
  7. Portfolio Optimization with Return Prediction Using Deep Learning and Machine Learning (2021) — link
  8. Portfolio Formation with Preselection Using Deep Learning from Long-Term Financial Data (2020) — link
  9. Decision-Making for Financial Trading: A Fusion Approach of Machine Learning and Portfolio Selection (2019) — link
  10. Applied Optimization in Global Efficient Portfolio Construction Using Earning Forecasts (2013) — link
  11. Trading on the Edge: Neural, Genetic, and Fuzzy Systems for Chaotic Financial Markets (1994)
  12. Are Transformers Effective for Time Series Forecasting? (2023) — link
  13. Smart ``Predict, then Optimize’’ (2022) — link
  14. Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities (2024) — link
  15. An End-to-End Portfolio Optimization Framework Based on the SPO Paradigm (2025)
  16. PyEPO: A PyTorch-Based End-to-End Predict-Then-Optimize Library for Linear and Integer Programming (2024) — link
  17. Deep Learning for Portfolio Optimization (2020) — link
  18. A survey on long short-term memory networks for time series prediction (2021) — link
  19. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024)
  20. A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers (2023)
  21. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2023)
  22. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting (2023)
  23. Robust Losses for Decision-Focused Learning (2024) — link
  24. Distributionally Robust Mean-Variance Portfolio Selection with Wasserstein Distances (2022) — link
  25. Decision Trees for Decision-Making under the Predict-then-Optimize Framework (2020)

1 of 33 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.