Paper: arXiv 2610.13126

Authors: Jianning Wang, Robert Ślepaczuk, Paweł Sakowski

Abstract

This paper studies exit timing for SPX short-straddle portfolios using a two-stage supervised and reinforcement-learning framework. Historical episodes first generate ex-post optimal hold/exit labels for training a multilayer perceptron (MLP) using only contemporaneously available state variables at deployment. The trained MLP then initializes reinforcement-learning actors that are fine-tuned with a risk-aware trading reward. Out-of-sample evaluation over 2024-2026 shows that the supervised MLP substantially outperforms passive holding and representative fixed-time and profit-and-loss-based exit rules. Across seven independent seeds, MLP-initialized Actor-Critic averages 37.54% cumulative return and a Sharpe ratio of 1.705, compared with 35.01% and 1.651 for the supervised MLP. Actor-Critic trained from scratch averages only 0.18% return. These findings show that oracle-guided supervised learning provides a stable foundation for reward-based policy fine-tuning in a sparse and path-dependent option-exit problem.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper demonstrates strong empirical rigor through extensive out-of-sample testing, multiple control policies, and detailed portfolio construction. Its novelty lies in the two-stage MLP-initialized RL approach for a challenging options exit timing problem. The mathematical complexity is moderate, combining supervised learning and reinforcement learning frameworks.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimize SPX Short-Straddle Exit Timing] --> B{Data: Historical SPX Options & Market Data};
    B --> C[Stage 1: Supervised Learning (MLP)];
    C -- Ex-post optimal hold/exit labels --> D{Computational Process: Train MLP};
    D -- Initializes --> E[Stage 2: Reinforcement Learning (Actor-Critic)];
    E -- Fine-tuned with risk-aware trading reward --> F{Outcomes: Performance Evaluation};
    F --> G[Key Findings: MLP-Initialized RL Outperforms Others];
    G -- MLP-Initialized RL: 37.54% Return, Sharpe 1.705 --> H[Impact: Stable Foundation for Reward-Based Policy Fine-Tuning];

Cited References (40)

  1. The Pricing of Options and Corporate Liabilities (1973) — link
  2. Theory of Rational Option Pricing (1973) — link
  3. Options Pricing (2026) — link
  4. Options Strategies Quick Guide (2019) — link
  5. S&P 500 Index Options Product Specifications (2026) — link
  6. Expected Option Returns (2001) — link
  7. Delta-Hedged Gains and the Negative Market Volatility Risk Premium (2003) — link
  8. Variance Risk Premiums (2009) — link
and 32 more
  1. Forecasting Volatility and Option Prices of the S&P 500 Index (1994)
  2. Bid-Ask Spreads and Trading Activity in the S&P 100 Index Options Market (1993) — link
  3. Trading Activity and Bid–Ask Spreads of Individual Equity Options (2010) — link
  4. Option Pricing and Replication with Transactions Costs (1985) — link
  5. The Cost of Transacting (1968) — link
  6. Optimal Timing to Purchase Options (2011) — link
  7. Optimization of Conditional Value-at-Risk (2000) — link
  8. Conditional Value-at-Risk for General Loss Distributions (2002) — link
  9. Reinforcement Learning: An Introduction (2018)
  10. Deep Learning (2016)
  11. Understanding the Difficulty of Training Deep Feedforward Neural Networks (2010) — link
  12. Human-Level Control through Deep Reinforcement Learning (2015) — link
  13. Proximal Policy Optimization Algorithms (2017) — link
  14. Actor-Critic Algorithms (2000)
  15. Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning (2019) — link
  16. AWAC : Accelerating Online Reinforcement Learning with Offline Datasets (2020) — link
  17. Deep Hedging (2019) — link
  18. Deep Hedging of Derivatives Using Reinforcement Learning (2020) — link
  19. Deep Reinforcement Learning for Option Replication and Hedging (2020) — link
  20. The QLBS Q-Learner Goes NuQLear : Fitted Q Iteration, Inverse RL , and Option Portfolios (2019) — link
  21. Reinforcement Learning for Options Trading (2021) — link
  22. Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review (2023) — link
  23. Mutual Fund Performance (1966) — link
  24. S&P 500 ( \textasciicircum GSPC ) Historical Data (2026) — link
  25. yfinance: Download Market Data from Yahoo! Finance’s API (2026) — link
  26. Expected Stock Returns and Variance Risk Premia (2009) — link
  27. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning (1992) — link
  28. High-Dimensional Continuous Control Using Generalized Advantage Estimation (2016) — link
  29. Learning to Trade via Direct Reinforcement (2001) — link
  30. When Do Stop-Loss Rules Stop Losses? (2014) — link
  31. A Survey of Robot Learning from Demonstration (2009) — link
  32. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (2011)

0 of 40 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.