Paper: arXiv 2609.33470

Authors: Haochen Luo, Yifan Li, Binh Minh An, Xiaolong Luo, Zhengzhao Lai, Yuan Zhang, Chen Liu

Abstract

Large language models (LLMs) and multi-agent systems (MAS) have shown promise in financial decision-making, yet existing evaluations focus on equity trading and primarily assess directional prediction, overlooking the structural complexity of derivative markets. Option trading introduces fundamentally different challenges, including nonlinear payoffs and multi-leg strategy construction, requiring structured decisions rather than simple directional bets. We introduce LiveOption, an evaluation framework for LLM-based agents in option trading. LiveOption formulates the problem as structured sequential decision-making under realistic execution and capital constraints, and provides a reproducible environment with standardized interaction protocols. The framework includes three task suites covering portfolio overlays, event-driven earnings trading, and 0DTE intraday trading. We further propose a hierarchical metric suite that evaluates action validity, decision quality, risk characteristics, and outcome-level performance. Experiments show that current agents often fail to achieve competitive returns in most scenarios. LiveOption offers a principled testbed for evaluating structured decision-making beyond outcome-based metrics.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper introduces a novel and much-needed evaluation framework for LLM agents in the complex domain of option trading, moving beyond simple equity predictions. It demonstrates strong empirical rigor through its detailed backtesting engine, multi-scenario evaluation, and hierarchical metric suite. The framework’s design for reproducibility and its focus on structured decision-making under realistic constraints are significant strengths.

Research Flowchart

  flowchart TD
    A[Research Goal: Evaluate LLM Agents in Structured Option Trading] --> B(Methodology: LiveOption Framework Development)
    B --> C{Data/Inputs: Realistic Market Data, Execution & Capital Constraints}
    C --> D[Computational Processes: LLM Agent Decision-Making, Multi-leg Strategy Construction, Nonlinear Payoff Calculation]
    D --> E{Evaluation: Hierarchical Metric Suite - Validity, Quality, Risk, Outcome}
    E --> F[Key Finding: Current LLM Agents Struggle to Achieve Competitive Returns in Option Trading]

Cited References (86)

  1. Leveraging large language models for predicting stock option valuation and financial risk mitigation (2024)
  2. Deep reinforcement learning for active high frequency trading (2021) — link
  3. Beat the Market An Effective Intraday Momentum Strategy for S&P500 ETF (SPY) (2024)
  4. Does 0DTE Options Trading Increase Volatility? (2023)
  5. Trading Options at Expiration: Strategies and Models for Winning the Endgame (2009)
  6. Day Trading Options: Profiting from Price Distortions in Very Brief Time Frames (2009)
  7. Simulating Dynamics of the Recognition-Primed Decision Model on Financial Markets Using LLM (2026)
  8. Machine learning methods for pricing financial derivatives — → 2406.00459
and 78 more
  1. Developing AI-Powered Predictive Analytics for Derivatives Trading: Machine Learning Models for Market Volatility Forecasting, Risk Mitigation, and Strategy Optimization (2023)
  2. Implied volatility directional forecasting: a machine learning approach (2021)
  3. AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets (2025) — link
  4. Learning to Use Tools via Cooperative and Interactive Agents (2024) — link
  5. Multi-agent collaboration mechanisms: A survey of llms (2025) — link
  6. A survey of agent interoperability protocols: Model context protocol (mcp), agent communication protocol (acp), agent-to-agent protocol (a2a), and agent network protocol (anp) (2025) — link
  7. Docfinqa: A long-context financial reasoning dataset (2024)
  8. Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance (2023)
  9. Finqa: A dataset of numerical reasoning over financial data (2021)
  10. On the estimation of security price volatilities from historical data (1980)
  11. AlphaBench: Benchmarking Large Language Models in Formulaic Alpha Factor Mining (2026)
  12. Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance — → 2309.03736
  13. Large Language Models as Tool Makers (2024) — link
  14. React: Synergizing reasoning and acting in language models (2022)
  15. Toolformer: Language Models Can Teach Themselves to Use Tools (2023) — link
  16. Gorilla: Large Language Model Connected with Massive API s (2024) — link
  17. Model context protocol (mcp): Landscape, security threats, and future research directions (2025)
  18. QF in Z ero: A Unified Financial Toolchain for LLM -Based Trading Agents (2026) — link
  19. A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions (2025)
  20. TradingAgents: Multi-Agents LLM Financial Trading Framework — → 2412.20138
  21. A comprehensive survey of contamination detection methods in large language models (2024) — link
  22. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs (2024)
  23. Constat: Performance-based contamination detection in large language models (2024)
  24. Finmem: A performance-enhanced llm trading agent with layered memory and character design (2025)
  25. Cfbenchmark: Chinese financial assistant benchmark for large language model (2023) — link
  26. Financebench: A new benchmark for financial question answering (2023) — link
  27. Finben: A holistic financial benchmark for large language models (2024)
  28. Investorbench: A benchmark for financial decision-making tasks with llm-based agent (2025)
  29. Flag-trader: Fusion llm-agent with gradient-based reinforcement learning for financial trading (2025)
  30. CryptoTrade: A reflective LLM-based agent to guide zero-shot cryptocurrency trading (2024)
  31. Qwen2.5 technical report (2024) — link
  32. Benchmarking benchmark leakage in large language models (2024) — link
  33. The memorization problem: Can we trust llms’ economic forecasts? — → 2504.14765
  34. Assessing look-ahead bias in stock return predictions generated by gpt sentiment analysis — → 2309.17322
  35. Evaluating LLMs in Finance Requires Explicit Bias Consideration — → 2602.14233
  36. C laude S onnet 4 — anthropic.com
  37. When Agents Trade: Live Multi-Market Trading Arena for LLM Agents (2026)
  38. Stockbench: Can llm agents trade stocks profitably in real-world markets? (2025) — link
  39. Gpt-4 technical report (2023) — link
  40. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities (2025) — link
  41. Deepseek-v3 technical report (2024) — link
  42. Forecasting implied volatility smile surface via deep learning and attention mechanism (2019) — link
  43. Improving option price forecasts with neural networks and support vector regressions (2009)
  44. Financial density forecasts: A comprehensive comparison of risk-neutral and historical schemes (2018)
  45. The pricing of options and corporate liabilities (1973)
  46. A neural network versus Black–Scholes: a comparison of pricing and hedging performances (2003)
  47. Deep hedging of derivatives using reinforcement learning (2021) — link
  48. Retail Traders Love 0DTE Options… But Should They? (2023)
  49. Pricing and hedging short sterling options using neural networks (2012)
  50. Deep learning for options trading: an end-to-end approach (2024)
  51. HedgeAgents: A Balanced-aware Multi-agent Financial Trading System (2025) — link
  52. Dynamic Hedging Strategies in Derivatives Markets with LLM-Driven Sentiment and News Analytics (2025)
  53. Options Pricing Platform with Neural Networks, LLMs and Reinforcement Learning (2025)
  54. Kronos: A Foundation Model for the Language of Financial Markets (2025) — link
  55. Stocktime: A time series specialized large language model architecture for stock price prediction — → 2409.08281
  56. Alpha-gpt: Human-ai interactive alpha mining for quantitative investment (2025)
  57. Stock Market Forecasting: From Traditional Predictive Models to Large Language Models: M. Darwish et al. (2025)
  58. Futurex: An advanced live benchmark for llm agents in future prediction (2025) — link
  59. FinVision: A multi-agent framework for stock market prediction (2024)
  60. Comparative analysis of LLM-based market prediction and human expertise with sentiment analysis and machine learning integration (2024)
  61. Cognitive Alpha Mining via LLM-Driven Code-Based Evolution (2025) — link
  62. AlphaAgent: LLM-driven alpha mining with regularized exploration to counteract alpha decay (2025)
  63. Navigating the alpha jungle: an LLM-Powered MCTS framework for formulaic factor mining (2025) — link
  64. Quantagent: Price-driven multi-agent llms for high-frequency trading (2025) — link
  65. From Natural Language to Executable Option Strategies via Large Language Models — → 2603.16434
  66. TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets (2025) — link
  67. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2024)
  68. Fintextqa: A dataset for long-form financial question answering (2024)
  69. Kelly-based options trading strategies on settlement date via supervised learning algorithms (2022)
  70. OPHR : Mastering Volatility Trading with Multi-Agent Deep Reinforcement Learning (2025) — link
  71. Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking (2025) — link
  72. DeepSeek-V4 : Towards Highly Efficient Million-Token Context Intelligence (2026) — link
  73. gpt-oss-120b & gpt-oss-20b Model Card (2025) — link
  74. Qwen3 Technical Report (2025) — link
  75. The Llama 3 Herd of Models (2024) — link
  76. MiniMax Sparse Attention (2026) — link
  77. GLM-5 : from Vibe Coding to Agentic Engineering (2026) — link
  78. GLM-5.3 — Model Overview (2026)

8 of 86 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.