Paper: arXiv 2609.33470
Authors: Haochen Luo, Yifan Li, Binh Minh An, Xiaolong Luo, Zhengzhao Lai, Yuan Zhang, Chen Liu
Abstract
Large language models (LLMs) and multi-agent systems (MAS) have shown promise in financial decision-making, yet existing evaluations focus on equity trading and primarily assess directional prediction, overlooking the structural complexity of derivative markets. Option trading introduces fundamentally different challenges, including nonlinear payoffs and multi-leg strategy construction, requiring structured decisions rather than simple directional bets. We introduce LiveOption, an evaluation framework for LLM-based agents in option trading. LiveOption formulates the problem as structured sequential decision-making under realistic execution and capital constraints, and provides a reproducible environment with standardized interaction protocols. The framework includes three task suites covering portfolio overlays, event-driven earnings trading, and 0DTE intraday trading. We further propose a hierarchical metric suite that evaluates action validity, decision quality, risk characteristics, and outcome-level performance. Experiments show that current agents often fail to achieve competitive returns in most scenarios. LiveOption offers a principled testbed for evaluating structured decision-making beyond outcome-based metrics.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: This paper introduces a novel and much-needed evaluation framework for LLM agents in the complex domain of option trading, moving beyond simple equity predictions. It demonstrates strong empirical rigor through its detailed backtesting engine, multi-scenario evaluation, and hierarchical metric suite. The framework’s design for reproducibility and its focus on structured decision-making under realistic constraints are significant strengths.
Research Flowchart
flowchart TD
A[Research Goal: Evaluate LLM Agents in Structured Option Trading] --> B(Methodology: LiveOption Framework Development)
B --> C{Data/Inputs: Realistic Market Data, Execution & Capital Constraints}
C --> D[Computational Processes: LLM Agent Decision-Making, Multi-leg Strategy Construction, Nonlinear Payoff Calculation]
D --> E{Evaluation: Hierarchical Metric Suite - Validity, Quality, Risk, Outcome}
E --> F[Key Finding: Current LLM Agents Struggle to Achieve Competitive Returns in Option Trading]
Cited References (86)
- Leveraging large language models for predicting stock option valuation and financial risk mitigation (2024)
- Deep reinforcement learning for active high frequency trading (2021) — link
- Beat the Market An Effective Intraday Momentum Strategy for S&P500 ETF (SPY) (2024)
- Does 0DTE Options Trading Increase Volatility? (2023)
- Trading Options at Expiration: Strategies and Models for Winning the Endgame (2009)
- Day Trading Options: Profiting from Price Distortions in Very Brief Time Frames (2009)
- Simulating Dynamics of the Recognition-Primed Decision Model on Financial Markets Using LLM (2026)
- Machine learning methods for pricing financial derivatives — → 2406.00459
and 78 more
- Developing AI-Powered Predictive Analytics for Derivatives Trading: Machine Learning Models for Market Volatility Forecasting, Risk Mitigation, and Strategy Optimization (2023)
- Implied volatility directional forecasting: a machine learning approach (2021)
- AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets (2025) — link
- Learning to Use Tools via Cooperative and Interactive Agents (2024) — link
- Multi-agent collaboration mechanisms: A survey of llms (2025) — link
- A survey of agent interoperability protocols: Model context protocol (mcp), agent communication protocol (acp), agent-to-agent protocol (a2a), and agent network protocol (anp) (2025) — link
- Docfinqa: A long-context financial reasoning dataset (2024)
- Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance (2023)
- Finqa: A dataset of numerical reasoning over financial data (2021)
- On the estimation of security price volatilities from historical data (1980)
- AlphaBench: Benchmarking Large Language Models in Formulaic Alpha Factor Mining (2026)
- Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance — → 2309.03736
- Large Language Models as Tool Makers (2024) — link
- React: Synergizing reasoning and acting in language models (2022)
- Toolformer: Language Models Can Teach Themselves to Use Tools (2023) — link
- Gorilla: Large Language Model Connected with Massive API s (2024) — link
- Model context protocol (mcp): Landscape, security threats, and future research directions (2025)
- QF in Z ero: A Unified Financial Toolchain for LLM -Based Trading Agents (2026) — link
- A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions (2025)
- TradingAgents: Multi-Agents LLM Financial Trading Framework — → 2412.20138
- A comprehensive survey of contamination detection methods in large language models (2024) — link
- Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs (2024)
- Constat: Performance-based contamination detection in large language models (2024)
- Finmem: A performance-enhanced llm trading agent with layered memory and character design (2025)
- Cfbenchmark: Chinese financial assistant benchmark for large language model (2023) — link
- Financebench: A new benchmark for financial question answering (2023) — link
- Finben: A holistic financial benchmark for large language models (2024)
- Investorbench: A benchmark for financial decision-making tasks with llm-based agent (2025)
- Flag-trader: Fusion llm-agent with gradient-based reinforcement learning for financial trading (2025)
- CryptoTrade: A reflective LLM-based agent to guide zero-shot cryptocurrency trading (2024)
- Qwen2.5 technical report (2024) — link
- Benchmarking benchmark leakage in large language models (2024) — link
- The memorization problem: Can we trust llms’ economic forecasts? — → 2504.14765
- Assessing look-ahead bias in stock return predictions generated by gpt sentiment analysis — → 2309.17322
- Evaluating LLMs in Finance Requires Explicit Bias Consideration — → 2602.14233
- C laude S onnet 4 — anthropic.com
- When Agents Trade: Live Multi-Market Trading Arena for LLM Agents (2026)
- Stockbench: Can llm agents trade stocks profitably in real-world markets? (2025) — link
- Gpt-4 technical report (2023) — link
- Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities (2025) — link
- Deepseek-v3 technical report (2024) — link
- Forecasting implied volatility smile surface via deep learning and attention mechanism (2019) — link
- Improving option price forecasts with neural networks and support vector regressions (2009)
- Financial density forecasts: A comprehensive comparison of risk-neutral and historical schemes (2018)
- The pricing of options and corporate liabilities (1973)
- A neural network versus Black–Scholes: a comparison of pricing and hedging performances (2003)
- Deep hedging of derivatives using reinforcement learning (2021) — link
- Retail Traders Love 0DTE Options… But Should They? (2023)
- Pricing and hedging short sterling options using neural networks (2012)
- Deep learning for options trading: an end-to-end approach (2024)
- HedgeAgents: A Balanced-aware Multi-agent Financial Trading System (2025) — link
- Dynamic Hedging Strategies in Derivatives Markets with LLM-Driven Sentiment and News Analytics (2025)
- Options Pricing Platform with Neural Networks, LLMs and Reinforcement Learning (2025)
- Kronos: A Foundation Model for the Language of Financial Markets (2025) — link
- Stocktime: A time series specialized large language model architecture for stock price prediction — → 2409.08281
- Alpha-gpt: Human-ai interactive alpha mining for quantitative investment (2025)
- Stock Market Forecasting: From Traditional Predictive Models to Large Language Models: M. Darwish et al. (2025)
- Futurex: An advanced live benchmark for llm agents in future prediction (2025) — link
- FinVision: A multi-agent framework for stock market prediction (2024)
- Comparative analysis of LLM-based market prediction and human expertise with sentiment analysis and machine learning integration (2024)
- Cognitive Alpha Mining via LLM-Driven Code-Based Evolution (2025) — link
- AlphaAgent: LLM-driven alpha mining with regularized exploration to counteract alpha decay (2025)
- Navigating the alpha jungle: an LLM-Powered MCTS framework for formulaic factor mining (2025) — link
- Quantagent: Price-driven multi-agent llms for high-frequency trading (2025) — link
- From Natural Language to Executable Option Strategies via Large Language Models — → 2603.16434
- TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets (2025) — link
- Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making (2024)
- Fintextqa: A dataset for long-form financial question answering (2024)
- Kelly-based options trading strategies on settlement date via supervised learning algorithms (2022)
- OPHR : Mastering Volatility Trading with Multi-Agent Deep Reinforcement Learning (2025) — link
- Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking (2025) — link
- DeepSeek-V4 : Towards Highly Efficient Million-Token Context Intelligence (2026) — link
- gpt-oss-120b & gpt-oss-20b Model Card (2025) — link
- Qwen3 Technical Report (2025) — link
- The Llama 3 Herd of Models (2024) — link
- MiniMax Sparse Attention (2026) — link
- GLM-5 : from Vibe Coding to Agentic Engineering (2026) — link
- GLM-5.3 — Model Overview (2026)
8 of 86 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.