Every strategy result comes from a simulation of some kind; the question is what that kind is structurally unable to tell you. This page separates three families, replay backtests, agent-based market simulators and generative synthetic markets, by what they can validate, then surveys how papers in the archive use them, including LLM trading agents.

Replay engines: the backtest

A replay engine feeds historical prices or order-book snapshots to a strategy and records hypothetical fills. It is the only simulator grounded entirely in what actually happened, which is its strength and its ceiling. It can validate signal timing, turnover and gross-to-net arithmetic. It cannot validate anything that depends on the market reacting to you: fills are assumed, impact is a cost model bolted on afterwards, and your orders never change the tape.

The failure modes are documented across this site: look-ahead and point-in-time errors, survivorship, overfitting through repeated trials, and cost assumptions that quietly set the result. The eight questions for a backtest cover the standard audit. Two corpus papers show where replay is the right tool. A 2025 study runs 480 million Monte Carlo simulations of crypto buy-and-hold across 378 assets, net of fees and T-bill opportunity cost, and finds a median excess return of minus 28.4% at the two-to-three-year horizon with gains concentrated in the top quartile. That is resampling of real paths, and it works because buy-and-hold has no impact to model. A 2025 crypto forecasting paper tagged trading simulation finds that lower prediction error did not translate into higher returns in its trading simulations, a result that only a P&L-level replay could surface.

Agent-based simulators: markets that react

Agent-based models (ABMs) populate a matching engine with heterogeneous agents and let price form endogenously. That is the only family in which market impact, queue position and reflexivity are emergent rather than assumed. The cost is calibration: an ABM reproduces the stylized facts you tuned it to reproduce.

A 2023 paper building an agent-based market microstructure simulation of the fragmented US equity market, with propagation delays and realistic auctions, argues that most ABMs fail on exactly two counts: missing microstructure mechanics and agents too simple to produce realistic data. ABIDES-MARL (2025) extends the ABIDES framework for multi-agent RL and validates it by recovering the gradual price discovery of an extended Kyle model. The most practitioner-facing entry is a 2026 paper by Noble, Rosenbaum and Souilmi on bridging the reality gap in LOB simulation for large-tick assets: event timing calibrated to show the latency-race mode at exchange round-trip time, a power-law decay feedback kernel that yields concave impact and partial post-trade reversion, and strategy case studies where profitability becomes highly sensitive to execution parameters. That sensitivity is the finding: a strategy whose P&L flips with fill assumptions has not been validated by any replay.

Point-process simulators are the statistical middle ground. A 2025 paper combines a deterministic C++ LOB with multivariate Hawkes order flow calibrated on LOBSTER and Binance data, finding the nearly-unstable subcritical regime essential for realistic clustering; code and data are public. The Multidimensional Deep Queue-Reactive model (2025) reproduces the square-root impact law and cross-queue correlations on Bund futures. A 2025 artificial market generator from Maitrier, Loeper and Bouchaud reproduces the square-root law from superposed correlated metaorders. For execution research, a 2025 paper evaluates RL execution strategies in a reactive ABM that decomposes slippage into impact and timing risk against the Almgren-Chriss frontier. Companion guides: evaluating a microstructure paper, market-making mechanics and the market-making research map.

Generative synthetic markets

Generative models learn the conditional distribution of order flow or returns and sample new paths, promising unlimited training data and stress scenarios that never occurred. They cannot promise the sampled paths are economically possible, and most have no mechanism for your own orders to change the next state.

LOB-Bench (2025) is the shared yardstick, measuring distributional distances plus impact-response metrics, and finds autoregressive generative models beating a CGAN and a parametric LOB model. TRADES (2025) applies transformer diffusion to order flow and tests responsiveness to an experimental agent. Painting the market (2025) treats the book as an image and uses diffusion inpainting to avoid autoregressive error accumulation, reporting state-of-the-art LOB-Bench results. TradeFM (2026) trains a 524M-parameter transformer on billions of trades across more than 9,000 equities and reports two to three times lower distributional error than compound Hawkes baselines.

The honest caveats come from the same literature. TABL-ABM (2025) couples a forecaster to a Chiarella agent model and reports realistic prices while parts of the microstructure are not accurately recreated. Towards Causal Market Simulators (2025) adds structural causal constraints for counterfactuals but validates only on synthetic Ornstein-Uhlenbeck data. ProteuS (2025) takes the semi-synthetic route, fitting ARMA-GARCH regimes to ETF data and splicing them with known break points, which is the only way to get ground truth for regime-detection methods. A 2026 paper uses a macro-conditioned GAN as an adversarial training environment for robust policies, evaluated out of sample through 2020.

LLM trading agents in simulation

With LLM agents, the agent, not the market, is the object under test. A 2025 open-source simulated stock market with LLM traders finds agents follow assigned strategies and that markets show price discovery and bubbles, while noting that shared prompts generate correlated behaviour. A 2025 experimental-finance replication finds LLM agents do not replicate human traders: they price near fundamentals and produce muted bubbles. A 2026 study of 15 LLMs in a simulated asset market finds a spectrum from stable coordination to speculative bubbles, with mixed-model markets varying substantially across repeated runs. The Agent Trading Arena (2025) lets agents move prices in a zero-sum market and finds LLMs struggle with numerical reasoning from plain text. Two audit papers matter most: Standard Benchmarks Fail (2025) argues return-based scores hide hallucination and stale-data risk, and a 2026 scoping review of execution realism in LLM trading papers charts point-in-time controls, temporal splits, costs, execution semantics and artifact availability across the literature. Read these alongside how to evaluate an RL trading paper and the NLP and LLM hub.

A checklist for judging a simulation-based result

  1. Which family? Replay, agent-based, generative, or a hybrid. Each has a different thing it cannot see.
  2. Does the market respond to the agent? If not, impact and reflexivity are assumptions, and the result is an upper bound.
  3. What is the fill model? Queue position, latency, partial fills, cancellations. The reality-gap paper above shows P&L can hinge on these alone.
  4. What was the simulator calibrated on, and is it the same data used to evaluate the strategy? Tuning and testing on one period is in-sample by another name.
  5. Which stylized facts are reproduced, and were they targets or out-of-sample checks? Heavy tails and volatility clustering are easy; impact response functions and cross-queue structure are hard.
  6. Run-to-run variance. Seeds, agent populations and sampled paths all vary. One run is an anecdote; the LLM papers show dispersion can dominate.
  7. Is there any real-market anchor? A replay on real data, a comparison of simulated to real fill distributions, or a live paper-trading period.
  8. Code and data. Several corpus simulators publish both. See reproducing quant research.

How to read these papers

Only two papers carry the exact tag; the related work lives under the market microstructure, reinforcement learning and HFT and execution hubs and the LOB prediction map. Start with the reality-gap simulator to see what fill realism costs a strategy, then LOB-Bench for how generative simulators are scored, then the execution realism review for the audit framework. When a paper’s headline depends on a simulator, apply the checklist before the Sharpe ratio.

References: Almgren & Chriss, “Optimal Execution of Portfolio Transactions” (2001); Kyle, “Continuous Auctions and Insider Trading” (1985); Huang, Lehalle & Rosenbaum, “Simulating and Analyzing Order Book Data: The Queue-Reactive Model” (2015).

Broader area: Market Simulation · All topics: research topics → · Full archive: every paper →