Paper: arXiv 2510.15995

Authors: Luigi Foscari, Emanuele Guidotti, Nicolò Cesa-Bianchi, Tatjana Chavdarova, Alfio Ferrara

Abstract

We study the emergence of tacit collusion between adaptive trading agents in a stochastic market with endogenous price formation. Using a two-player repeated game between a market maker and a market taker, we characterize feasible and collusive strategy profiles that raise prices beyond competitive levels. We show that, when agents follow simple learning algorithms (e.g., gradient ascent) to maximize their own wealth, the resulting dynamics converge to collusive strategy profiles, even in highly liquid markets with small trade sizes. By highlighting how simple learning strategies naturally lead to tacit collusion, our results offer new insights into the dynamics of AI-driven markets.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper employs advanced mathematical modeling including stochastic processes, game theory, and formal proofs of convergence for gradient ascent dynamics, indicating high math complexity. However, it relies on theoretical simulations and stylized models rather than real-world historical data, backtesting, or implementation-heavy empirical validation.

Research Flowchart

  flowchart TD
  A["Research Goal<br>How do adaptive agents<br>achieve tacit collusion?"] --> B["Methodology<br>Stochastic Game Framework"]
  B --> C["Setup<br>Two-Player Repeated Game<br>Market Maker vs Taker"]
  C --> D["Learning Process<br>Gradient Ascent Optimization"]
  D --> E["Simulation<br>Adaptive Agents in<br>Stochastic Market"]
  E --> F{"Convergence?"}
  F -->|Yes| G["Outcome 1<br>Collusive Equilibrium<br>Price > Competitive Level"]
  F -->|No| H["Outcome 2<br>Competitive Dynamics<br>Price ≈ Competitive Level"]
  G --> I["Key Insight<br>Simple learning strategies<br>naturally lead to tacit collusion"]
  H --> I

Cited References (107)

  1. Potential games (1996)
  2. Markov Games as a Framework for Multi-Agent Reinforcement Learning (1994)
  3. On Multiagent Q-Learning in a Semi-Competitive Domain (1995)
  4. The extragradient method for finding saddle points and other problems (1976)
  5. Competing mechanisms in a common value environment (2000)
  6. Heterogeneous beliefs and routes to chaos in a simple asset pricing model (1998) — link
  7. A modification of the Arrow–Hurwicz method for search of saddle points (1980)
  8. Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm (1998)
and 99 more
  1. Foresight-based pricing algorithms in an economy of software agents (1998) — link
  2. Foresight-based pricing algorithms in agent economies (2000) — link
  3. Pricing in agent economies using multi-agent Q-learning (2002) — link
  4. Price dynamics and quality in information markets (2000) — link
  5. Developing Competition Law for Collusion by Autonomous Artificial Agents (2018) — link
  6. Q-learning agents in a Cournot oligopoly model (2008)
  7. Stochastic Games* (1953) — link
  8. Q-learning (1992) — link
  9. Deep Q-Learning in Airline Dynamic Pricing and Tacit Collusion: Experimental deep RL in airline revenue management in a duopoly setting (2023)
  10. The complexity of computing a Nash equilibrium (2009) — link
  11. Human-level control through deep reinforcement learning (2015)
  12. A Model of Price Formation in Frictionless Markets
  13. Universal portfolios with side information (1996) — link
  14. On-line portfolio selection (1996) — link
  15. A Modern Introduction to Online Learning (2023) — link
  16. Online portfolio selection: A survey (2014) — link
  17. Growth Optimal Investment with Transaction Costs (2008) — link
  18. Universal portfolios with and without transaction costs (1997) — link
  19. Weak aggregating specialist algorithm for online portfolio selection (2023) — link
  20. Game theory, alive (2017)
  21. Competing Against Nash Equilibria in Adversarially Changing Zero - Sum Games (2019) — link
  22. Computing the optimal strategy to commit to — link
  23. Optimal Machine Strategies to Commit to in Two-Person Repeated Games (2015) — link
  24. Reinforcement learning for optimized trade execution (2006)
  25. Market making and mean reversion (2011)
  26. Adaptive market making via online learning (2013)
  27. On the Stackelberg strategy in nonzero-sum games (1973) — link
  28. On the dominance of capitalists leadership in a ` Feedback-Stackelberg ’ solution of a differential game model of capitalism (1985) — link
  29. An anticipative feedback solution for the infinite-horizon, linear-quadratic, dynamic, Stackelberg game (2002) — link
  30. On Coincidence of Feedback Nash Equilibria and Stackelberg Equilibria in Economic Applications of Differential Games (2006) — link
  31. Model-free Resilient Controller Design based on Incentive Feedback Stackelberg Game and Q-learning (2024) — link
  32. Equilibrium computation for extensive games
  33. Computing equilibria for two-person games — link
  34. Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Games — link
  35. Equilibrium computation for two-player games in strategic and extensive form
  36. Meta- Learning in Games (2023) — link
  37. Global Convergence of Multi - Agent Policy Gradient in Markov Potential Games (2021) — link
  38. On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points (2003) — link
  39. Existence of stationary equilibrium strategies in non-zero sum discounted stochastic games with uncountable state space and state-independent transitions (1989) — link
  40. A Simple Adaptive Procedure Leading to Correlated Equilibrium (2000) — link
  41. An introduction to the theory of asymptotic martingales (1983) — link
  42. Artificial Intelligence , Algorithmic Pricing , and Collusion (2020) — link
  43. Artificial Intelligence and Auction Design (2022) — link
  44. Algorithmic collusion: A critical review (2021) — link
  45. Market Making without Regret (2025) — link
  46. Uniswap Liquidity Provision : An Online Learning Approach (2023) — link
  47. Finding the Right Curve : Optimal Design of Constant Function Market Makers (2023) — link
  48. Bandits with Knapsacks (2018) — link
  49. Bandits with Replenishable Knapsacks : the Best of both Worlds (2023) — link
  50. Non-monotonic Resource Utilization in the Bandits with Knapsacks Problem (2022) — link
  51. A Unifying Framework for Online Optimization with Long - Term Constraints (2022) — link
  52. Algorithmic Pricing and Liquidity in Securities Markets (2022) — link
  53. On Stochastic and Worst-case Models for Investing (2009)
  54. Universal portfolios with short sales and margin (1998) — link
  55. Minimax regret portfolios for restricted stock sequences (2004) — link
  56. Model-based Reinforcement Learning for Predictions and Control for Limit Order Books (2019) — link
  57. Learning to simulate realistic limit order book markets from data as a World Agent (2022) — link
  58. Reinforcement Learning for Market Making in a Multi -agent Dealer Market (2019) — link
  59. Deep reinforcement learning for high-frequency market making (2023) — link
  60. Market making via reinforcement learning (2018)
  61. Robust market making via adversarial reinforcement learning (2020)
  62. Robust market making via adversarial reinforcement learning (2021)
  63. Efficient market making via convex optimization, and a connection to online learning (2013) — link
  64. Adapting to a Market Shock: Optimal Sequential Market-Making (2008)
  65. Efficient algorithms for universal portfolios (2000) — link
  66. Cooperation between independent market makers (2022) — link
  67. Trading and exchanges: Market microstructure for practitioners (2002)
  68. The pricing of options and corporate liabilities (1973)
  69. Continuous Auctions and Insider Trading (1985) — link
  70. AI -powered trading, algorithmic collusion, and price efficiency (2025)
  71. Efficient capital markets: A review of theory and empirical work (1970)
  72. Proof that properly anticipated prices fluctuate randomly (2016)
  73. Proof That Properly Anticipated Prices Fluctuate Randomly (1965)
  74. Master curve for price-impact function (2003)
  75. Anomalous price impact and the critical nature of liquidity in financial markets (2011)
  76. Agent-based models for latent liquidity and concave price impact (2014)
  77. Trades, quotes and prices: financial markets under the microscope (2018)
  78. Toward a fully automated stock exchange, part I (1971)
  79. The Square-Root Impace Law Also Holds for Option Markets (2016)
  80. How efficiency shapes market impact (2013)
  81. Q-learning agents in a Cournot oligopoly model (2008) — link
  82. Markets with a continuum of traders (1964)
  83. On the impossibility of informationally efficient markets (1980)
  84. Adaptive Algorithms and Collusion via Coupling (2023) — link
  85. Strategic decompositions of normal form games: Zero -sum games and potential games (2020) — link
  86. Best response equivalence (2004)
  87. Game Theory (2013) — link
  88. Collusion by mistake: Does algorithmic sophistication drive supra-competitive profits? (2024)
  89. Risk aversion in the small and in the large (1978) — link
  90. An online portfolio selection algorithm with regret logarithmic in price variation (2015) — link
  91. Pushing the Efficiency - Regret Pareto Frontier for Online Learning of Portfolios and Quantum States (2022) — link
  92. Efficient and Near - Optimal Online Portfolio Selection (2025) — link
  93. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches (2024) — link
  94. Algorithmic collusion and a folk theorem from learning with bounded rationality (2026) — link
  95. Data dependent regret bounds for online portfolio selection with predicted returns (2025) — link
  96. The Real Effects of Financial Markets (2012)
  97. Learning Unfair Trading : a Market Manipulation Analysis From the Reinforcement Learning Perspective (2015) — link
  98. Mispricing and Algorithm Trading (2025) — link
  99. Recent Advances in Reinforcement Learning in Finance (2021) — link

0 of 107 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.