Paper: arXiv 2407.18957

Abstract

Can AI Agents simulate real-world trading environments to investigate the impact of external factors on stock trading activities (e.g., macroeconomics, policy changes, company fundamentals, and global events)? These factors, which frequently influence trading behaviors, are critical elements in the quest for maximizing investors’ profits. Our work attempts to solve this problem through large language model based agents. We have developed a multi-agent AI system called StockAgent, driven by LLMs, designed to simulate investors’ trading behaviors in response to the real stock market. The StockAgent allows users to evaluate the impact of different external factors on investor trading and to analyze trading behavior and profitability effects. Additionally, StockAgent avoids the test set leakage issue present in existing trading simulation systems based on AI Agents. Specifically, it prevents the model from leveraging prior knowledge it may have acquired related to the test data. We evaluate different LLMs under the framework of StockAgent in a stock trading environment that closely resembles real-world conditions. The experimental results demonstrate the impact of key external factors on stock market trading, including trading behavior and stock price fluctuation rules. This research explores the study of agents’ free trading gaps in the context of no prior knowledge related to market data. The patterns identified through StockAgent simulations provide valuable insights for LLM-based investment advice and stock recommendation. The code is available at https://github.com/MingyuJ666/Stockagent.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper is primarily a system-building and experimental work using LLM agents for simulated trading, relying heavily on implementation, code availability, and empirical evaluation of trading behaviors rather than advanced mathematical derivations. It features a GitHub link, backtesting environments, and analysis of external factors, indicating high empirical rigor but relatively low mathematical complexity.

Research Flowchart

  flowchart TD
  Start["Research Goal<br>Simulate real-world trading to evaluate external factors' impact"] --> Methodology["Methodology<br>Build StockAgent: Multi-agent LLM system avoiding test leakage"]
  Methodology --> Inputs["Data/Inputs<br>Real stock market data, macroeconomic indicators, news events"]
  Inputs --> Process["Computational Process<br>LLM agents trade in simulated environment without prior knowledge of test data"]
  Process --> Findings["Key Findings<br>1. Identified impact of external factors on trading behavior<br>2. Revealed stock price fluctuation patterns<br>3. LLMs can provide valuable investment advice<br>4. Code available: github.com/MingyuJ666/Stockagent"]

Cited References (79)

  1. Does the stock market overreact? (1985)
  2. Market liquidity: theory, evidence, and policy (2013)
  3. Behavior pattern of individual investors in stock market (2014)
  4. Institutional traders’ behavior in an emerging stock market: Empirical evidence on polish pension fund investors (2005)
  5. The real effects of political uncertainty: Elections and investment sensitivity to stock prices (2010)
  6. Who gambles in the stock market? (2009)
  7. The effects of public sentiments and feelings on stock market behavior: Evidence from Australia (2022)
  8. A hybrid stock trading framework integrating technical analysis with machine learning techniques (2016)
and 71 more
  1. Automated trading systems statistical and machine learning methods and hardware implementation: a survey (2019)
  2. Decision-making for financial trading: A fusion approach of machine learning and portfolio selection (2019)
  3. The impact of sentiment and attention measures on stock market volatility (2020)
  4. Stakeholder integration, environmental sustainability orientation and financial performance (2020)
  5. Generative agents: Interactive simulacra of human behavior (2023)
  6. V-irl: Grounding virtual intelligence in real life (2024) — link
  7. Evaluating multi-agent coordination abilities in large language models (2023) — link
  8. Presidential decision-making during the Cuban missile crisis: a computer simulation (1970)
  9. BioFinBERT: Finetuning Large Language Models (LLMs) to Analyze Sentiment of Press Releases and Financial Text Around Inflection Points of Biotech Stocks — → 2401.11011
  10. Analysis of CBDC narrative by central banks using large language models (2023)
  11. FinBERT: A large language model for extracting information from financial text (2023)
  12. Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations (2023) — link
  13. Optimized Financial Planning: Integrating Individual and Cooperative Budgeting Models with LLM Recommendations (2023)
  14. An attempt to simulate the outbreak of World War I (1967)
  15. Gauging similarity with n-grams: Language-independent categorization of text (1995)
  16. Recurrent neural networks (2001)
  17. Long short-term memory-networks for machine reading (2016) — link
  18. Attention is all you need (2017)
  19. Don’t stop pretraining: Adapt language models to domains and tasks (2020) — link
  20. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  21. A survey of zero-shot learning: Settings, methods, and applications (2019)
  22. The rise and potential of large language model based agents: A survey (2023) — link
  23. Connectionist AI, symbolic AI, and the brain (1987)
  24. Experiences with an architecture for intelligent, reactive agents (1997)
  25. Application of reinforcement learning for agent-based production scheduling (2005)
  26. Fast adaptation via Meta learning in multi-agent cooperative tasks (2019)
  27. War and peace (waragent): Large language model-based multi-agent simulation of world wars (2023) — link
  28. The Impact of Reasoning Step Length on Large Language Models (2024) — link
  29. Exploring large language models for communication games: An empirical study on werewolf (2023) — link
  30. Put your money where your mouth is: Evaluating strategic planning and execution of llm agents in an auction arena (2023) — link
  31. Openagi: When llm meets domain experts (2023) — link
  32. Fundamentals of corporate finance (2013)
  33. Language models are few-shot learners (2020)
  34. Gpt-4 technical report (2023) — link
  35. Gemini: a family of highly capable multimodal models (2023) — link
  36. Behavioral finance (2015)
  37. Behavioral finance: an overview (2010)
  38. Machine Learning for Algorithmic Trading: Predictive models to extract signals from market and alternative data for systematic trading strategies with Python (2020)
  39. Backtesting of Trading Strategies for Bitcoin (2019)
  40. Python for Algorithmic Trading (2020)
  41. ZipLine: in-network compression at line speed (2020)
  42. Stock Trading Quantified: An Exploration of Algorithmic Trading Principles using QuantConnect (2020)
  43. ABIDES-gym: gym environments for multi-agent discrete event simulation and application to financial markets (2021)
  44. Trading and Investing Systems Analysis (2021)
  45. A review of backtesting and backtesting procedures (2005)
  46. What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents (2024) — link
  47. Revolutionizing finance with llms: An overview of applications and insights (2024) — link
  48. Communicative agents for software development (2023) — link
  49. Metagpt: Meta programming for multi-agent collaborative framework (2023) — link
  50. Chain-of-thought prompting elicits reasoning in large language models (2022)
  51. Self-consistency improves chain of thought reasoning in language models (2022) — link
  52. Star: Self-taught reasoner bootstrapping reasoning with reasoning (2022)
  53. A behavioral model of rational choice (1955)
  54. Prospect theory: An analysis of decision under risk (1979)
  55. Irrational exuberance: Revised and expanded third edition (2015)
  56. Risk as feelings. (2001)
  57. Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. (1974)
  58. FinAgent: A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist — → 2402.18485
  59. Multi-agent systems (2008)
  60. Trading Gym: An open source project for the development of reinforcement learning algorithms in the context of trading (2024)
  61. The probability of backtest overfitting (2016)
  62. Backtest Overfitting in the Machine Learning Era: A Comparison of Out-of-Sample Testing Methods in a Synthetic Controlled Environment (2024)
  63. Autogen: Enabling next-gen llm applications via multi-agent conversation framework (2023) — link
  64. BUILDING AN ECONOMETRIC MODEL OF SELECTED FACTORS’IMPACT ON STOCK PRICE: A CASE STUDY. (2020)
  65. Stock exchanges and the new markets for securities laws (2008)
  66. Trading Mechanism (2023) — link
  67. Nasdaq Listing Center (2024)
  68. Mengnan Du, Yongfeng Zhang, and Yanda Meng. 2024. Health-LLM: Personalized Retrieval-Augmented Disease Prediction Model (2024) — link
  69. Knowledge Graph Large Language Model (KG-LLM) for Link Prediction (2024) — link
  70. Goal-guided Generative Prompt Injection Attack on Large Language Models (2024) — link
  71. Target-driven Attack for Large Language Models (2024)

2 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.