Paper: arXiv 2411.08899

Abstract

Financial trading has been a challenging task, as it requires the integration of vast amounts of data from various modalities. Traditional deep learning and reinforcement learning methods require large training data and often involve encoding various data types into numerical formats for model input, which limits the explainability of model behavior. Recently, LLM-based agents have demonstrated remarkable advancements in handling multi-modal data, enabling them to execute complex, multi-step decision-making tasks while providing insights into their thought processes. This research introduces a multi-modal multi-agent system designed specifically for financial trading tasks. Our framework employs a team of specialized LLM-based agents, each adept at processing and interpreting various forms of financial data, such as textual news reports, candlestick charts, and trading signal charts. A key feature of our approach is the integration of a reflection module, which conducts analyses of historical trading signals and their outcomes. This reflective process is instrumental in enhancing the decision-making capabilities of the system for future trading scenarios. Furthermore, the ablation studies indicate that the visual reflection module plays a crucial role in enhancing the decision-making capabilities of our framework.

Complexity vs Empirical Score

  • Math Complexity: 1.5/10
  • Empirical Rigor: 5.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper uses straightforward LLM prompting and agent workflows without advanced mathematical derivations, but it includes a specific backtest over seven months on three stocks, ablation studies, and performance metrics compared to baselines.

Research Flowchart

  flowchart TD
  A["Research Goal: Explainable Multi-Modal Stock Market Prediction"] --> B["Data Input: Text News, Candlestick & Trading Signal Charts"]
  B --> C["Framework: FinVision Multi-Agent System"]
  C --> D["Core Process: Multi-Modal LLM Agents + Reflection Module"]
  D --> E["Computational Process: Historical Signal Analysis & Visual Reflection"]
  E --> F["Key Finding: Visual Reflection significantly enhances decision-making"]
  F --> G["Outcome: Highly Explainable & Adaptive Algorithmic Trading"]

Cited References (152)

  1. Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance (2024)
  2. Automatic chain of thought prompting in large language models (2022) — link
  3. Agent-pro: Learning to evolve via policy-level reflection and optimization (2024) — link
  4. Agent-pro: Learning to evolve via policy-level reflection and optimization (2024) — link
  5. Visual Reasoning and Multi-Agent Approach in Multimodal Large Language Models (MLLMs): Solving TSP and mTSP Combinatorial Challenges (2024) — link
  6. AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence (2024) — link
  7. Chatdev: Communicative agents for software development (2024)
  8. Can large language models provide useful feedback on research papers? A large-scale empirical analysis (2024)
and 144 more
  1. Multi-Agent Software Development through Cross-Team Collaboration (2024) — link
  2. Narrative asset pricing: Interpretable systematic risk factors from news text (2023)
  3. Explainable dimensionality reduction (XDR) to unbox AI ‘black box’models: A study of AI perspectives on the ethnic styles of village dwellings (2023)
  4. An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth (2023)
  5. Proximal policy optimization algorithms (2017) — link
  6. Playing atari with deep reinforcement learning (2013) — link
  7. FinAgent: A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist — → 2402.18485
  8. Finqa: A dataset of numerical reasoning over financial data (2021) — link
  9. Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering (2022) — link
  10. TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance (2021) — link
  11. Soargraph: Numerical reasoning over financial table-text data via semantic-oriented hierarchical graphs (2023)
  12. Answering numerical reasoning questions in table-text hybrid contents with graph-based encoder and tree-based decoder (2022) — link
  13. A survey of large language models (2023) — link
  14. A survey on evaluation of large language models (2024)
  15. Are large language models really good logical reasoners? a comprehensive evaluation from deductive, inductive and abductive views (2023) — link
  16. Solving math word problems via cooperative reasoning induced language models (2022) — link
  17. Critic: Large language models can self-correct with tool-interactive critiquing (2023) — link
  18. " Going on a vacation" takes longer than" Going for a walk": A Study of Temporal Commonsense Understanding (2019) — link
  19. Truthfulqa: Measuring how models mimic human falsehoods (2021) — link
  20. Auto-GPT: an autonomous GPT-4 experiment (2023) (2023)
  21. React: Synergizing reasoning and acting in language models (2022) — link
  22. Reasoning with language model is planning with world model (2023) — link
  23. Toolllm: Facilitating large language models to master 16000+ real-world apis (2023) — link
  24. Improving factuality and reasoning in language models through multiagent debate (2023) — link
  25. Self-refine: Iterative refinement with self-feedback (2024)
  26. Reflexion: Language agents with verbal reinforcement learning (2024)
  27. Unirpg: Unified discrete reasoning over table and text as program generation (2022) — link
  28. Llama: Open and efficient foundation language models (2023) — link
  29. Sparks of artificial general intelligence: Early experiments with gpt-4 (2023) — link
  30. ChatGPT (2022) — link
  31. GPT-4 Technical Report (2023)
  32. Tat-llm: A specialized language model for discrete reasoning over tabular and textual data (2024) — link
  33. Society of mind (1988)
  34. Training verifiers to solve math word problems (2021) — link
  35. Reconcile: Round-table conference improves reasoning via consensus among diverse llms (2023) — link
  36. Autogen: Enabling next-gen llm applications via multi-agent conversation framework (2023) — link
  37. Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning (2024) — link
  38. Technical analysis and sentiment embeddings for market trend prediction (2019)
  39. Stock market trend prediction using high-order information of time series (2019)
  40. Diffusion variational autoencoder for tackling stochasticity in multi-step regression stock price prediction — → 2309.00073
  41. Decision fusion for stock market prediction: a systematic review (2022)
  42. Earnhft: Efficient hierarchical reinforcement learning for high frequency trading (2024)
  43. Select and trade: Towards unified pair trading with hierarchical reinforcement learning (2023)
  44. A survey on deep reinforcement learning architectures, applications and emerging trends (2022)
  45. Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key? (2024) — link
  46. Linking microblogging sentiments to stock price movement: An application of GPT-4 — → 2308.16771
  47. Temporal Data Meets LLM–Explainable Financial Time Series Forecasting — → 2306.11025
  48. Fingpt: Open-source financial large language models — → 2306.06031
  49. FinMem: A performance-enhanced LLM trading agent with layered memory and character design (2024)
  50. Enhancing financial sentiment analysis via retrieval augmented large language models — → 2310.04027
  51. The wall street neophyte: A zero-shot analysis of chatgpt over multimodal stock movement prediction challenges — → 2304.05351
  52. AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence (2024) — link
  53. The patent holder’s dilemma: Buy, sell, or troll? (2007) — link
  54. Deciding equivalances among conjunctive aggregate queries (2007) — link
  55. Special issue: Digital Libraries (1996)
  56. Understanding Policy-Based Networking (2001)
  57. First-Order Dynamic Logic (1979) — link
  58. The title of book one (2007) — link
  59. The title of book two (2008) — link
  60. Achieving application requirements (1990) — link
  61. Statecarts in use: structured analysis and object-orientation (1998) — link
  62. The Art of Computer Programming, Vol. 1: Fundamental Algorithms (3rd. ed.) (1997)
  63. The Art of Computer Programming (1998)
  64. An experiment in bibliographic mark-up: Parsing metadata for XML export (2010) — link
  65. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2007)
  66. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2008)
  67. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2009)
  68. Predicate Path expressions (1979) — link
  69. LOGICS of Programs: AXIOMATICS and DESCRIPTIVE POWER (1978)
  70. Optimal Motion Control of a Ground Vehicle (2003)
  71. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  72. Introduction to Bayesian Statistics (2001) — link
  73. CLIFFORD: a Maple 11 Package for Clifford Algebra Computations, version 11 (2007) — link
  74. Stats and Analysis (2006) — link
  75. A more perfect union (2008) — link
  76. The fountain of youth (2009)
  77. Solder man (2003) — link
  78. Interview with Bill Kinder: January 13, 2005 (2005) — link
  79. The Enabling of Digital Libraries (2008)
  80. (old) Finding minimum congestion spanning trees (2000) — link
  81. (new) Finding minimum congestion spanning trees (2000) — link
  82. (old) Distributed data source verification in wireless sensor networks (2009) — link
  83. (new) Distributed data source verification in wireless sensor networks (2009) — link
  84. Portalis: using competitive online interactions to support aid initiatives for the homeless (2008) — link
  85. Visual Basic 6: Design, Specification, and Objects with Other (1999)
  86. The Latex Web Companion: Integrating TEX, HTML, and XML (1999)
  87. Vertex Types in Book-Embeddings (1987)
  88. Vertex Types in Book-Embeddings (1987)
  89. CHI ‘08: CHI ‘08 extended abstracts on Human factors in computing systems (2008)
  90. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  91. IEEE TCSC Executive Committee (2004) — link
  92. Distributed systems (2nd Ed.) (1993)
  93. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  94. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  95. Seminumerical Algorithms (1981)
  96. E-commerce and cultural values (2001) — link
  97. Chapter 9 (2002) — link
  98. The implementation of electronic commerce in SMEs in Singapore (Incoll) (2003) — link
  99. E-commerce and cultural values - (InBook-num-in-chap) (2004) — link
  100. E-commerce and cultural values (Inbook-text-in-chap) (2005) — link
  101. E-commerce and cultural values (Inbook-num chap) (2006) — link
  102. A library-based synthesis methodology for reversible logic (2010)
  103. Synthesis of Reversible Circuit Using Cycle-Based Approach (2010)
  104. Algorithmic Enumeration of Ideal Classes for Quaternion Orders (2010) — link
  105. Chapter II: Notes on data structuring (1972) — link
  106. Transcript of question and answer session (1981) — link
  107. Go to statement considered harmful (1979) — link
  108. Three-dimensional virtual acoustic displays (1992) — link
  109. Managerial expert systems and organizational change: some critical research issues (1987) — link
  110. Simplified Structured COBOL with Microsoft/MicroFocus COBOL (1990)
  111. A multi-radio unification protocol for IEEE 802.11 wireless networks (2004)
  112. Wireless Sensor Networks: A Survey (2002)
  113. A Survey on Wireless Multimedia Sensor Networks (2007)
  114. SSCH : Slotted Seeded Channel Hopping for Capacity Improvement in IEEE 802.11 Ad-Hoc Wireless Networks (2004)
  115. XBOW Sensor Motes Specifications (2008)
  116. Overview of Sensor Networks (2004)
  117. CodeBlue : Sensor Networks for Medical Care (2008)
  118. Investigating Network Architectures for Body Sensor Networks (2007) — link
  119. Channel-Hopping Multiple Access (2000)
  120. Body Sensor Networks (2008)
  121. Mapping Powerlists onto Hypercubes (1994)
  122. Automatic Parallelization for Distributed-Memory Multiprocessing Systems (1989)
  123. User recovery and reversal in interactive systems (1984)
  124. Generalizing specifications for uniformly implemented loops (1985)
  125. Towards monolingual programming environments (1985)
  126. The \TeX book (1984)
  127. Distributed algorithms for finding centers and medians in networks (1984)
  128. \it \LaTeX : A Document Preparation System (1986)
  129. Program transformations in a denotational setting (1985)
  130. Seminumerical Algorithms (1981)
  131. A high-level approach to computer document formatting (1980)
  132. A multifrequency MAC specially designed for wireless sensor network applications (2010) — link
  133. Institutional members of the \TeX Users Group (2017) — link
  134. acmart— C lass for typesetting publications of ACM (2017) — link
  135. Reasoning About Naming Systems (1993) — link
  136. Babel, a Multilingual Style-Option System for Use with LaTeX’s Standard Document Styles (1991)
  137. A Methodology for Implementing Highly Concurrent Data Objects (1993) — link
  138. Publication quality tables in \LaTeX (2005)
  139. Using the amsthm Package (2015)
  140. R: A Language and Environment for Statistical Computing (2019) — link
  141. UMass Citation Field Extraction Dataset (2013) — link
  142. Citation concept analysis (CCA) —A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S.~Kuhn and Karl R.~Popper (2019) — link
  143. Learning Soft Linear Constraints with Application to Citation Field Extraction (2014) — link
  144. Maintaining Discrete Probability Distributions Optimally (1993)

7 of 152 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.