Paper: arXiv 2311.13743

Abstract

Recent advancements in Large Language Models (LLMs) have exhibited notable efficacy in question-answering (QA) tasks across diverse domains. Their prowess in integrating extensive web knowledge has fueled interest in developing LLM-based autonomous agents. While LLMs are efficient in decoding human instructions and deriving solutions by holistically processing historical inputs, transitioning to purpose-driven agents requires a supplementary rational architecture to process multi-source information, establish reasoning chains, and prioritize critical tasks. Addressing this, we introduce \textsc{“FinMem”}, a novel LLM-based agent framework devised for financial decision-making. It encompasses three core modules: Profiling, to customize the agent’s characteristics; Memory, with layered message processing, to aid the agent in assimilating hierarchical financial data; and Decision-making, to convert insights gained from memories into investment decisions. Notably, \textsc{“FinMem”}’s memory module aligns closely with the cognitive structure of human traders, offering robust interpretability and real-time tuning. Its adjustable cognitive span allows for the retention of critical information beyond human perceptual limits, thereby enhancing trading outcomes. This framework enables the agent to self-evolve its professional knowledge, react agilely to new investment cues, and continuously refine trading decisions in the volatile financial environment. We first compare \textsc{“FinMem”} with various algorithmic agents on a scalable real-world financial dataset, underscoring its leading trading performance in stocks. We then fine-tuned the agent’s perceptual span and character setting to achieve a significantly enhanced trading performance. Collectively, \textsc{“FinMem”} presents a cutting-edge LLM agent framework for automated trading, boosting cumulative investment returns.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel LLM agent framework with layered memory and character design, involving complex cognitive modeling and adaptive mechanisms that align with financial decision-making. It demonstrates empirical rigor through backtesting on real-world financial datasets, comparing performance against other algorithmic agents and reporting enhanced trading results.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Develop LLM Agent for<br>Financial Decision-Making"] --> B["Data Input:<br>Scalable Real-World<br>Financial Dataset"]
  B --> C["Methodology: FinMem Framework<br>Profiling | Memory | Decision-Making"]
  C --> D{"Computational Process:<br>Layered Memory Processing<br>Human-like Cognitive Structure"}
  D --> E["Performance Evaluation<br>vs. Algorithmic Agents"]
  E --> F["Key Findings:<br>Leading Trading Performance<br>Self-Evolving Knowledge<br>Enhanced Cumulative Returns"]

Cited References (89)

  1. Financial trading as a game: A deep reinforcement learning approach (2018) — link
  2. Automatic financial trading agent for low-risk portfolio management using deep reinforcement learning (2019) — link
  3. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  4. An Overview of Machine Learning, Deep Learning, and Reinforcement Learning-Based Techniques in Quantitative Finance: Recent Progress and Challenges (2023)
  5. CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools (2023) — link
  6. Improving Factuality and Reasoning in Language Models through Multiagent Debate (2023) — link
  7. Human cognitive architecture: Why some instructional procedures work and others do not. (2012)
  8. Replication and analysis of Ebbinghaus’ forgetting curve (2015)
and 81 more
  1. Generative agents: Interactive simulacra of human behavior (2023) — link
  2. Generative Agents: Interactive Simulacra of Human Behavior (2023) — link
  3. Code Llama: Open Foundation Models for Code (2023) — link
  4. Improving language understanding by generative pre-training (2018)
  5. GPT-4 Technical Report (2023) — link
  6. FinGPT: Open-Source Financial Large Language Models — → 2306.06031
  7. Bloomberggpt: A large language model for finance (2023) — link
  8. Sentiment-influenced trading system based on multimodal deep reinforcement learning (2021)
  9. A Deep Multimodal Reinforcement Learning System Combined with CNN and LSTM for Stock Trading (2019) — link
  10. Design and implementation of a multiagent stock trading system (2012)
  11. MSPM: A modularized and scalable multi-agent reinforcement learning-based system for financial portfolio management (2022)
  12. A multi-agent Q-learning framework for optimizing stock trading systems (2002)
  13. A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets (2022)
  14. word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method (2014) — link
  15. Glove: Global vectors for word representation (2014)
  16. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  17. Long short-term memory (1997)
  18. Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition (2014) — link
  19. Interactive narrative: A novel application of artificial intelligence for computer games (2012)
  20. Weaviate — link
  21. Cohere: Privacy Management in Large Scale Systems (2023) — link
  22. Human memory: A proposed system and its control processes (1968)
  23. What are the differences between long-term, short-term, and working memory? (2008)
  24. Llama: Open and efficient foundation language models (2023) — link
  25. Billion-scale similarity search with GPUs (2019)
  26. MedGPT: Medical concept prediction from clinical narratives (2021) — link
  27. Efficient market hypothesis (EMH): past, present and future (2008)
  28. Capital asset prices: A theory of market equilibrium under conditions of risk (1964)
  29. Proximal policy optimization algorithms (2017) — link
  30. Playing atari with deep reinforcement learning (2013) — link
  31. Asynchronous methods for deep reinforcement learning (2016)
  32. Desiderata for cognitive architectures (2004)
  33. Working memory (2020)
  34. A survey on large language model-based autonomous agents (2023) — link
  35. Cognitive architectures for language agents (2023) — link
  36. Deep reinforcement learning for financial trading using multi-modal features (2023)
  37. The neurobiology of deep reinforcement learning (2020)
  38. Deep reinforcement learning for optimizing finance portfolio management (2019)
  39. Application of deep reinforcement learning in stock trading strategies and stock forecasting (2020)
  40. Levels of processing: A framework for memory research (1972)
  41. A survey on deep reinforcement learning architectures, applications and emerging trends (2022)
  42. A sparsity algorithm for finding optimal counterfactual explanations: Application to corporate credit rating (2023)
  43. Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality (1993)
  44. Deep reinforcement learning for trading—A critical survey (2021)
  45. Technical analysis of stock trends (2018)
  46. Reinforcement learning in financial markets-a survey (2018)
  47. Noise (1986)
  48. Automated trading systems statistical and machine learning methods and hardware implementation: a survey (2019)
  49. How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda (2022)
  50. Dynamic Datasets and Market Environments for Financial Reinforcement Learning (2023) — link
  51. Psychological study of human judgment: Implications for investment decision making (1972)
  52. Making language models better reasoners with step-aware verifier (2023)
  53. The magical number seven, plus or minus two: Some limits on our capacity for processing information. (1956)
  54. Guardrails AI
  55. FinRL : Deep reinforcement learning framework to automate trading in quantitative finance (2021)
  56. gym-anytrading (2023)
  57. Performance of moving average trading strategies over varying stock market conditions: the Finnish evidence (2014)
  58. Forecasting the Taiwan stock market with a novel momentum-based fuzzy time-series (2012)
  59. Moving Average Convergence-Divergence (MACD) Trading Rule: An Application in Nepalese Stock Market" NEPSE" (2020)
  60. Reinforcement learning in stock trading (2019)
  61. Adaptive stock trading strategies with deep reinforcement learning methods (2020)
  62. Practical deep reinforcement learning approach for stock trading (2018) — link
  63. Adaptive stock trading with dynamic asset allocation using reinforcement learning (2006)
  64. Stock trading rule discovery with double deep Q-network (2021)
  65. FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance (2020) — link
  66. Memory sandbox: Transparent and interactive memory management for conversational agents (2023) — link
  67. Codehelp: Using large language models with guardrails for scalable support in programming classes (2023) — link
  68. Social simulacra: Creating populated prototypes for social computing systems (2022)
  69. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  70. Building cooperative embodied agents modularly with large language models (2023) — link
  71. RecAgent: A Novel Simulation Paradigm for Recommender Systems (2023) — link
  72. Out of one, many: Using language models to simulate human samples (2023)
  73. How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings (2019) — link
  74. Deep reinforcement learning in quantitative algorithmic trading: A review (2021) — link
  75. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  76. Stable-Baselines3: Reliable Reinforcement Learning Implementations (2021) — link
  77. Volatility Tests and Efficient Markets: A Review Essay (1988)
  78. Risk Management and Financial Institutions (2007)
  79. The Sharpe Ratio (1994)
  80. Downside Risk (2003)
  81. A survey of large language models (2023) — link

1 of 89 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.