Paper: arXiv 2411.01368

Abstract

Predicting financial markets and stock price movements requires analyzing a company’s performance, historic price movements, industry-specific events alongside the influence of human factors such as social media and press coverage. We assume that financial reports (such as income statements, balance sheets, and cash flow statements), historical price data, and recent news articles can collectively represent aforementioned factors. We combine financial data in tabular format with textual news articles and employ pre-trained Large Language Models (LLMs) to predict market movements. Recent research in LLMs has demonstrated that they are able to perform both tabular and text classification tasks, making them our primary model to classify the multi-modal data. We utilize retrieval augmentation techniques to retrieve and attach relevant chunks of news articles to financial metrics related to a company and prompt the LLMs in zero, two, and four-shot settings. Our dataset contains news articles collected from different sources, historic stock price, and financial report data for 20 companies with the highest trading volume across different industries in the stock market. We utilized recently released language models for our LLM-based classifier, including GPT- 3 and 4, and LLaMA- 2 and 3 models. We introduce an LLM-based classifier capable of performing classification tasks using combination of tabular (structured) and textual (unstructured) data. By using this model, we predicted the movement of a given stock’s price in our dataset with a weighted F1-score of 58.5% and 59.1% and Matthews Correlation Coefficient of 0.175 for both 3-month and 6-month periods.

Complexity vs Empirical Score

  • Math Complexity: 3.0/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper applies established LLM techniques (zero/few-shot prompting, retrieval augmentation) with minimal novel mathematical formalization, but demonstrates strong empirical implementation through detailed data collection, model selection (GPT-3/4, LLaMA), and reported metrics (F1-score, MCC).

Research Flowchart

  flowchart TD
  A["Research Goal<br>Predict stock price movement<br>using multi-modal data"] --> B{"Data Collection"}
  B --> C["Financial Data<br>Tabular: Income, Balance, Cash Flow"]
  B --> D["Market Data<br>Tabular: Historical Prices"]
  B --> E["News Articles<br>Textual: Sentiment & Events"]
  C & D & E --> F["Preprocessing & RAG<br>Retrieve relevant news chunks"]
  F --> G["LLM Classifier<br>GPT-3/4, LLaMA-2/3<br>Zero/Few-shot prompting"]
  G --> H["Outcome<br>Weighted F1: 58.5% - 59.1%<br>MCC: 0.175"]

Cited References (37)

  1. The Changing Landscape of Investors in the United States: A Report of the National Financial Capability Study. FINRA Investor Education Foundation (2022)
  2. Retrieval-augmented generation for knowledge-intensive NLP tasks (2020) — link
  3. A comparative analysis of sentence embedding techniques for document ranking (2022)
  4. Sentence similarity based on contexts (2022)
  5. RAPTOR : Recursive abstractive processing for tree-organized retrieval (2024) — link
  6. Specialized document embeddings for aspect-based similarity of research papers (2022)
  7. FinGPT : Instruction Tuning benchmark for open-source large language models in financial datasets (2023) — link
  8. FinGPT : Open-source financial large language models (2023) — link
and 29 more
  1. LoRA : Low-Rank Adaptation of large language models (2021) — link
  2. R e2 G : Retrieve, Rerank, Generate (2022) — link
  3. A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture (2023) — link
  4. Large language models in finance: A survey (2023)
  5. Ploutos: Towards interpretable stock movement prediction with financial large language model (2024) — link
  6. LLMFactor : Extracting profitable factors through prompts for explainable stock movement prediction (2024) — link
  7. Stock market one-day ahead movement prediction using disparate data sources (2017)
  8. Making LLMs worth every penny: Resource-limited text classification in banking (2023)
  9. Learning mutual fund categorization using natural language processing (2022)
  10. Enhancing financial sentiment analysis via retrieval augmented large language models — → 2310.04027
  11. LLMs for financial advisement: A fairness and efficacy study in personal decision making (2023)
  12. Efficient capital markets: A review of theory and empirical work (1970)
  13. Can ChatGPT forecast stock price movements? Return predictability and large language models (2023)
  14. Forecasting stock market indices using the recurrent neural network based hybrid models: CNN-LSTM , GRU-CNN , and ensemble models (2023)
  15. Predicting stock market time-series data using CNN-LSTM Neural Network model (2023) — link
  16. Stock market one-day ahead movement prediction using disparate data sources (2017)
  17. Stock market’s price movement prediction with LSTM neural networks (2017)
  18. Prediction of stock market index movement by ten data mining techniques (2009)
  19. SelfCheckGPT : Zero-resource black-box hallucination detection for generative large language models (2023)
  20. PromptRank : Unsupervised Keyphrase Extraction Using Prompt (2023) — link
  21. Data Augmentation for Abstractive Query-Focused Multi-Document Summarization (2021)
  22. Language Models are Few-Shot Learners (2020) — link
  23. TabLLM : Few-shot classification of tabular data with large language models (2022) — link
  24. Large language models(LLMs ) on tabular data: Prediction, generation, and understanding – A survey (2024) — link
  25. Text Classification via Large Language Models (2023) — link
  26. Enhancing financial sentiment analysis via retrieval augmented large language models — → 2310.04027
  27. Making llms worth every penny: Resource-limited text classification in banking (2023)
  28. Learning Mutual Fund Categorization using Natural Language Processing (2022)
  29. P rompt R ank: Unsupervised Keyphrase Extraction Using Prompt (2023) — link

2 of 37 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.