Paper: arXiv 2409.08281

Abstract

The stock price prediction task holds a significant role in the financial domain and has been studied for a long time. Recently, large language models (LLMs) have brought new ways to improve these predictions. While recent financial large language models (FinLLMs) have shown considerable progress in financial NLP tasks compared to smaller pre-trained language models (PLMs), challenges persist in stock price forecasting. Firstly, effectively integrating the modalities of time series data and natural language to fully leverage these capabilities remains complex. Secondly, FinLLMs focus more on analysis and interpretability, which can overlook the essential features of time series data. Moreover, due to the abundance of false and redundant information in financial markets, models often produce less accurate predictions when faced with such input data. In this paper, we introduce StockTime, a novel LLM-based architecture designed specifically for stock price data. Unlike recent FinLLMs, StockTime is specifically designed for stock price time series data. It leverages the natural ability of LLMs to predict the next token by treating stock prices as consecutive tokens, extracting textual information such as stock correlations, statistical trends and timestamps directly from these stock prices. StockTime then integrates both textual and time series data into the embedding space. By fusing this multimodal data, StockTime effectively predicts stock prices across arbitrary look-back periods. Our experiments demonstrate that StockTime outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper employs advanced deep learning architectures (LLM adaptation, autoregressive encoders, multimodal fusion) with significant mathematical modeling, but the empirical rigor is limited to experimental claims without providing code, datasets, or detailed statistical backtesting metrics.

Research Flowchart

  flowchart TD
  A["Research Goal: Design specialized LLM for stock price prediction"] --> B["Core Methodology: Treat stock prices as consecutive tokens"]
  B --> C["Data Inputs: Time series data & textual correlations"]
  C --> D["Multimodal Integration: Fuse data into embedding space"]
  D --> E["Computational Process: LLM next token prediction"]
  E --> F["Key Findings: Improved accuracy, reduced memory/runtime"]

Cited References (52)

  1. Blackboard Systems (1986)
  2. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education (1983)
  3. Classification Problem Solving (1984)
  4. New Ways to Make Microcircuits Smaller (1980) — link
  5. New Ways to Make Microcircuits Smaller—Duplicate Entry (1980)
  6. Strategic explanations for a diagnostic consultation system (1984) — link
  7. Strategic Explanations in Consultation—Duplicate (1983)
  8. Poligon: A System for Parallel Problem Solving (1986)
and 44 more
  1. Transfer of Rule-Based Expertise through a Tutorial Dialogue (1979)
  2. The Engineering of Qualitative Models (2021)
  3. Attention Is All You Need (2017) — link
  4. Pluto: The ‘Other’ Red Planet (2015)
  5. Bloomberggpt: A large language model for finance (2023) — link
  6. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
  7. When flue meets flang: Benchmarks and large pre-trained language model for financial domain (2022) — link
  8. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  9. Electra: Pre-training text encoders as discriminators rather than generators (2020) — link
  10. Fingpt: Open-source financial large language models — → 2306.06031
  11. Pixiu: A large language model, instruction data and evaluation benchmark for finance (2023) — link
  12. Ploutos: Towards interpretable stock movement prediction with financial large language model — → 2403.00782
  13. A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges — → 2406.11903
  14. Large language models: A survey (2024) — link
  15. Autotimes: Autoregressive time series forecasters via large language models (2024) — link
  16. Time-LLM : Time series forecasting by reprogramming large language models (2024)
  17. Pytorch: An imperative style, high-performance deep learning library (2019)
  18. Adam: A method for stochastic optimization (2015)
  19. Stock price pattern recognition-a recurrent neural network approach (1990)
  20. A survey of large language models in finance (finllms) (2024) — link
  21. Stock Movement and Volatility Prediction from Tweets, Macroeconomic Factors and Historical Prices (2023)
  22. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2023)
  23. Accurate stock movement prediction with self-supervised learning from sparse noisy tweets (2022)
  24. Stock movement prediction from tweets and historical prices (2018)
  25. Hybrid deep sequential modeling for social text-driven stock prediction (2018)
  26. itransformer: Inverted transformers are effective for time series forecasting (2024)
  27. Alpha-gpt: Human-ai interactive alpha mining for quantitative investment — → 2308.00016
  28. Enhancing Stock Movement Prediction with Adversarial Training. (2019)
  29. Add: augmented disentanglement distillation framework for improving stock trend forecasting (2020) — link
  30. MASTER: Market-Guided Stock Transformer for Stock Price Forecasting (2024)
  31. Deep learning-based stock price prediction using LSTM and bi-directional LSTM model (2020)
  32. Combined deep learning classifiers for stock market prediction: integrating stock price and news sentiments (2023)
  33. ALERTA-Net: A Temporal Distance-Aware Recurrent Networks for Stock Movement and Volatility Prediction (2023)
  34. Shortlisting machine learning-based stock trading recommendations using candlestick pattern recognition (2023)
  35. AMA-LSTM: Pioneering Robust and Fair Financial Audio Analysis for Stock Volatility Prediction — → 2407.18324
  36. Large language models are zero-shot time series forecasters (2024)
  37. Lag-llama: Towards foundation models for time series forecasting (2023) — link
  38. Promptcast: A new prompt-based learning paradigm for time series forecasting (2023)
  39. The wall street neophyte: A zero-shot analysis of chatgpt over multimodal stock movement prediction challenges — → 2304.05351
  40. One fits all: Power general time series analysis by pretrained lm (2023)
  41. Mistral 7B (2023) — link
  42. The Llama 3 Herd of Models (2024) — link
  43. StockMixer: A Simple Yet Strong MLP-Based Architecture for Stock Price Forecasting (2024)
  44. Large language models for time series: A survey (2024) — link

6 of 52 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.