Paper: arXiv 2412.09880

Abstract

Large models have shown unprecedented capabilities in natural language processing, image generation, and most recently, time series forecasting. This leads us to ask the question: treating market prices as a time series, can large models be used to predict the market? In this paper, we answer this by evaluating the performance of the latest time series foundation model TimesFM on price prediction. We find that due to the irregular nature of price data, directly applying TimesFM gives unsatisfactory results and propose to fine-tune TimeFM on financial data for the task of price prediction. This is done by continual pre-training of the latest time series foundation model TimesFM on price data containing 100 million time points, spanning a range of financial instruments spanning hourly and daily granularities. The fine-tuned model demonstrates higher price prediction accuracy than the baseline model. We conduct mock trading for our model in various financial markets and show that it outperforms various benchmarks in terms of returns, sharpe ratio, max drawdown and trading cost.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper primarily relies on applying an existing foundation model with minor modifications (log transform, masking) rather than introducing dense theoretical mathematics, but demonstrates high empirical rigor through extensive dataset curation (100M points), backtesting across multiple markets, and published code/model weights for reproducibility.

Research Flowchart

  flowchart TD
  A["Research Goal: Can large time series models predict financial markets?"] --> B["Methodology: Continual Pre-training"]
  B --> C["Data: 100M time points<br>(Hourly & Daily prices)"]
  C --> D["Model: TimesFM<br>(Fine-tuned on Financial Data)"]
  D --> E["Evaluation: Mock Trading"]
  E --> F["Outcome: Outperformed Benchmarks<br>(Returns, Sharpe Ratio, Max Drawdown)"]
  D --> G["Comparison: Baseline Model<br>(Direct Application)"]
  G --> H["Result: Unsatisfactory Accuracy<br>(Irregular price data)"]
  F & H --> I["Conclusion: Financial Fine-tuning Essential for Price Prediction"]

Cited References (38)

  1. Attention Is All You Need (2017)
  2. Time series analysis: forecasting and control (1970)
  3. General exponential smoothing and the equivalent ARMA process (1984)
  4. A New Approach to Linear Filtering and Prediction Problems (1960)
  5. Generalized autoregressive conditional heteroskedasticity (1986) — link
  6. Hidden Markov Models for Stock Market Prediction (2023) — link
  7. Stock Market Trend Analysis Using Hidden Markov Model and Long Short Term Memory (2021) — link
  8. Learning internal representations by error propagation (1986) — link
and 30 more
  1. Long Short-Term Memory (1997) — link
  2. Language Models are Few-Shot Learners (2020) — link
  3. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2020)
  4. A time series is worth 64 words: Long-term forecasting with transformers (2022)
  5. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)
  6. Language Models are Few-Shot Learners (2020) — link
  7. Large Language Models Are Zero-Shot Time Series Forecasters (2023)
  8. LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language (2024) — link
  9. Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (2024) — link
  10. Large Language Models for Time Series: A Survey (2024) — link
  11. TEST: Text Prototype Aligned Embedding to Activate LLM’s Ability for Time Series (2024) — link
  12. Are Language Models Actually Useful for Time Series Forecasting? (2024) — link
  13. A decoder-only foundation model for time-series forecasting (2024) — link
  14. TimeGPT-1 (2024) — link
  15. Darts: User-Friendly Modern Machine Learning for Time Series (2022) — link
  16. Monash Time Series Forecasting Archive (2021) — link
  17. Informer: Beyond efficient transformer for long sequence time-series forecasting (2021)
  18. LLaMA: Open and Efficient Foundation Language Models (2023) — link
  19. Mixtral of Experts (2024) — link
  20. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2021)
  21. Scalable Diffusion Models with Transformers (2023) — link
  22. ViViT: A Video Vision Transformer (2021) — link
  23. On Model Evaluation under Non-constant Class Imbalance (2020) — link
  24. Macro F1 and Macro F1 (2021) — link
  25. FinGPT: Open-Source Financial Large Language Models (2023) — link
  26. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution (2022) — link
  27. Understanding intermediate layers using linear classifier probes (2018) — link
  28. How Transferable are Video Representations Based on Synthetic Data? (2022) — link
  29. Synthetic Data from Diffusion Models Improves ImageNet Classification (2023) — link
  30. LoRA: Low-Rank Adaptation of Large Language Models (2021) — link

0 of 38 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.