Paper: arXiv 2511.18578

Authors: Eghbal Rahimikia, Hao Ni, Weiguan Wang

Abstract

Financial time series forecasting is central to trading, portfolio optimization, and risk management, yet it remains challenging due to noisy, non-stationary, and heterogeneous data. Recent advances in time series foundation models (TSFMs), inspired by large language models, offer a new paradigm for learning generalizable temporal representations from large and diverse datasets. This paper presents the first comprehensive empirical study of TSFMs in global financial markets. Using a large-scale dataset of daily excess returns across diverse markets, we evaluate zero-shot inference, fine-tuning, and pre-training from scratch against strong benchmark models. We find that off-the-shelf pre-trained TSFMs perform poorly in zero-shot and fine-tuning settings, whereas models pre-trained from scratch on financial data achieve substantial forecasting and economic improvements, underscoring the value of domain-specific adaptation. Increasing the dataset size, incorporating synthetic data augmentation, and applying hyperparameter tuning further enhance performance.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced machine learning architectures (TSFMs) and complex evaluation methodologies, but its primary strength is the large-scale empirical study using real financial data (2 billion observations) with robust backtesting protocols, economic metrics, and benchmarks.

Research Flowchart

  flowchart TD
  A["Research Goal: Evaluate TSFMs in Global Financial Markets"] --> B["Methodology: Zero-Shot, Fine-Tuning, Pre-train from Scratch"]
  B --> C["Data: Daily Excess Returns<br>Global Equities Markets"]
  C --> D["Computational Process: Benchmarking vs. TSFMs"]
  D --> E["Key Findings: Domain-Specific Pre-training<br>Significantly Outperforms Off-the-Shelf TSFMs"]
  E --> F["Outcomes: Improved Forecasting & Economics via<br>Data Scaling, Synthetic Augmentation, Hyperparameter Tuning"]

Cited References (223)

  1. A Semiparametric Intraday GARCH Model (2016) — link
  2. DeepSeek-V3 Technical Report (2024) — link
  3. Training Language Models to Follow Instructions with Human Feedback (2022)
  4. Adam: A Method for Stochastic Optimization (2015) — link
  5. Quant GANs: deep generation of financial time series (2020) — link
  6. Fin-GAN: Forecasting and Classifying Financial Time Series via Generative Adversarial Networks (2024) — link
  7. Sig-Wasserstein GANs for Conditional Time Series Generation (2024)
  8. Generative Learning for Financial Time Series with Irregular and Scale-Invariant Patterns (2024)
and 215 more
  1. Japanese and Korean voice search (2012)
  2. Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018)
  3. Big data analytics (2011)
  4. Forecasting volatility in financial markets: A review (2003)
  5. Answering the skeptics: Yes, standard volatility models do provide accurate forecasts (1998)
  6. Non-Gaussian Ornstein–Uhlenbeck-based models and some of their uses in financial economics (2001)
  7. A simple approximate long-memory model of realized volatility (2009)
  8. HAR volatility modelling with heterogeneous leverage and jumps (2009)
  9. Generalized Method of Integrated Moments for High-Frequency Data (2016)
  10. Exploiting the errors: A simple approach for improved volatility forecasting (2016)
  11. Do we need the constant term in the heterogenous autoregressive model for forecasting realized volatilities? (2018)
  12. Forecasting Realized Volatility: A Review (2018) — link
  13. Forecasting realized volatility: A review (2018)
  14. Order book slope and price volatility (2008)
  15. Good volatility, bad volatility: Signed jumps and the persistence of volatility (2015)
  16. Dynamics of the limit order book and the volume-volatility relation (2018)
  17. Sequence classification of the limit order book using recurrent neural networks (2018)
  18. Modelling high-frequency limit order book dynamics with support vector machines (2015)
  19. Benchmark dataset for mid-price forecasting of limit order book data with machine learning methods (2018)
  20. Convolutional neural networks applied to high-frequency market microstructure forecasting (2017)
  21. Universal features of price formation in financial markets: perspectives from deep learning (2018)
  22. Deep learning for limit order books (2019)
  23. Prediction of Hidden Liquidity in theLimit Order Book of GLOBEX Futures (2013)
  24. Forecasting of jump arrivals in stock prices: New attention-based network architecture using limit order book data (2018) — link
  25. Multivariate realised kernels: consistent positive semi-definite estimators of the covariation of equity prices with noise and non-synchronous trading (2011)
  26. A tale of two time scales: Determining integrated volatility with noisy high-frequency data (2005)
  27. Volatility forecast comparison using imperfect volatility proxies (2011)
  28. Forecasting the volatility of crude oil futures using HAR-type models with structural breaks (2016)
  29. Forecasting volatility of the US oil market (2014)
  30. Volatility transmission in emerging European foreign exchange markets (2011)
  31. Quantile forecasts of daily exchange rate returns from forecasts of realized volatility (2008)
  32. Relative liquidity and future volatility (2015)
  33. Limit orders, depth, and volatility: Evidence from the stock exchange of Hong Kong (2001)
  34. Does the open limit order book matter in explaining informational volatility? (2009)
  35. LOBSTER: Limit order book reconstruction system (2011)
  36. Realized kernels in practice: Trades and quotes (2009)
  37. Realized variance and market microstructure noise (2006)
  38. Order book characteristics and the volume–volatility relation: Empirical evidence from a limit order market (2006)
  39. Measuring stock illiquidity: An investigation of the demand and supply schedules at the TASE (2004)
  40. Time-series classification using neural bag-of-features (2017)
  41. Order aggressiveness in limit order book markets (2004)
  42. Canceling liquidity (2015)
  43. A simple nonparametric test of predictive performance (1992)
  44. Estimating the persistence and the autocorrelation function of a time series that is measured with error (2014)
  45. Unstructured data and the 80 percent rule (2008)
  46. News implied volatility and disaster concerns (2017)
  47. When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks (2011)
  48. Textual Analysis in Accounting and Finance: A Survey (2016)
  49. Predicting Returns With Text Data (2019)
  50. Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007)
  51. Can stock market forecasters forecast? (1933)
  52. The impact of sentiment and attention measures on stock market volatility (2020)
  53. Swisscheese at semeval-2016 task 4: Sentiment classification using an ensemble of convolutional neural networks with distant supervision (2016)
  54. Leveraging large amounts of weakly supervised data for multi-language sentiment classification (2017)
  55. Predicting financial markets: Comparing survey, news, twitter and search engine data (2011) — link
  56. Distillation of news flow into analysis of stock reactions (2016)
  57. The Twitter myth revisited: Intraday investor sentiment, Twitter activity and individual-level stock return volatility (2018)
  58. Intraday online investor sentiment and return patterns in the US stock market (2017)
  59. Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards (2004)
  60. Stock return predictability and investor sentiment: A high-frequency perspective (2016)
  61. Power and bipower variation with stochastic volatility and jumps (2004)
  62. A reality check for data snooping (2000)
  63. MFE MATLAB function reference financial econometrics (2009)
  64. The stationary bootstrap (1994)
  65. Comparing predictive accuracy (2002)
  66. Yahoo! for Amazon: Sentiment extraction from small talk on the web (2007)
  67. Measuring and testing the impact of news on volatility (1993)
  68. Long short-term memory (1997)
  69. Learning to forget: Continual prediction with LSTM (1999)
  70. Deep learning in asset pricing (2019)
  71. Deep learning in asset pricing (2020)
  72. Deep learning (2016)
  73. Improving neural networks by preventing co-adaptation of feature detectors (2012) — link
  74. Dropout: a simple way to prevent neural networks from overfitting (2014)
  75. Adam: A Method for Stochastic Optimization (2014) — link
  76. Asymptotic inference about predictive ability (1996)
  77. Evaluating volatility and correlation forecasts (2009)
  78. Alternative Data for Realised Volatility Forecasting: Limit Order Book and News Stories (2020)
  79. Efficient Estimation of Word Representations in Vector Space (2013) — link
  80. Advances in Pre-Training Distributed Word Representations (2018)
  81. Enriching word vectors with subword information (2017)
  82. Simlex-999: Evaluating semantic models with (genuine) similarity estimation (2015)
  83. A study on similarity and relatedness using distributional and wordnet-based approaches (2009)
  84. Convolutional Neural Networks for Sentence Classification (2014) — link
  85. Measuring and hedging geopolitical risk (2020)
  86. The common and specific components of dynamic volatility (2006)
  87. Text as Data (2019)
  88. Empirical Asset Pricing via Machine Learning (2020)
  89. Autoencoder asset pricing models (2021)
  90. (Re-) Imag (in) ing Price Trends (2020)
  91. Machine learning for realised volatility forecasting (2020)
  92. The structure of economic news (2020)
  93. Distributed representations of words and phrases and their compositionality (2013) — link
  94. Exploiting similarities among languages for machine translation (2013) — link
  95. Hierarchical probabilistic neural network language model. (2005)
  96. FastText.zip: Compressing text classification models (2016) — link
  97. Normalized (pointwise) mutual information in collocation extraction (2009)
  98. Axiomatic attribution for deep networks (2017)
  99. Natural language processing (almost) from scratch (2011)
  100. A Unified Approach to Interpreting Model Predictions (2017) — link
  101. Learning important features through propagating activation differences (2017)
  102. Modelling Volatility Cycles: the (MF)^ 2 GARCH Model (2021)
  103. Predictably unequal? the effects of machine learning on credit markets (2021)
  104. Enhancing time-series momentum strategies using deep neural networks (2019)
  105. Deep reinforcement learning for trading (2020)
  106. A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news (2021)
  107. Machine learning and portfolio optimization (2018)
  108. Forecasting the equity premium: mind the news! (2020)
  109. Bond risk premiums with machine learning (2021)
  110. Corrigendum: Bond risk premiums with machine learning (2021)
  111. The role of corporate culture in bad times: Evidence from the COVID-19 pandemic (2020)
  112. Venture capital communities (2020)
  113. A cross-sectional machine learning approach for hedge fund return prediction and selection (2020)
  114. Measuring news sentiment (2020)
  115. Deeplob: Deep convolutional neural networks for limit order books (2019)
  116. Universal features of price formation in financial markets: perspectives from deep learning (2019)
  117. DeepLOB: Deep Convolutional Neural Networks for Limit Order Books (2019)
  118. Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units (2021) — link
  119. Building Cross-Sectional Systematic Strategies by Learning to Rank (2021)
  120. Slow Momentum with Fast Reversion: A Trading Strategy Using Deep Learning and Changepoint Detection (2021) — link
  121. Shap values for explaining cnn-based text classification models (2020) — link
  122. Reducing duplicate filters in deep neural networks (2017)
  123. A machine learning approach to volatility forecasting (2021)
  124. Textual factors: A scalable, interpretable, and data-driven approach to analyzing unstructured information (2019)
  125. A primer on neural network models for natural language processing (2016)
  126. Modeling price volatility based on a genetic programming approach (2019)
  127. Measuring firms’ market orientation using textual analysis of 10-K filings (2020)
  128. Identifying linguistic markers of CEO hubris: a machine learning approach (2022)
  129. Forecasting realized volatility with machine learning: Panel data perspective (2023)
  130. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018) — link
  131. RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019) — link
  132. Language Models are Unsupervised Multitask Learners (2019)
  133. Language Models are Few-Shot Learners (2020) — link
  134. LLaMA: Open and Efficient Foundation Language Models (2023) — link
  135. Llama 2: Open Foundation and Fine-Tuned Chat Models (2023) — link
  136. The Llama 3 Herd of Models (2024) — link
  137. Realised Volatility Forecasting: Machine Learning via Financial Word Embedding (2021) — link
  138. FinBERT: A Large Language Model for Extracting Information from Financial Text (2023)
  139. Expected Returns and Large Language Models (2022)
  140. How Much Can Machines Learn Finance from Chinese Text Data? (2024)
  141. Lookahead Bias in Pretrained Language Models (2024)
  142. Financial Statement Analysis with Large Language Models — → 2407.17866
  143. Bloated Disclosures: Can ChatGPT Help Investors Process Information? (2024)
  144. Time Machine GPT (2024)
  145. Deep Contextualized Word Representations (2018) — link
  146. Well-Read Students Learn Better: On the Importance of Pre-training Compact Models (2019) — link
  147. ZeRO: Memory optimizations Toward Training Trillion Parameter Models (2020)
  148. Decoupled Weight Decay Regularization (2017) — link
  149. On Persistence in Mutual Fund Performance (1997)
  150. Trading Costs of Asset Pricing Anomalies (2012)
  151. Estimating Number of Factors by Adjusted Eigenvalues Thresholding (2022)
  152. Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models — → 2304.07619
  153. ChatGPT, Stock Market Predictability and Links to the Macroeconomy (2023)
  154. Is There a Replication Crisis in Finance? (2023)
  155. A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix (1986)
  156. Chronos: Learning the Language of Time Series (2024) — link
  157. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2020)
  158. A Decoder-Only Foundation Model for Time-Series Forecasting (2024)
  159. MOMENT: A Family of Open Time-series Foundation Models (2024) — link
  160. Kairos: Towards Adaptive and Generalizable Time Series Foundation Models (2025) — link
  161. Attention Is All You Need (2017)
  162. Foundation Models for Time Series Analysis : A Tutorial and Survey (2024) — link
  163. Unified Training of Universal Time Series Forecasting Transformers (2024) — link
  164. CatBoost: Unbiased Boosting with Categorical Features (2018)
  165. XGBoost: A Scalable Tree Boosting System (2016) — link
  166. LightGBM: A Highly Efficient Gradient Boosting Decision Tree (2017) — link
  167. TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning (2025) — link
  168. xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference (2025) — link
  169. Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting (2023) — link
  170. Simplified State Space Layers for Sequence Modeling (2022) — link
  171. FlowState: Sampling Rate Invariant Time Series Forecasting (2025) — link
  172. Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series (2024)
  173. Sundial: A Family of Highly Capable Time Series Foundation Models (2025) — link
  174. Toto: Time Series Optimized Transformer for Observability (2024) — link
  175. Empirical Asset Pricing: The Cross Section of Stock Returns (2016)
  176. Returns to Buying Winners and Selling Losers : Implications for Stock Market Efficiency (1993) — link
  177. Evidence of Predictable Behavior of Security Returns (1990) — link
  178. Foundations of Technical Analysis : Computational Algorithms , Statistical Inference , and Empirical Implementation (2000) — link
  179. (Re-) Imag (in) ing price trends (2023)
  180. Charting by Machines (2024) — link
  181. Machine Learning from a ``Universe’’ of Signals: The Role of Feature Engineering (2025) — link
  182. Market Efficiency in the Age of Big Data (2022)
  183. Forecasting the Equity Risk Premium : The Role of Technical Indicators (2014) — link
  184. Simple Technical Trading Rules and the Stochastic Properties of Stock Returns (1992)
  185. Machine Learning in the Chinese Stock Market (2022) — link
  186. Re(Visiting) Large Language Models in Finance (2024)
  187. Chronologically Consistent Large Language Models — → 2502.21206
  188. Deep Learning in Asset Pricing (2024)
  189. Artificial Intelligence Asset Pricing Models (2025)
  190. Isambard-AI: A Leadership-Class Supercomputer Optimised Specifically for Artificial Intelligence (2024)
  191. Time Series Momentum (2012)
  192. Value and Momentum Everywhere (2013)
  193. Risks and Returns of Cryptocurrency (2021)
  194. Momentum Has Its Moments (2015)
  195. Currency Momentum Strategies (2012)
  196. Factor Momentum and the Momentum Factor (2022)
  197. Factor Momentum Everywhere (2018)
  198. Machine Learning from a ``Universe’’ of Signals: The Role of Feature Engineering (2025)
  199. The Virtue of Complexity in Return Prediction (2024)
  200. Quant GANs: Deep Generation of Financial Time Series (2020)
  201. Ridge Regression: Biased Estimation for Nonorthogonal Problems (1970)
  202. Regression Shrinkage and Selection via the Lasso (1996)
  203. Regularization and Variable Selection via the Elastic Net (2005)
  204. Principal Components Regression in Exploratory Statistical Research (1965)
  205. Robust Statistics (2011)
  206. Simplified State Space Layers for Sequence Modeling (2022) — link
  207. Understanding Transformers for Time Series: Rank Structure, Flow-of-ranks, and Compressibility (2025) — link
  208. Simplified: A Closer Look at the Virtue of Complexity in Return Prediction (2025)
  209. Seemingly Virtuous Complexity in Return Prediction (2025)
  210. Understanding The Virtue of Complexity (2025)
  211. The Limited Virtue of Complexity in a Noisy World (2025)
  212. Comment on “The Virtue of Complexity in Return Prediction” (2023)
  213. Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts (2024) — link
  214. Training Compute-Optimal Large Language Models (2022) — link
  215. Neural Machine Translation of Rare Words with Subword Units (2016)

3 of 223 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.