Paper: arXiv 2407.18519

Abstract

Recently, the incorporation of both temporal features and the correlation across time series has become an effective approach in time series prediction. Spatio-Temporal Graph Neural Networks (STGNNs) demonstrate good performance on many Temporal-correlation Forecasting Problem. However, when applied to tasks lacking periodicity, such as stock data prediction, the effectiveness and robustness of STGNNs are found to be unsatisfactory. And STGNNs are limited by memory savings so that cannot handle problems with a large number of nodes. In this paper, we propose a novel approach called the Temporal-Correlation Graph Pre-trained Network (TCGPN) to address these limitations. TCGPN utilize Temporal-correlation fusion encoder to get a mixed representation and pre-training method with carefully designed temporal and correlation pre-training tasks. Entire structure is independent of the number and order of nodes, so better results can be obtained through various data enhancements. And memory consumption during training can be significantly reduced through multiple sampling. Experiments are conducted on real stock market data sets CSI300 and CSI500 that exhibit minimal periodicity. We fine-tune a simple MLP in downstream tasks and achieve state-of-the-art results, validating the capability to capture more robust temporal correlation patterns.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel Temporal-Correlation Fusion Encoder and pre-training tasks with mathematical formulations, indicating advanced ML theory. It is heavily backed by experiments on real stock datasets (CSI300, CSI500) with SOTA results, ablation studies, and implementation details like memory optimization, making it empirically rigorous.

Research Flowchart

  flowchart TD
  A["Research Goal: Improve Stock Prediction<br>for non-periodic time series data"] --> B["Data Input: CSI300 & CSI500<br>(Stock market datasets)"]
  B --> C["Methodology: TCGPN Architecture<br>Temporal-Correlation Fusion Encoder"]
  C --> D["Pre-training Tasks<br>Temporal & Correlation tasks"]
  D --> E["Memory Optimization<br>Node-independent structure + Sampling"]
  E --> F["Downstream Task: MLP Fine-tuning"]
  F --> G["Outcomes:<br>State-of-the-Art Results<br>Robust Pattern Capture<br>Reduced Memory Consumption"]

Cited References (42)

  1. Characterisation of mental health conditions in social media using Informed Deep Learning (2017)
  2. Hierarchical Adaptive Temporal-Relational Modeling for Stock Trend Prediction. (2021)
  3. Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues (2022)
  4. On a method of investigating periodicities in disturbed series with special reference to Wolfer’s sunspot numbers (1971)
  5. The proof and measurement of association between two things. (1961)
  6. Time series analysis: forecasting and control (2015)
  7. Introduction to time series analysis and forecasting (2015)
  8. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation (1982)
and 34 more
  1. Modelling the persistence of conditional variances (1986)
  2. A dual-stage attention-based recurrent neural network for time series prediction (2017) — link
  3. Price change prediction of ultra high frequency financial data based on temporal convolutional network (2022)
  4. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting (2022)
  5. itransformer: Inverted transformers are effective for time series forecasting (2023) — link
  6. Self-supervised contrastive pre-training for time series via time-frequency consistency (2022)
  7. AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme (2023) — link
  8. Graph attention networks (2017) — link
  9. Attention is all you need (2017)
  10. Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction. (2020)
  11. Stock ranking with multi-task learning (2022)
  12. Temporal pattern attention for multivariate time series forecasting (2019)
  13. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting (2019)
  14. Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification (2021)
  15. Connecting the dots: Multivariate time series forecasting with graph neural networks (2020)
  16. Spectral temporal graph neural network for multivariate time-series forecasting (2020)
  17. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting (2020)
  18. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting (2017) — link
  19. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  20. Improving language understanding by generative pre-training (2018)
  21. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting (2022)
  22. Mining spatio-temporal relations via self-paced graph contrastive learning (2022)
  23. Semi-supervised classification with graph convolutional networks (2016) — link
  24. Hybrid spatio-temporal graph convolutional network: Improving traffic prediction with navigation data (2020)
  25. GSTNet: Global Spatial-Temporal Network for Traffic Flow Prediction. (2019)
  26. Traffic flow forecasting with spatial-temporal graph diffusion network (2021)
  27. Deep spatio-temporal residual networks for citywide crowd flows prediction (2017)
  28. Adaptive graph convolutional recurrent network for traffic forecasting (2020)
  29. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting (2017) — link
  30. T-gcn: A temporal graph convolutional network for traffic prediction (2019)
  31. Traffic flow prediction via spatial temporal graph neural network (2020)
  32. Gman: A graph multi-attention network for traffic prediction (2020)
  33. Hierarchical Adaptive Temporal-Relational Modeling for Stock Trend Prediction. (2021)
  34. Graph wavenet for deep spatial-temporal graph modeling (2019) — link

0 of 42 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.