Paper: arXiv 2411.15674

Abstract

Uncertainty quantification is crucial in time series prediction, and quantile regression offers a valuable mechanism for uncertainty quantification which is useful for extreme value forecasting. Although deep learning models have been prominent in multi-step ahead prediction, the development and evaluation of quantile deep learning models have been limited. We present a novel quantile regression deep learning framework for multi-step time series prediction. In this way, we elevate the capabilities of deep learning models by incorporating quantile regression, thus providing a more nuanced understanding of predictive values. We provide an implementation of prominent deep learning models for multi-step ahead time series prediction and evaluate their performance under high volatility and extreme conditions. We include multivariate and univariate modelling, strategies and provide a comparison with conventional deep learning models from the literature. Our models are tested on two cryptocurrencies: Bitcoin and Ethereum, using daily close-price data and selected benchmark time series datasets. The results show that integrating a quantile loss function with deep learning provides additional predictions for selected quantiles without a loss in the prediction accuracy when compared to the literature. Our quantile model has the ability to handle volatility more effectively and provides additional information for decision-making and uncertainty quantification through the use of quantiles when compared to conventional deep learning models.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper utilizes advanced statistical concepts like quantile regression and deep learning architectures (LSTM, CNN) but remains accessible to quant practitioners, while the empirical rigor is high due to implementation on real cryptocurrency datasets (Bitcoin, Ethereum), use of benchmark time series data, and explicit comparison with conventional models.

Research Flowchart

  flowchart TD
  Start["Research Goal: Develop & Evaluate<br/>Quantile Deep Learning<br/>for Multi-Step Time Series"] --> Inputs["Data & Inputs<br/>Bitcoin, Ethereum, Benchmark Datasets<br/>(High Volatility/Extreme Conditions)"]
  Inputs --> Method["Key Methodology<br/>Quantile Regression Loss Function<br/>Embedded in Deep Learning Models<br/>Multi-Step Ahead Forecasting"]
  Method --> Process["Computational Process<br/>Train Models (Univariate/Multivariate)<br/>Evaluate against Baselines<br/>Quantify Uncertainty"]
  Process --> Results["Key Findings & Outcomes<br/>Improved Extreme Value Handling<br/>Accurate Quantile Predictions<br/>Better Uncertainty Quantification<br/>No Loss in Accuracy"]

Cited References (146)

  1. An empirical study on modeling and prediction of bitcoin prices with bayesian neural networks based on blockchain information (2017)
  2. Bayesian neural networks (1989)
  3. Bayesian approach for neural networks—review and case studies (2001)
  4. Cryptocurrency icons (2023) — link
  5. Gears icons (2023) — link
  6. Feature extraction, classification and forecasting of time series signal using fuzzy and GARCH techniques (2013)
  7. Time Series Prediction and Neural Networks (2001)
  8. Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks (2012) — link
and 138 more
  1. Evaluation of deep learning models for Australian climate extremes: prediction of streamflow and floods (2024) — link
  2. Machine learning approaches to forecasting cryptocurrency volatility: Considering internal and external determinants (2023)
  3. Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction (2012)
  4. On hyperparameter optimization of machine learning algorithms: Theory and practice (2020)
  5. LASSO: A feature selection technique in predictive modeling for machine learning (2016)
  6. Performance metrics (error measures) in machine learning regression, forecasting and prognostics: Properties and typology (2018) — link
  7. Investigation of performance metrics in regression analysis and machine learning-based prediction models (2022)
  8. Time-series forecasting with deep learning: a survey (2021)
  9. Deep learning for time series forecasting: a survey (2021)
  10. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021)
  11. Evaluation of deep learning models for multi-step ahead time series prediction (2021)
  12. Long short-term memory (1997)
  13. Deep learning framework with Bayesian data imputation for modelling and forecasting groundwater levels (2024)
  14. A novel time series probabilistic prediction approach based on the monotone quantile regression neural network (2023)
  15. Neural Networks for Partially Linear Quantile Regression (2023) — link
  16. Review of deep learning models for crypto price prediction: implementation and evaluation (2024) — link
  17. A review of recurrent neural networks: LSTM cells and network architectures (2019)
  18. Ensemble deep learning: A review (2022)
  19. Sur la loi de probabilit 'e de l’ 'e cart maximum (1927)
  20. Limiting forms of the frequency distribution of the largest and smallest member of a sample (1928)
  21. Characterization of the tail of the distribution of earthquake magnitudes by combining the GEV and GPD descriptions of extreme value theory (2014)
  22. The generalized extreme value distribution (2003)
  23. Fitting the generalized Pareto distribution to data (1997)
  24. Extreme value theory for risk managers (1999)
  25. On prediction of extreme values (1973)
  26. Imbalanced regression and extreme value prediction (2020)
  27. Climate variation and prediction of rapid intensification in tropical cyclones in the western North Pacific (2008)
  28. Regression quantiles (1978) — link
  29. Quantile regression: A short story on how and why (2018) — link
  30. Using quantile regression forest to estimate uncertainty of digital soil mapping products (2017)
  31. Estimation of predictive hydrologic uncertainty using the quantile regression and UNEEC methods and their comparison on contrasting catchments (2015)
  32. The heterogeneous effects of urbanization and income inequality on CO2 emissions in BRICS economies: evidence from panel quantile regression (2018)
  33. Applications for quantile regression in epidemiology (2019)
  34. Economic applications of quantile regression (2013)
  35. Quantile regression for longitudinal data (2004)
  36. Impact of oil price, economic globalization, and inflation on economic output: Evidence from Latin American oil-producing countries using the quantile-on-quantile approach (2024) — link
  37. Application of quantile regression to recent genetic and-omic studies (2014)
  38. Quantile regression: its application in investment analysis (2009)
  39. Changes in the US wage structure 1963-1987: Application of quantile regression (1994)
  40. Theory of the Backpropagation Neural Network (1992)
  41. Linear least squares regression (1967)
  42. Partial least-squares regression: a tutorial (1986)
  43. Quantile regression and its applications: a primer for anesthesiologists (2019)
  44. Quantile regression: 40 years on (2017) — link
  45. A review of predictive uncertainty estimation with machine learning (2024) — link
  46. Regression quantiles for time series (2002) — link
  47. Time series quantile regressions (2012) — link
  48. Uncertainty estimation in spatial interpolation of satellite precipitation with ensemble learning (2024)
  49. A Review on the Long Short-Term Memory Model (2020) — link
  50. The vanishing gradient problem during learning recurrent neural nets and problem solutions (1998)
  51. Framewise phoneme classification with bidirectional LSTM and other neural network architectures (2005)
  52. Finding structure in time (1990)
  53. Sequence to sequence learning with neural networks (2014)
  54. Convolutional LSTM network: A machine learning approach for precipitation nowcasting (2015)
  55. A global time series of traffic volumes on extra-urban roads (2024) — link
  56. A Probabilistic Runoff Prediction Model Based on Improved Long Short-Term Memory and Interval Correction (2024)
  57. A novel time series probabilistic prediction approach based on the monotone quantile regression neural network (2024) — link
  58. An improved quantile regression neural network for probabilistic load forecasting (2018)
  59. Problems in the extreme value analysis (2008)
  60. Temporal prediction modeling for rainfall-induced shallow landslide hazards using extreme value distribution (2021)
  61. The casualty prediction of earthquake disaster based on Extreme Learning Machine method (2020)
  62. Extreme value prediction via a quantile function model (2013)
  63. Gradient boosting for extreme quantile regression (2023)
  64. Extreme value modelling for forecasting market crisis impacts (2014)
  65. Quantifying the influence of corrosion defects on the failure prediction of natural gas pipelines using generalized extreme value distribution (GEVD) model and Copula function with a case study (2024)
  66. Early warning systems for currency crises: A multivariate extreme value approach (2013)
  67. Application of the statistics of extremes to the reliability assessment and performance prediction of monitored highway bridges (2019)
  68. Representing external hazard initiating events using a Bayesian approach and a generalized extreme value model (2020)
  69. Statistical inference using extreme order statistics (1975)
  70. On the block maxima method in extreme value theory: PWM estimators (2015) — link
  71. Deep learning for quantile regression under right censoring: DeepQuantreg (2022)
  72. A horse race between the block maxima method and the peak–over–threshold approach (2021) — link
  73. The effect of the generalized extreme value distribution parameter estimation methods in extreme wind speed prediction (2015)
  74. Predicting extreme value at risk: Nonparametric quantile regression with refinements from extreme value theory (2012)
  75. A semiparametric Bayesian approach to extreme value estimation (2012)
  76. Deep learning for time-series analysis (2017) — link
  77. A conceptual model of investment-risk prediction in the stock market using extreme value theory with machine learning: a semisystematic literature review (2023) — link
  78. The asymptotic decision scenarios of an emerging stock exchange market: Extreme value theory and artificial neural network (2018) — link
  79. Modeling of Machine Learning-Based Extreme Value Theory in Stock Investment Risk Prediction: A Systematic Literature Review (2024) — link
  80. Survey on the application of deep learning in extreme weather prediction (2021)
  81. High-speed arithmetic in binary computers (1961)
  82. Computer Organization and Design: The Hardware/Software Interface (2013)
  83. Random search for hyper-parameter optimization (2012)
  84. EvaNet: An extreme value attention network for long-term air quality prediction (2020)
  85. Early prediction of extreme rainfall events: a deep learning approach (2016)
  86. Community detection in graphs (2010)
  87. Finding and evaluating community structure in networks (2004)
  88. Visualizing Fuzzy Overlapping Communities in Networks (2013)
  89. Adam: A Method for Stochastic Optimization (2017) — link
  90. Near linear time algorithm to detect community structures in large-scale networks (2007)
  91. Robust network community detection using balanced propagation (2011)
  92. Detecting community structure using label propagation with weighted coherent neighborhood propinquity (2013)
  93. Finding community structure in very large networks (2004)
  94. Fast unfolding of communities in large networks (2008)
  95. General optimization technique for high-quality community detection in complex networks (2014)
  96. Resolution limit in community detection (2007)
  97. Unfolding communities in large complex networks: Combining defensive and offensive label propagation for core extraction (2011)
  98. Detecting communities by the core-vertex and intimate degree in complex networks (2013)
  99. Detecting overlapping communities by seed community in weighted complex networks (2013)
  100. Profiling core-periphery network structure by random walkers (2013)
  101. Detecting local community structure in complex networks based on local degree central nodes (2013)
  102. Identification of overlapping community structure in complex networks using fuzzy c-means clustering (2007)
  103. Fuzzy communities and the concept of bridgeness in complex networks (2008)
  104. Fuzzy community structure detection by particle competition and cooperation (2013)
  105. Identification of overlapping and non-overlapping community structure by fuzzy clustering in complex networks (2011)
  106. Fuzzy overlapping community detection based on local random walk and multidimensional scaling (2013)
  107. Overlapping community detection using Bayesian non-negative matrix factorization (2011)
  108. Overlapping Community Detection via Bounded Nonnegative Matrix Tri-Factorization (2012)
  109. Fuzzy modularity and fuzzy community structure in networks (2010)
  110. Oscillation and chaos in physiological control systems (1977) — link
  111. Bayesian Learning for Neural Networks (1996)
  112. A Comparison of Discriminant Analysis versus Artificial Neural Networks (1993) — link
  113. A Practical Bayesian Framework for Backpropagation Networks (1992) — link
  114. Bayesian neural networks for stock price forecasting before and during COVID-19 pandemic (2021) — link
  115. Forecasting financial volatilities with extreme values: the conditional autoregressive range (CARR) model (2005)
  116. Extreme value modelling for forecasting market crisis impacts (2014)
  117. Analysis of extreme values (1950)
  118. Flood risk curves and uncertainty bounds (2009)
  119. Mackey-Glass Timeseries Dataset (2021)
  120. A Soft Modularity Function For Detecting Fuzzy Communities in Social Networks (2013)
  121. Network data (2013)
  122. The Statistical Prediction of Solutions of Dynamic Equations (1960) — link
  123. Ubiquitousness of link-density and link-pattern communities in real-world networks (2012)
  124. Benchmark graphs for testing community detection algorithms (2008)
  125. Lecture notes on ridge regression (2015)
  126. Inferring parameters and structure of latent variable models by variational Bayes (2013)
  127. Cyclical stochastic gradient MCMC for Bayesian deep learning (2019)
  128. Sentiment analysis in the era of large language models: A reality check (2023)
  129. Multi-model fusion framework using deep learning for visual-textual sentiment classification (2023)
  130. Dropout training as adaptive regularization (2013)
  131. Detecting Communities Based on Network Topology (2014)
  132. Comparing community structure identification (2005)
  133. Fuzzy overlapping communities in networks (2011)
  134. Missing data imputation of high‐resolution temporal climate time series data (2020)
  135. Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities (2009)
  136. A Fuzzy Variant of the Rand Index for Comparing Clustering Structures (2009)
  137. Using the Bitcoin Transaction Graph to Predict the Price of Bitcoin (2015) — link
  138. Frequentist approach: Modelling and simulation in statistics and probability teaching (2011)

0 of 146 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.