Paper: arXiv 2504.02518

Abstract

Probabilistic electricity price forecasting (PEPF) is vital for short-term electricity markets, yet the multivariate nature of day-ahead prices - spanning 24 consecutive hours - remains underexplored. At the same time, real-time decision-making requires methods that are both accurate and fast. We introduce an online algorithm for multivariate distributional regression models, allowing an efficient modelling of the conditional means, variances, and dependence structures of electricity prices. The approach combines multivariate distributional regression with online coordinate descent and LASSO-type regularization, enabling scalable estimation in high-dimensional covariate spaces. Additionally, we propose a regularized estimation path over increasingly complex dependence structures, allowing for early stopping and avoiding overfitting. In a case study of the German day-ahead market, our method outperforms a wide range of benchmarks, showing that modeling dependence improves both calibration and predictive accuracy. Furthermore, we analyse the trade-off between predictive accuracy and computational costs for batch and online estimation and provide an high-performing open-source Python implementation in the ondil package.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced multivariate distributional regression with LASSO regularization and online coordinate descent, indicating high mathematical sophistication; it includes a comprehensive case study with benchmark comparisons, open-source implementation, and detailed computational cost analysis, demonstrating strong empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>High-dimensional Probabilistic<br>Electricity Price Forecasting"] --> B["Methodology:<br>Online Multivariate<br>Distributional Regression"]
  
  B --> C{"Data & Inputs:<br>German Day-Ahead Market<br>24-hour price vectors + Covariates"}
  
  C --> D["Computational Process:<br>Online Coordinate Descent<br>+ LASSO Regularization"]
  
  D --> E["Regularization Path:<br>Iterative Complexity<br>Selection & Early Stopping"]
  
  E --> F["Key Outcomes:<br>• Superior Accuracy<br>• Improved Calibration<br>• Efficient Computation<br>• Open-source Python package"]
  
  F --> G["Conclusion:<br>Modeling dependence<br>enhances PEPF performance"]

Cited References (104)

  1. Covariance structures for high-dimensional energy forecasting (2022)
  2. Ante-dependence analysis of an ordered set of variables (1962)
  3. Scalable estimation for structured additive distributional regression (2025)
  4. Distribution of the estimators for autoregressive time series with a unit root (1979)
  5. Lag order and critical values of the augmented Dickey–Fuller test (1995)
  6. Probabilistic intraday electricity price forecasting using generative machine learning (2025) — link
  7. Asymmetric penalties underlie proper loss functions in probabilistic forecasting (2025) — link
  8. Note on a method for calculating corrected sums of squares and products (1962)
and 96 more
  1. bashtage/arch: Release 7.2 (2024) — link
  2. Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks (2018)
  3. On the choice of initial guesses for the Newton-Raphson algorithm (2021)
  4. Choosing starting values for certain Newton–Raphson iterations (2006)
  5. Bootstrap prediction bands for forecast paths from vector autoregressive models (2011)
  6. Comparison of methods for constructing joint confidence bands for impulse response functions (2015)
  7. Trading on short-term path forecasts of intraday electricity prices (2022)
  8. Hedging renewable power purchase agreements (2024)
  9. The value of coordination in multimarket bidding of grid energy storage (2023)
  10. Sparse inverse covariance estimation with the graphical lasso (2008)
  11. State of the German short-term power market (2017)
  12. Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark (2021)
  13. Structured antedependence models for longitudinal data (1997)
  14. Computational aspects of likelihood-based estimation of first-order antedependence models (1998)
  15. LASSO-type penalization in the framework of generalized additive models for location, scale and shape (2019)
  16. Incorporating second-order functional knowledge for better option pricing (2000)
  17. Activation functions in deep learning: A comprehensive survey and benchmark (2022)
  18. Noncanonical links in generalized linear models–when is the effort justified? (2000)
  19. Using the softplus function to construct alternative link functions in generalized linear models and beyond (2024)
  20. Additive covariance matrix models: modeling regional electricity net-demand in Great Britain (2025)
  21. Scalable Fitting Methods for Multivariate Gaussian Additive Models with Covariate-dependent Covariance Matrices (2025) — link
  22. Online Distributional Regression (2024) — link
  23. Scikit-learn: Machine learning in Python (2011)
  24. Numba: A llvm-based python jit compiler (2015)
  25. gamlss. lasso: Extra Lasso-Type Additive Terms for GAMLSS (2021)
  26. Probabilistic forecasting in day-ahead electricity markets: Simulating peak and off-peak prices (2020)
  27. Covariance Estimation: The GLM and Regularization Perspectives (2011)
  28. Numerical analysis for statisticians (2010)
  29. Generalized additive models for location, scale and shape (2005)
  30. Probabilistic multivariate electricity price forecasting using implicit generative ensemble post-processing (2020)
  31. Multivariate probabilistic forecasting of electricity prices with trading applications (2025)
  32. Multiple split approach–multidimensional probabilistic forecasting of electricity markets — → 2407.07795
  33. Bidding and scheduling in energy markets: Which probabilistic forecast do we need? (2022)
  34. The value of probabilistic forecasts for electricity market bidding and scheduling under uncertainty (2024)
  35. Semi-structured distributional regression (2024)
  36. Deep distributional time series models and the probabilistic forecasting of intraday electricity prices (2023)
  37. Marginally calibrated deep distributional regression (2021)
  38. Forecasting electricity prices with expert, linear, and nonlinear models (2023)
  39. M5 competition uncertainty: Overdispersion, distributional forecasting, GAMLSS, and beyond (2022)
  40. Ensemble forecasting for intraday electricity prices: Simulating trajectories (2020)
  41. Online adaptive estimation of sparse signals: Where RLS meets the $\ell_1$-norm (2010)
  42. Iteratively reweighted adaptive lasso for conditional heteroscedastic time series with applications to ar–arch type processes (2016)
  43. Adaptive Covariance Matrix Estimation through block thresholding (2012)
  44. On-line estimation with the multivariate Gaussian distribution (2007)
  45. Online mixed missing value imputation using gaussian copula (2020)
  46. Probabilistic price forecasts for short-term trade optimization (2024) — link
  47. Bayesian copulae distributions, with application to operational risk management—some comments (2013)
  48. Regions of reliability in the evaluation of multivariate probabilistic forecasts (2023)
  49. Multivariate forecasting evaluation: On sensitive and strictly proper scoring rules (2019) — link
  50. Strictly proper scoring rules, prediction, and estimation (2007)
  51. Estimation of the continuous ranked probability score with limited information and applications to ensemble weather forecasts (2018)
  52. Generalized additive models for location, scale and shape: a distributional regression approach, with applications (2024)
  53. Distributional regression for data analysis (2024)
  54. Scoringrules: a python library for probabilistic forecast evaluation (2024) — link
  55. Comparing predictive accuracy, twenty years later: A personal perspective on the use and abuse of Diebold–Mariano tests (2015)
  56. Comparing predictive accuracy (2002)
  57. Discrimination ability of the energy score (2013)
  58. Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities (2015)
  59. Evaluating the discrimination ability of proper multi-variate scoring rules (2024)
  60. Coherent dispersion criteria for optimal experimental design (1999)
  61. Probabilistic forecasts, calibration and sharpness (2007)
  62. Online coordinate descent for adaptive estimation of sparse signals (2009)
  63. Adaptive generalized logit-normal distributions for wind power short-term forecasting (2021)
  64. Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting (2019)
  65. Conformal prediction interval estimation and applications to day-ahead and intraday power markets (2021)
  66. Modeling multivariate intraday forecast update processes for wind power (2024)
  67. Recent developments in multivariate wind and solar power forecasting (2022) — link
  68. Distributional neural networks for electricity price forecasting (2023)
  69. NBMLSS: probabilistic forecasting of electricity prices via Neural Basis Models for Location Scale and Shape (2024) — link
  70. On-line conformalized neural networks ensembles for probabilistic forecasting of day-ahead electricity prices (2025)
  71. Adaptive conformal inference under distribution shift (2021)
  72. Conformal inference for online prediction with arbitrary distribution shifts (2024)
  73. Adaptive conformal predictions for time series (2022)
  74. Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression (2024)
  75. An introduction to multivariate probabilistic forecast evaluation (2021)
  76. Assessing the performance of deep learning models for multivariate probabilistic energy forecasting (2021)
  77. Probabilistic Multivariate Time Series Forecasting and Robust Uncertainty Quantification with Applications in Electricity Price Prediction (2023)
  78. Rage against the mean–a review of distributional regression approaches (2023)
  79. Distributional modeling and short-term forecasting of electricity prices by generalized additive models for location, scale and shape (2011)
  80. From point forecasts to multivariate probabilistic forecasts: The Schaake shuffle for day-ahead electricity price forecasting (2023)
  81. Multivariate probabilistic crps learning with an application to day-ahead electricity prices — → 2303.10019
  82. Recent advances in electricity price forecasting: A review of probabilistic forecasting (2018)
  83. Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives (1984)
  84. Smoothing reference centile curves: the LMS method and penalized likelihood (1992)
  85. Probabilistic Prediction of Oceanographic Velocities with Multivariate Gaussian Natural Gradient Boosting (2023)
  86. XGBoostLSS–An extension of XGBoost to probabilistic forecasting (2019) — link
  87. High-dimensional multivariate forecasting with low-rank gaussian copula processes (2019)
  88. Distributional random forests: Heterogeneity adjustment and multivariate distributional regression (2022)
  89. Multi-target xgboostlss regression (2022) — link
  90. Variable selection using a smooth information criterion for distributional regression models (2023)
  91. Robust fitting for generalized additive models for location, scale and shape (2021)
  92. Truly multivariate structured additive distributional regression (2025)
  93. Cholesky-based multivariate Gaussian regression (2022)
  94. Online missing value imputation and change point detection with the gaussian copula (2022)
  95. Online heterogeneous mixture modeling with marginal and copula selection (2011)
  96. A streaming algorithm for bivariate empirical copulas (2019)

2 of 104 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.