Paper: arXiv 2404.02270

Abstract

Operational decisions relying on predictive distributions of electricity prices can result in significantly higher profits compared to those based solely on point forecasts. However, the majority of models developed in both academic and industrial settings provide only point predictions. To address this, we examine three postprocessing methods for converting point forecasts of day-ahead electricity prices into probabilistic ones: Quantile Regression Averaging, Conformal Prediction, and the recently introduced Isotonic Distributional Regression. We find that while the latter demonstrates the most varied behavior, it contributes the most to the ensemble of the three predictive distributions, as measured by Shapley values. Remarkably, the performance of the combination is superior to that of state-of-the-art Distributional Deep Neural Networks over two 4.5-year test periods from the German and Spanish power markets, spanning the COVID pandemic and the war in Ukraine.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced statistical methods (quantile regression, conformal prediction, isotonic regression, Shapley values) with significant mathematical formulation, while also demonstrating strong empirical rigor through extensive backtesting on real market data over long periods, including major crises, and comparing against state-of-the-art benchmarks.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Enhance probabilistic forecasting of<br>day-ahead electricity prices"] --> B["Data: German & Spanish Power Markets<br>4.5-year periods<br>COVID & War in Ukraine"]
  B --> C["Input: Point Forecasts"]
  C --> D["Postprocessing Methods"]
  
  subgraph D [" "]
      direction LR
      D1["Quantile Regression Averaging"]
      D2["Conformal Prediction"]
      D3["Isotonic Distributional Regression"]
  end
  
  D --> E["Ensemble of Predictive Distributions"]
  E --> F["Key Findings"]
  
  subgraph F [" "]
      F1["Isotonic Distributional Regression<br>contributes most via Shapley values"]
      F2["Combination outperforms<br>state-of-the-art Deep Neural Networks"]
      F3["Operational decisions based on<br>predictive distributions yield higher profits"]
  end

Cited References (46)

  1. Combining predictive distributions for the statistical post-processing of ensemble forecasts (2018)
  2. CRPS learning (2023)
  3. Forecasting electricity prices with expert, linear, and nonlinear models (2023)
  4. Generative machine learning methods for multivariate ensemble postprocessing (2024)
  5. A probabilistic forecast methodology for volatile electricity prices in the Australian National Electricity Market (2024)
  6. Least angle regression (2004)
  7. Regularization paths for generalized linear models via coordinate descent (2010)
  8. Additive models and robust aggregation for GEFCom2014 probabilistic electric load and electricity price forecasting (2016)
and 38 more
  1. Tests of conditional predictive ability (2006)
  2. Probabilistic solar forecasting: Benchmarks, post-processing, verification (2023)
  3. Strictly Proper Scoring Rules, Prediction, and Estimation (2007)
  4. Combining prediction intervals in the M4 competition (2020)
  5. Quantile evaluation, sensitivity to bracketing, and sharing business payoffs (2017)
  6. Statistical Learning with Sparsity: The Lasso and Generalizations (2015)
  7. Accelerating the Pool-Adjacent-Violators Algorithm for Isotonic Distributional Regression (2022)
  8. Isotonic distributional regression (2021)
  9. Conformal prediction interval estimation and applications to day-ahead and intraday power markets (2021)
  10. Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark (2021)
  11. Is It Better to Average Probabilities or Quantiles? (2013)
  12. Probabilistic load forecasting via Q uantile R egression A veraging on sister forecasts (2017)
  13. From local explanations to global understanding with explainable AI for trees (2020)
  14. Understanding global feature contributions with additive importance measures (2020)
  15. A hybrid model for GEFCom2014 probabilistic electricity price forecasting (2016)
  16. Forecasting Electricity Prices (2023)
  17. Forecasting electricity prices using deep neural networks: A robust hyper-parameter selection scheme (2020)
  18. Distributional neural networks for electricity price forecasting (2023)
  19. Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts? (2020)
  20. Econometric modelling and forecasting of intraday electricity prices (2020)
  21. Combining predictive distributions of electricity prices. Does minimizing the CRPS lead to optimal decisions in day-ahead bidding? (2023)
  22. Computing electricity spot price prediction intervals using quantile regression and forecast averaging (2015)
  23. Recent advances in electricity price forecasting: A review of probabilistic forecasting (2018)
  24. Neural basis expansion analysis with exogenous variables: F orecasting electricity prices with NBEATSx (2023)
  25. Neural Networks for Postprocessing Ensemble Weather Forecasts (2018)
  26. A tutorial on conformal prediction (2008)
  27. On the importance of the long-term seasonal component in day-ahead electricity price forecasting: Part II – P robabilistic forecasting (2019)
  28. Regularized Quantile Regression Averaging for probabilistic electricity price forecasting (2021)
  29. Variance Stabilizing Transformations for Electricity Spot Price Forecasting (2018)
  30. Statistical postprocessing for weather forecasts: Review, challenges, and avenues in a big data world (2021)
  31. Easy Uncertainty Quantification ( EasyUQ ): Generating Predictive Distributions from Single-Valued Model Output (2024)
  32. Combining Probabilistic Load Forecasts (2019)
  33. Electricity price forecasting: A review of the state-of-the-art with a look into the future (2014)
  34. Combining quantiles of calibrated solar forecasts from ensemble numerical weather prediction (2023)
  35. Adaptive Conformal Predictions for Time Series (2022)
  36. Probabilistic mid- and long-term electricity price forecasting (2018)
  37. Day-ahead electricity price forecasting with high-dimensional structures: U nivariate vs. multivariate modeling frameworks (2018)
  38. PostForecasts.jl – J ulia package for postprocessing forecasts (2024) — link

0 of 46 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.