Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

Paper: arXiv 2511.13616

Authors: Katarzyna Maciejowska, Arkadiusz Lipiecki, Bartosz Uniejewski

Abstract

In recent years, a rapid development of forecasting methods has led to an increase in the accuracy of predictions. In the literature, forecasts are typically evaluated using metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). While appropriate for statistical assessment, these measures do not adequately reflect the economic value of forecasts. This study addresses the decision-making problem faced by a battery energy storage system, which must determine optimal charging and discharging times based on day-ahead electricity price forecasts. To explore the relationship between forecast accuracy and economic value, we generate a pool of 192 forecasts. These are evaluated using seven statistical metrics that go beyond RMSE and MAE, capturing various characteristics of the predictions and associated errors. We calculate the dynamic correlation between the statistical measures and gained profits to reveal that both RMSE and MAE are only weakly correlated with revenue. In contrast, measures that assess the alignment between predicted and actual daily price curves have a stronger relationship with profitability and are thus more effective for selecting optimal forecasts.

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced statistical and econometric methods, including complex forecasting models (ARX, NARX, LEAR) and a custom battery trading simulation, requiring significant data processing and implementation. However, it lacks explicit code or algorithmic details in the excerpt, focusing on empirical results and methodology rather than mathematical derivations.

Research Flowchart

  flowchart TD
  A["Research Goal: Assess the economic value<br>of electricity price forecasts<br>for battery storage operation"] --> B{"Methodology"}
  
  B --> C["Input: 192 diverse day-ahead<br>electricity price forecasts"]
  B --> D["Statistical Evaluation:<br>7 metrics beyond RMSE/MAE<br>e.g., MAPE, Correlation"]
  
  C --> E["Simulation: Battery Storage<br>Optimization based on forecasts"]
  D --> F["Comparison: Dynamic correlation<br>between statistical metrics<br>and actual profits"]
  
  E --> G["Outcome: Calculated revenue<br>for each forecast"]
  G --> F
  
  F --> H{"Key Findings"}
  H --> I["RMSE & MAE show<br>weak correlation with revenue"]
  H --> J["Measures of curve alignment<br>strongly predict profitability"]
  H --> K["Best practice: Select forecasts<br>based on economic utility, not<br>just statistical error"]

Cited References (47)

1. Electricity price forecasting on the day-ahead market using machine learning (2022)
2. Electricity Price Forecasting: The Dawn of Machine Learning (2022)
3. Multi-service based economic valuation of grid-connected battery energy storage systems (2022)
4. On the participation of energy storage systems in reserve markets using Decision Focused Learning (2025)
5. Decision-Focused Learning Enhanced by Automated Feature Engineering for Energy Storage Optimisation (2026)
6. Monetizing energy storage (2023)
7. Prescriptive Trees for Integrated Forecasting and Optimization Applied in Trading of Renewable Energy (2022)
8. Economic Impact of Electricity Market Price Forecasting Errors: A Demand-Side Analysis (2010)

1 of 47 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.