Leaky-integrator reconstruction: taming error accumulation in recursive differenced time-series forecasting

Paper: arXiv 2609.23378

Authors: Zijiang Yang

Abstract

Recursive differenced forecasting, the standard remedy for non-stationarity, predicts one-step changes and integrates them by cumulative summation. We show that this reconstruction is a discrete integrator with a pole on the unit circle, so the biased increment errors of a learned nonlinear model are summed without bound and the rollout diverges: at 336 steps its normalised MAE reaches 1.6-3.8 for every neural architecture tested, against 0.80 for a stable linear recursion. We then introduce leaky-integrator reconstruction, a training-free fix that moves the pole inside the unit circle with H(z) = 1/(1 - gamma z^-1), gamma < 1, bounding the accumulation of the model’s own increment errors. Applied post hoc with a single fixed gamma=0.9 (no retraining, a two-line change to any deployed one-step or foundation-model forecaster), it beats the traditional recursive integrator at every horizon, with the mean gain over seven diverging architectures and twenty datasets growing from ~3% at H=24 to 23% at H=96, 37% at H=192 and 51% at H=336 (43-75% across those architectures; 78% with an oracle pole), bringing all of them to 0.87-0.97. Based on these extensive empirical experiments, adding a leaky integrator thus improves recursive differenced time-series forecasting.

Complexity vs Empirical Score

• Math Complexity: 6.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a novel and effective solution to a known problem in time-series forecasting, backed by extensive empirical validation across multiple architectures and datasets. The mathematical foundation is sound, and the practical implications are significant for deployed models.

Research Flowchart

  flowchart TD
    A[Research Goal: Tame Error Accumulation in Recursive Differenced TS Forecasting] --> B{Standard Approach: Recursive Differenced Forecasting};
    B -- Problem Identified --> C[Problem: Traditional Integrator (pole on unit circle) causes unbounded error accumulation];
    C --> D[Proposed Solution: Leaky-Integrator Reconstruction (pole inside unit circle)];
    D -- Applied to --> E[Data/Models: 7 Neural Architectures, 20 Datasets, Post-hoc application with gamma=0.9];
    E -- Evaluation --> F[Computational Process: Compare MAE of Traditional vs. Leaky Integrator across horizons];
    F --> G[Key Finding: Leaky-Integrator consistently beats Traditional Integrator across horizons, architectures, and datasets, reducing MAE significantly];

Cited References (22)

1. Time Series Analysis: Forecasting and Control (1970)
2. Long Short-Term Memory (1997)
3. Reversible Instance Normalization for Accurate Time-Series Forecasting Against Distribution Shift (2022)
4. Are Transformers Effective for Time Series Forecasting? (2023)
5. Informer : Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021)
6. A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers (2023)
7. iTransformer : Inverted Transformers Are Effective for Time Series Forecasting (2024)
8. Autoformer : Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021)

0 of 22 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.