A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition

Multi-step ahead forecasting is still an open challenge in time series forecasting. Several approaches that deal with this complex problem have been proposed in the literature but an extensive comparison on a large number of tasks is still missing. This paper aims to fill this gap by reviewing existing strategies for multi-step ahead forecasting and comparing them in theoretical and practical terms. To attain such an objective, we performed a large scale comparison of these different strategies using a large experimental benchmark (namely the 111 series from the NN5 forecasting competition). In addition, we considered the effects of deseasonalization, input variable selection, and forecast combination on these strategies and on multi-step ahead forecasting at large. The following three findings appear to be consistently supported by the experimental results: Multiple-Output strategies are the best performing approaches, deseasonalization leads to uniformly improved forecast accuracy, and input selection is more effective when performed in conjunction with deseasonalization.

💡 Research Summary

This paper addresses the challenging problem of multi‑step‑ahead time‑series forecasting by providing a comprehensive review and large‑scale empirical comparison of five widely cited forecasting strategies: Recursive, Direct, DirRec, Multi‑Input Multi‑Output (MIMO), and DIRMO. The authors first present a unified theoretical framework that clarifies the assumptions, model‑complexity, error‑propagation characteristics, and the way each method handles inter‑step dependencies. Recursive forecasting iterates a single one‑step model H times, which is simple but suffers from error accumulation, especially when the forecasting horizon exceeds the embedding dimension. Direct forecasting builds H independent models, eliminating error propagation but ignoring the stochastic dependence among future values. DirRec combines the two by using separate models for each horizon while augmenting inputs with previously predicted values, thereby partially preserving long‑term dependencies. MIMO predicts the entire H‑step horizon in a single shot with a multi‑output model, naturally capturing joint dynamics and minimizing error propagation. DIRMO lies between Direct and MIMO, partitioning the horizon into blocks and learning separate models for each block, aiming to balance flexibility and dependency preservation.

To evaluate these strategies, the authors conduct an extensive experimental study on the NN5 forecasting competition dataset, which comprises 111 daily series featuring multiple seasonalities, outliers, missing values, and occasional structural breaks. All experiments use a Lazy Learning algorithm (a local‑learning nearest‑neighbor regression) as the underlying predictive engine, implemented in both single‑output and multi‑output variants. The experimental protocol follows best‑practice guidelines: each strategy is tested under four configurations—raw series, deseasonalized series, input‑variable selection, and forecast combination (ensemble). Performance is measured with the competition’s standard metrics (MASE, sMAPE, RMSE) and statistical significance is assessed using the Friedman test with Nemenyi post‑hoc analysis.

The results are clear and consistent across all configurations. The MIMO strategy consistently yields the lowest error rates, outperforming Recursive, Direct, DirRec, and DIRMO with statistical significance. Deseasonalization improves accuracy for every strategy, typically reducing errors by 5–10 %. Input‑variable selection alone has modest impact, but when combined with deseasonalization it leads to further gains, especially for the Recursive and DirRec approaches. Forecast combination provides modest additional benefits for Direct and DirRec but does not substantially affect MIMO, which already captures joint dynamics efficiently. Moreover, the Lazy Learning model itself ranks within the top 10 % of submissions to the NN5 competition, demonstrating that a well‑tuned non‑parametric local learner can compete with sophisticated neural‑network and statistical models.

The paper also discusses practical implications: for practitioners needing reliable long‑horizon forecasts, adopting a multi‑output approach such as MIMO is advisable, particularly when paired with proper seasonal adjustment and thoughtful input selection. The authors acknowledge limitations, including the focus on a single learning algorithm and the absence of deep‑learning baselines, and suggest future work on hybrid meta‑learning ensembles, computational cost analysis for real‑time deployment, and extensions to other data domains.

In summary, this study provides a rigorous, theory‑backed, and empirically validated comparison of multi‑step forecasting strategies, establishing MIMO as the most effective approach on a realistic, heterogeneous benchmark and offering clear guidance on preprocessing choices that enhance forecast quality.