How to Compare Copula Forecasts?

This paper lays out a principled approach to compare copula forecasts via strictly consistent scores. We first establish the negative result that, in general, copulas fail to be elicitable, implying that copula predictions cannot sensibly be compared on their own. A notable exception is on Fréchet classes, that is, when the marginal distribution structure is given and fixed, in which case we give suitable scores for the copula forecast comparison. As a remedy for the general non-elicitability of copulas, we establish novel multi-objective scores for copula forecast along with marginal forecasts. They give rise to two-step tests of equal or superior predictive ability which admit attribution of the forecast ranking to the accuracy of the copulas or the marginals. Simulations show that our two-step tests work well in terms of size and power. We illustrate our new methodology via an empirical example using copula forecasts for international stock market indices.

💡 Research Summary

The paper addresses the fundamental problem of how to compare copula forecasts in a principled way. It begins by establishing a negative result: the copula functional is generally not elicitable. Using the necessary condition of convex level sets (CxLS) for elicitability, the authors show in Proposition 4 that even on very small classes of distributions with bounded support, translations, and mixtures, the copula functional fails to have convex level sets. Consequently, no strictly consistent scoring function exists for copula forecasts taken in isolation, meaning that direct ranking of copula predictions is theoretically impossible.

The authors then identify an important exception: Fréchet classes, where all members share the same marginal distributions and differ only in their dependence structure (the copula). In this setting, any two distributions with the same copula are identical, and convex combinations preserve the copula, satisfying the CxLS condition. Proposition 5 demonstrates that any proper scoring rule defined on the full joint distribution (e.g., log‑score, continuous ranked probability score) can be restricted to the copula component while keeping the margins fixed, yielding a strictly consistent score for the copula functional. This formally justifies the common practice of using Kullback‑Leibler information criteria for copula selection under known margins.

To handle the general case where margins are also forecasted, the paper adopts the recent notion of multi‑objective elicitability. By pairing the copula with all marginal distributions, the authors consider the tuple (C, F₁,…,F_d) as the object of interest. Theorem 10 proves the existence of strictly consistent multi‑objective scoring functions for this tuple, using a lexicographic ordering of the expected scores. Building on these scores, the authors propose a two‑step Diebold‑Mariano (DM) testing procedure. Step 1 tests the equality of marginal predictive ability; Step 2, conditional on the marginal null, tests the equality or superiority of copula forecasts. Both steps use the classic DM statistic, but critical values are adjusted so that the overall test maintains the nominal size across the two stages.

Monte‑Carlo simulations assess the finite‑sample performance. The two‑step tests preserve size even in relatively small samples (n≈50) and exhibit high power to discriminate between marginal and copula forecast quality. Power in the copula stage is higher when marginal forecasts are less misspecified, reflecting the intuition that errors in margins can mask copula differences. Moreover, the attribution property works well: differences in marginal accuracy are almost exclusively detected in Step 1, while copula differences appear in Step 2.

An empirical application uses international stock‑index return data. The authors compare state‑of‑the‑art models: DCC‑GARCH and t‑GAS, each providing marginal density forecasts and dynamic copula forecasts. The two‑step tests reveal that while both models produce comparable marginal density forecasts, the t‑GAS copula forecasts are statistically superior to those from the DCC‑GARCH model. This illustrates how the proposed methodology can guide model selection in practice.

In summary, the paper makes four major contributions: (1) a rigorous proof of the non‑elicitability of copulas in general; (2) identification of Fréchet classes where copulas become elicitable and provision of suitable strictly consistent scores; (3) construction of multi‑objective strictly consistent scores for the joint object (copula + marginals) and the derivation of a two‑step DM testing framework; (4) extensive simulation and real‑data evidence that the proposed tools have correct size, high power, and useful attribution properties. By bridging the gap between theoretical decision‑theoretic foundations and practical forecast evaluation, the paper offers a comprehensive toolkit for researchers and practitioners working with copula‑based predictive models across finance, economics, and other fields.