Variable transformations in consistent loss functions

The empirical use of variable transformations within (strictly) consistent loss functions is widespread, yet a theoretical understanding is lacking. To address this gap, we develop a theoretical framework that establishes formal characterizations of (strict) consistency for such transformed loss functions. Our analysis focuses on two interrelated cases: (a) transformations applied solely to the realization variable and (b) bijective transformations applied jointly to both the realization and prediction variables. These cases extend the well-established framework of transformations applied exclusively to the prediction variable, as formalized by Osband’s revelation principle. We further develop analogous characterizations for (strict) identification functions. The resulting theoretical framework is broadly applicable to statistical and machine learning methodologies. For instance, we apply the framework to Bregman and expectile loss functions to interpret empirical findings from models trained with transformed loss functions and systematically construct new identifiable and elicitable functionals, which we term respectively $g$-transformed expectation and $g$-transformed expectile. Applications of the framework to simulated and real-world data illustrate its practical utility in diverse settings. By unifying theoretical insights with practical applications, this work advances principled methodologies for designing loss functions in complex predictive tasks.

💡 Research Summary

The paper addresses a gap in the theoretical understanding of variable transformations applied within (strictly) consistent loss functions, a practice that is common in many applied fields but has lacked rigorous justification. The authors develop a comprehensive framework that characterizes when transformed loss functions retain (strict) consistency for a given statistical functional, and they extend the analysis to identification functions. Their work focuses on two main transformation scenarios: (a) applying a bijective transformation g solely to the realization variable Y, and (b) applying the same bijective transformation jointly to both the prediction x and the realization Y. These scenarios complement the well‑known Osband revelation principle, which deals with transformations of the prediction variable only.

Theoretical contributions begin with a concise review of consistent loss functions, elicitable functionals, and identification functions, establishing the necessary notation. Theorem 3 proves that if a base loss L is (strictly) consistent for a functional Γ, then the transformed loss L⁽ᵍ⁾(x, y)=L(x, g(y)) is (strictly) consistent for the g‑transformed functional Γᵍ = g⁻¹∘Γ. This defines a new class of functionals, termed g‑transformed expectations, which generalize means, quantiles, and expectiles under the transformation g. Theorem 4 handles the joint transformation case, showing that L⁽ᵍ⁾(x, y)=L(g(x), g(y)) is (strictly) consistent for Γᵍ = g⁻¹∘Γ∘g, provided g is bijective and sufficiently smooth. Remark 1 synthesizes the two cases, giving a unified consistency condition for loss functions of the form L(g(x), y) or L(x, g(y)). Parallel results are derived for identification functions, demonstrating that the duality between identification and consistent loss functions is preserved under these transformations.

To illustrate the theory, the authors apply it to Bregman losses and expectile losses. By choosing specific bijections (e.g., logarithm, square‑root, reciprocal), they construct new loss functions that are consistent for g‑transformed expectations and g‑transformed expectiles. The analysis shows that such transformed losses can attenuate the influence of outliers or adapt to skewed distributions, explaining empirical observations in prior work where transformed losses yielded better predictive performance.

Empirical validation proceeds in two stages. First, Monte‑Carlo simulations generate data from a variety of distributions (normal, log‑normal, heavy‑tailed, skewed). For each setting, the authors compare the bias and variance of estimators obtained by minimizing the original loss versus the g‑transformed loss. Results consistently demonstrate that the transformed losses produce estimators that are closer to the target functional, confirming the theoretical consistency claims. Second, real‑world case studies involve air‑quality (PM2.5) measurements and financial return series. Models trained with g‑transformed expectile loss achieve lower mean absolute error, root mean squared error, and improved risk metrics (e.g., VaR, CVaR) compared with models using standard expectile or squared‑error losses.

The discussion emphasizes practical implications: practitioners can deliberately select a transformation g to tailor the loss function’s sensitivity to domain‑specific characteristics such as asymmetry or extreme values, without sacrificing the theoretical guarantee that the loss remains consistent for a well‑defined functional. Moreover, because identification functions transform analogously, one can design new identifiable functionals by first specifying a suitable g and then deriving the corresponding loss via Osband’s principle.

In conclusion, the paper makes four key contributions: (1) it provides the first rigorous consistency theory for transformations applied to the realization variable; (2) it integrates joint transformations of prediction and realization with the classic Osband revelation principle; (3) it introduces the novel concepts of g‑transformed expectation and g‑transformed expectile, expanding the family of elicitable and identifiable functionals; and (4) it validates the theory through extensive simulations and real‑data applications, demonstrating tangible benefits in predictive modeling. Future work is suggested on multivariate extensions, non‑bijective transformations, and generalization to proper scoring rules for probabilistic forecasts.