CRPS-Optimal Binning for Conformal Regression

We propose a method for non-parametric conditional distribution estimation based on partitioning covariate-sorted observations into contiguous bins and using the within-bin empirical CDF as the predictive distribution. Bin boundaries are chosen to minimise the total leave-one-out Continuous Ranked Probability Score (LOO-CRPS), which admits a closed-form cost function with $O(n^2 \log n)$ precomputation and $O(n^2)$ storage; the globally optimal $K$-partition is recovered by a dynamic programme in $O(n^2 K)$ time. Minimisation of Within-sample LOO-CRPS turns out to be inappropriate for selecting $K$ as it results in in-sample optimism. So we instead select $K$ by evaluating test CRPS on an alternating held-out split, which yields a U-shaped criterion with a well-defined minimum. Having selected $K^*$ and fitted the full-data partition, we form two complementary predictive objects: the Venn prediction band and a conformal prediction set based on CRPS as the nonconformity score, which carries a finite-sample marginal coverage guarantee at any prescribed level $\varepsilon$. On real benchmarks against split-conformal competitors (Gaussian split conformal, CQR, and CQR-QRF), the method produces substantially narrower prediction intervals while maintaining near-nominal coverage.

💡 Research Summary

The paper introduces a fully non‑parametric method for estimating conditional distributions in regression by partitioning a one‑dimensional covariate into contiguous bins and using the empirical cumulative distribution function (ECDF) of the responses within each bin as the predictive distribution. The central contribution is the alignment of the binning criterion with the predictive objective through the Continuous Ranked Probability Score (CRPS), a proper scoring rule that simultaneously rewards calibration (closeness of the forecast to the observation) and sharpness (concentration of the forecast).

Closed‑form LOO‑CRPS cost.
For a bin containing m observations with responses (y_1,\dots,y_m), the total leave‑one‑out CRPS can be expressed as
\