Backward Conformal Prediction

We introduce $\textit{Backward Conformal Prediction}$, a method that guarantees conformal coverage while providing flexible control over the size of prediction sets. Unlike standard conformal prediction, which fixes the coverage level and allows the conformal set size to vary, our approach defines a rule that constrains how prediction set sizes behave based on the observed data, and adapts the coverage level accordingly. Our method builds on two key foundations: (i) recent results by Gauthier et al. [2025] on post-hoc validity using e-values, which ensure marginal coverage of the form $\mathbb{P}(Y_{\rm test} \in \hat C_n^{\tildeα}(X_{\rm test})) \ge 1 - \mathbb{E}[\tildeα]$ up to a first-order Taylor approximation for any data-dependent miscoverage $\tildeα$, and (ii) a novel leave-one-out estimator $\hatα^{\rm LOO}$ of the marginal miscoverage $\mathbb{E}[\tildeα]$ based on the calibration set, ensuring that the theoretical guarantees remain computable in practice. This approach is particularly useful in applications where large prediction sets are impractical such as medical diagnosis. We provide theoretical results and empirical evidence supporting the validity of our method, demonstrating that it maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.

💡 Research Summary

Conformal prediction provides distribution‑free marginal coverage guarantees by constructing prediction sets that contain the true label with probability at least (1-\alpha) under exchangeability. A major practical limitation, however, is that the size of these sets is entirely data‑driven and can become prohibitively large, especially in multi‑class classification problems. Recent work has focused on shrinking the sets while keeping a fixed coverage level, but the fundamental trade‑off between set size and coverage remains.

The present paper flips this paradigm. Instead of fixing a miscoverage level (\alpha) and letting the set size vary, it first specifies a size‑constraint rule (T) that maps the calibration data ({(X_i,Y_i)}{i=1}^n) and a test feature (X{\text{test}}) to a maximum allowable cardinality for the prediction set. Given this rule, the method adaptively chooses a data‑dependent miscoverage level (\tilde\alpha) that is just small enough to keep the set size within the bound imposed by (T).

The technical backbone relies on conformal e‑prediction, a recent development that replaces traditional p‑value rank comparisons with e‑values (non‑negative random variables with expectation ≤ 1). Using the e‑value
\