ElliCE: Efficient and Provably Robust Algorithmic Recourse via the Rashomon Sets

Machine learning models now influence decisions that directly affect people’s lives, making it important to understand not only their predictions, but also how individuals could act to obtain better results. Algorithmic recourse provides actionable input modifications to achieve more favorable outcomes, typically relying on counterfactual explanations to suggest such changes. However, when the Rashomon set - the set of near-optimal models - is large, standard counterfactual explanations can become unreliable, as a recourse action valid for one model may fail under another. We introduce ElliCE, a novel framework for robust algorithmic recourse that optimizes counterfactuals over an ellipsoidal approximation of the Rashomon set. The resulting explanations are provably valid over this ellipsoid, with theoretical guarantees on uniqueness, stability, and alignment with key feature directions. Empirically, ElliCE generates counterfactuals that are not only more robust but also more flexible, adapting to user-specified feature constraints while being substantially faster than existing baselines. This provides a principled and practical solution for reliable recourse under model uncertainty, ensuring stable recommendations for users even as models evolve.

💡 Research Summary

The paper tackles a critical gap in algorithmic recourse: most counterfactual generation methods assume a fixed, perfectly known predictive model, yet in high‑stakes domains (credit, healthcare, insurance) models are routinely retrained or replaced. When the Rashomon set—the collection of near‑optimal models— is large, a counterfactual that works for one model may fail for another, undermining trust and usability.
To address this, the authors propose ElliCE, a framework that builds robustness directly on the geometry of the Rashomon set. They approximate the ε‑Rashomon set in parameter space by an ellipsoid defined by the Hessian of the regularized loss at the empirical risk minimizer (ERM). For convex losses such as logistic regression, the Hessian H = XᵀWX + λI is positive‑definite, guaranteeing a bounded, invertible ellipsoid. Even for neural networks where the Hessian may be singular, a small stabilization term αI is added to ensure invertibility.
The key technical contribution is a closed‑form expression for the worst‑case model within this ellipsoid. Theorem 1 shows that for any candidate counterfactual x_c, the minimal prediction over all θ in the ellipsoid equals ˆθᵀx_c − √(2ε)·‖x_c‖{H^{-1}}. Consequently, a counterfactual is robust if and only if this quantity exceeds the target score t. Substituting this into the original min‑distance objective yields a quadratically constrained quadratic program (QCQP):
 min ‖x_c − x₀‖² subject to ˆθᵀx_c − √(2ε)·‖x_c‖{H^{-1}} ≥ t.
QCQPs are convex and can be solved efficiently; the authors employ a gradient‑based solver with Lagrangian relaxation, achieving millisecond‑scale runtimes.
Four major contributions are highlighted:

1. Theoretical foundation – By leveraging the Hessian‑based ellipsoidal approximation, ElliCE provides provable guarantees of validity, uniqueness, and stability for the generated counterfactuals across the entire approximated Rashomon set.
2. Geometric intuition – The robustness term aligns with the principal curvature directions of the loss landscape, naturally emphasizing features that have the greatest influence on model predictions.
3. Actionability – The framework supports ℓ₂ distance as well as a mixed distance that handles continuous and categorical features, and it can incorporate sparsity, immutability, and range constraints, allowing practitioners to tailor recourse to domain‑specific feasibility requirements.
4. Empirical validation – Across several tabular datasets (e.g., credit scoring, medical diagnosis, insurance claims), ElliCE consistently outperforms baselines that rely on local parameter perturbations, MILP‑based robust optimization, or ensemble agreement. It achieves higher robustness rates while being 10–1000× faster, and it maintains comparable proximity and plausibility of the counterfactuals.
   Experiments vary the Rashomon tolerance ε to test sensitivity to the size of the ellipsoid. As ε grows, baseline methods see a sharp drop in success rate, whereas ElliCE automatically adjusts the counterfactual magnitude to satisfy the robust constraint. User‑specified feature weights demonstrate that the method can enforce realistic recourse policies such as “only income may change” or “age must remain unchanged.”
   The paper positions the Rashomon effect not as a flaw to be eliminated but as an intrinsic source of model uncertainty that can be mathematically characterized and mitigated. By exploiting the Hessian‑derived ellipsoidal geometry, ElliCE offers a principled, scalable, and practically useful solution for delivering stable algorithmic recourse even as models evolve. This contribution opens a new direction for robust explainability that directly accounts for model multiplicity.