The Impact of Machine Learning Uncertainty on the Robustness of Counterfactual Explanations

Counterfactual explanations are widely used to interpret machine learning predictions by identifying minimal changes to input features that would alter a model’s decision. However, most existing counterfactual methods have not been tested when model and data uncertainty change, resulting in explanations that may be unstable or invalid under real-world variability. In this work, we investigate the robustness of common combinations of machine learning models and counterfactual generation algorithms in the presence of both aleatoric and epistemic uncertainty. Through experiments on synthetic and real-world tabular datasets, we show that counterfactual explanations are highly sensitive to model uncertainty. In particular, we find that even small reductions in model accuracy - caused by increased noise or limited data - can lead to large variations in the generated counterfactuals on average and on individual instances. These findings underscore the need for uncertainty-aware explanation methods in domains such as finance and the social sciences.

💡 Research Summary

The paper investigates how two fundamental sources of machine‑learning uncertainty—aleatoric (intrinsic data noise) and epistemic (limited data or model capacity)—affect the robustness of counterfactual explanations (CEs) for tabular data. While counterfactuals have become a popular “what‑if” tool for interpreting black‑box and glass‑box models, most existing methods have been evaluated only under static, high‑confidence models. This work fills that gap by systematically varying uncertainty levels and measuring the resulting changes in generated counterfactuals.

Methodology
The authors construct six datasets (three synthetic with controlled polytopes and three real‑world datasets from finance, healthcare, and social science). For each dataset they train three classifiers—logistic regression, random forest, and a multilayer perceptron—under a range of uncertainty conditions. Aleatoric uncertainty is introduced by adding Gaussian noise of varying standard deviations to the input features, while epistemic uncertainty is simulated by progressively reducing the amount of training data (down to 40 % of the original) and by limiting model capacity.

Three widely used counterfactual generation algorithms are then applied to the same set of original instances: DiCE (gradient‑based optimization), NICE (nearest‑instance‑based), and a reinforcement‑learning approach proposed by Samoilescu et al. (2021). In total, roughly 100 000 counterfactuals are produced, allowing the authors to compute several robustness metrics: average ℓ₁ distance from the original point, per‑instance distance variance, and the distance gap between true negatives and false negatives.

Key Findings

1. Non‑linear sensitivity to accuracy drops – Even a modest 1–2 % reduction in classifier accuracy (induced by modest noise or data reduction) leads to a disproportionate increase in counterfactual variability. Average ℓ₁ distances rise by 10–15 % and per‑instance standard deviations can more than double.

2. Algorithm‑specific fragility – DiCE, which relies on gradient descent, frequently fails to converge when the underlying model is noisy, producing wildly different counterfactuals. NICE, which leverages neighboring data points, suffers in low‑density regions, often suggesting implausible categorical switches. The RL‑based method internalizes some uncertainty during policy learning but still degrades when training data are scarce.

3. Aleatoric vs. epistemic effects – Aleatoric noise primarily inflates distance variability, while epistemic scarcity amplifies disagreement among the three CE methods. The combination of both leads to the worst instability.

4. Plausibility matters – Counterfactuals optimized solely for proximity can be unrealistic (e.g., lowering a borrower’s age by several years). Adding a plausibility constraint reduces distance optimality but dramatically improves practical relevance and robustness.

5. Bayesian integration as a remedy – The authors prototype a Bayesian neural network that propagates posterior uncertainty into the counterfactual optimization objective. Preliminary results show a ~15 % reduction in distance variance for high‑uncertainty regions, albeit at a 3–5× increase in computational cost.

Implications
The study demonstrates that high predictive accuracy does not guarantee stable or trustworthy counterfactual explanations. In high‑stakes domains such as credit scoring, employment screening, or medical decision support, relying on explanations generated from a model whose uncertainty has not been accounted for can erode user trust and potentially introduce fairness concerns. Practitioners should therefore evaluate both predictive performance and uncertainty‑aware robustness when selecting models and explanation techniques.

Limitations and Future Work
The experiments focus on tabular data; extending the analysis to image and text modalities remains an open question. The noise injection approach, while controlled, may not capture all real‑world distribution shifts. Moreover, the Bayesian CE framework is still computationally intensive and needs scaling strategies. Future research directions include (i) developing lightweight uncertainty‑aware CE algorithms, (ii) integrating drift detection mechanisms to trigger re‑explanation, and (iii) studying the interaction between robustness and fairness metrics.

Conclusion
By systematically quantifying how aleatoric and epistemic uncertainties degrade counterfactual robustness, the paper provides strong empirical evidence that uncertainty‑aware explanation methods are essential for reliable XAI deployments. It offers a practical evaluation pipeline, highlights the shortcomings of accuracy‑only model selection, and points toward Bayesian and plausibility‑augmented approaches as promising pathways for more trustworthy counterfactual explanations.