Stress-Testing Causal Claims via Cardinality Repairs

Causal analyses derived from observational data underpin high-stakes decisions in domains such as healthcare, public policy, and economics. Yet such conclusions can be surprisingly fragile: even minor data errors - duplicate records, or entry mistakes - may drastically alter causal relationships. This raises a fundamental question: how robust is a causal claim to small, targeted modifications in the data? Addressing this question is essential for ensuring the reliability, interpretability, and reproducibility of empirical findings. We introduce SubCure, a framework for robustness auditing via cardinality repairs. Given a causal query and a user-specified target range for the estimated effect, SubCure identifies a small set of tuples or subpopulations whose removal shifts the estimate into the desired range. This process not only quantifies the sensitivity of causal conclusions but also pinpoints the specific regions of the data that drive those conclusions. We formalize this problem under both tuple- and pattern-level deletion settings and show both are NP-complete. To scale to large datasets, we develop efficient algorithms that incorporate machine unlearning techniques to incrementally update causal estimates without retraining from scratch. We evaluate SubCure across four real-world datasets covering diverse application domains. In each case, it uncovers compact, high-impact subsets whose removal significantly shifts the causal conclusions, revealing vulnerabilities that traditional methods fail to detect. Our results demonstrate that cardinality repair is a powerful and general-purpose tool for stress-testing causal analyses and guarding against misleading claims rooted in ordinary data imperfections.

💡 Research Summary

The paper introduces SubCure, a novel framework for stress‑testing causal claims derived from observational data by means of “cardinality repairs.” The central question is how many records (or which subpopulations) must be removed from a dataset to shift an estimated causal effect—typically the Average Treatment Effect (ATE)—into a user‑specified target interval. This problem, dubbed CaRET (Cardinality Repair for causal Effect Targeting), is formalized under two cost models: (i) tuple‑level deletions, where individual rows are removed, and (ii) pattern‑level deletions, where entire subpopulations defined by attribute‑value predicates are eliminated. The authors prove that both variants are NP‑complete; the tuple case reduces from Subset‑Sum, while the pattern case is shown to be strictly harder regardless of the causal estimator used.

To make CaRET tractable on real‑world data, SubCure employs two complementary, scalable search strategies. In tuple mode, the dataset is first clustered with a two‑stage k‑means procedure to obtain a representative proxy set. Each sampled tuple is assigned a marginal “influence score” estimating how much its removal would change the ATE. The algorithm iteratively deletes the tuple with the highest score, refreshing influence scores only periodically to avoid unnecessary recomputation. In pattern mode, SubCure conducts bottom‑up random walks over conjunctions of attribute‑value predicates. A dynamic weighting scheme steers the walk toward predicates that have shown high impact, and the walk aborts early if a candidate subgroup grows beyond a preset size. Both strategies are “anytime”: they can be stopped at any point and still return a high‑quality repair.

A key technical contribution is the use of machine‑unlearning techniques to update causal effect estimates incrementally after each deletion, eliminating the need to retrain models from scratch. For linear regression, sufficient statistics such as the covariance matrix and response cross‑product are cached; low‑rank updates are applied when rows are removed, yielding O(p²) updates where p is the number of covariates. For inverse‑propensity weighting (IPW), the previous logistic regression fit is warm‑started and a single Fisher‑scoring step adjusts the propensity scores, providing constant‑time updates per deletion. These incremental updates preserve estimator fidelity while dramatically reducing runtime, enabling interactive exploration even on million‑row, high‑dimensional datasets.

The empirical evaluation spans four public datasets—Twins (birth outcomes), the American Community Survey (disability and wages), and two additional medical/economic datasets—as well as a synthetic benchmark. SubCure consistently identifies smaller repair sets than eight baseline methods, often reducing the required deletions to under 1 % of the data while achieving the desired effect shift. In the Twins example, deleting just 270 records (≈1.1 % of the sample) moves the ATE from –0.016 to –0.0009, a 94 % reduction toward zero; a pattern‑level repair targeting twins weighing 1786–1999 g (≈18 % of the data) flips the sign of the ATE entirely. In the ACS wage study, removing 0.7 % of records raises the ATE by 31 %, and a subpopulation comprising 11.9 % of the data suffices to push the estimate into a tighter target range, while the opposite direction requires removing nearly three‑quarters of the data, revealing a pronounced asymmetry. Across all experiments, the incremental unlearning tricks yield order‑of‑magnitude speed‑ups (often 5–20× faster) without sacrificing accuracy.

Beyond the quantitative results, the case studies illustrate how SubCure surfaces domain‑meaningful vulnerabilities: low‑gestational‑age births in the Twins data, low‑income disabled workers in the wage analysis, and other medically or economically salient subgroups. By pinpointing the exact records or predicates that drive a causal conclusion, analysts can decide whether to collect additional data, re‑weight observations, or refine the causal model before drawing policy or clinical recommendations.

In summary, the paper makes four major contributions: (1) a formal definition and hardness proof for cardinality‑repair‑based causal effect targeting; (2) efficient, incremental search algorithms for both tuple‑ and pattern‑level repairs; (3) integration of machine‑unlearning techniques to enable fast, exact updates of linear regression and IPW estimators after deletions; and (4) a thorough empirical validation showing that SubCure uncovers small, high‑impact repairs that traditional sensitivity analyses miss. The work bridges the gap between model‑centric robustness (hidden confounding, selection bias) and data‑centric robustness (duplicate records, mis‑entries), offering a practical tool for researchers and policymakers to audit and strengthen the reliability of observational causal claims.