A roadmap for systematic identification and analysis of multiple biases in causal inference

Observational studies examining causal effects rely on unverifiable assumptions, the violation of which can induce multiple biases. Quantitative bias analysis (QBA) methods examine the sensitivity of findings to such violations, generally, by producing estimates under alternative assumptions, incorporating external information. Although substantial guidance exists for implementing QBA, there is limited guidance on how to systematically determine the assumptions underlying a primary causal analysis and the potential violations that should guide bias analysis. Consequently, many assumptions remain implicit, leading to selective and therefore misleading QBA. To address this gap, we propose a roadmap for systematically identifying and analysing multiple biases. Briefly, this consists of (1) articulating the assumptions underlying the primary analysis through specification and emulation of the ideal trial that defines the causal estimand and depicting these assumptions using a causal diagram; (2) extending the diagram to depict alternative assumptions under which biases may arise; (3) obtaining a single estimate that simultaneously corrects for all potential biases. We illustrate the roadmap using an investigation of the effect of breastfeeding on risk of childhood asthma, and through simulations illustrate the need for analysing multiple biases jointly rather than one at a time.

💡 Research Summary

Observational studies rely on unverifiable assumptions—exchangeability, consistency, and positivity—to identify causal effects. When these assumptions are violated, systematic biases such as confounding, measurement, and selection bias arise, potentially distorting the estimated average causal effect (ACE). Existing quantitative bias analysis (QBA) guidance focuses on implementing bias corrections but offers limited direction on how to systematically uncover the underlying assumptions and the plausible ways they might be breached. Consequently, many analyses treat assumptions implicitly, leading to selective bias assessment, and when multiple biases are considered, they are often examined one at a time, ignoring possible interactions.

This paper proposes a three‑step roadmap for systematic identification and simultaneous correction of multiple biases.
Step 1 – Define the “ideal trial” and enumerate primary assumptions. The ideal trial is a hypothetical randomized experiment with an infinite, perfectly representative sample, flawless adherence, no missing data, and error‑free measurement. By specifying its protocol (eligibility, treatment, assignment, follow‑up, outcome) the researcher can translate the target causal estimand (here the risk ratio of any breastfeeding versus none on childhood asthma) into a set of explicit assumptions. These assumptions are then visualized in a directed acyclic graph (DAG) that captures the causal structure assumed in the primary analysis.

Step 2 – Extend the DAG to represent plausible violations. The researcher systematically asks how each assumption could be broken: unmeasured confounders, differential measurement error, selection mechanisms linked to consent, language, or loss‑to‑follow‑up, etc. Each potential source of bias is added to an “alternative” DAG, highlighting non‑causal pathways (open back‑doors, colliders) that would introduce bias. The paper categorizes biases into three types—confounding, measurement, selection—and shows how each appears graphically.

Step 3 – Conduct a quantitative multiple‑bias analysis. Rather than performing separate QBA for each bias, the authors advocate a joint modelling approach. Using a Bayesian framework, separate sub‑models are specified for measurement error (e.g., misclassification probabilities), unmeasured confounding (distribution of the hidden variable and its effects), and selection bias (probability of inclusion or missingness conditional on observed and unobserved factors). External information (literature‑based priors, validation studies) informs the priors. All sub‑models are combined, and posterior inference yields a single bias‑adjusted estimate of the ACE together with a credible interval that reflects uncertainty from all sources simultaneously.

The roadmap is illustrated with a case study from the HealthNuts cohort examining whether any breastfeeding in the first year reduces asthma risk at age six. The primary analysis used parental reports for both exposure and outcome, adjusted for a set of measured confounders, and assumed no missing data bias. The authors identify three concrete violations: (1) misclassification of breastfeeding and asthma; (2) an unmeasured confounder—gestational hypertension; (3) selection bias due to consent and English‑language requirements, potentially interacting with eczema status (an effect‑modifier). These are incorporated into an expanded DAG.

Simulation studies based on the case‑study structure compare three strategies: (i) naïve analysis ignoring bias; (ii) separate one‑at‑a‑time QBA for each bias; and (iii) the proposed simultaneous multiple‑bias model. Results show that separate corrections can either over‑ or under‑adjust, depending on the direction and correlation of biases, whereas the joint model recovers the true ACE more accurately and yields wider, more realistic uncertainty intervals. Notably, when measurement error and unmeasured confounding are positively correlated, the one‑by‑one approach underestimates bias, while the joint model captures the compounded effect.

The authors provide reproducible code (R/Stan) and a public GitHub repository, facilitating adoption. They argue that the roadmap improves transparency (by forcing explicit articulation of assumptions), completeness (by systematically scanning for plausible violations), and validity (by jointly correcting for interacting biases). While the approach demands more modelling effort and external information, it offers a principled path for rigorous causal inference in observational research where multiple, interdependent biases are the norm.