Are Bayesian networks typically faithful?

Faithfulness is a common assumption in causal inference, often motivated by the fact that the faithful parameters of linear Gaussian and discrete Bayesian networks are typical, and the folklore belief that this should also hold for other classes of Bayesian networks. We address this open question by showing that among all Bayesian networks over a given DAG, the faithful Bayesian networks are indeed typical': they constitute a dense, open set with respect to the total variation metric. This does not directly imply that faithfulness is typical in restricted classes of Bayesian networks that are often considered in statistical applications. To this end we consider the class of Bayesian networks parametrised by conditional exponential families, for which we show that under regularity conditions, the faithful parameters constitute a dense and open set, the unfaithful parameters have Lebesgue measure zero, and the induced faithful distributions are open and dense in the weak topology. This extends the existing results for linear Gaussian and discrete Bayesian networks. We also show for nonparametric classes of Bayesian networks with uniformly equicontinuous and uniformly bounded conditional densities that the faithful Bayesian networks are open and dense in the weak topology. All these results also hold for Bayesian networks with latent variables, if faithfulness is only required to hold with respect to the latent projection. Finally, for the considered conditional exponential family parametrisations and nonparametric conditional density models, the topological properties of conditional independence imply the existence of a consistent conditional independence test. Together with the topological properties of faithfulness, this implies that sound constraint-based causal discovery algorithms like PC and FCI are consistent on an open and dense -- and hence typical’ – set of Bayesian networks.

💡 Research Summary

This paper addresses a fundamental question in causal inference: how typical is the faithfulness assumption for Bayesian networks? Faithfulness—requiring that every conditional independence in the observed distribution corresponds exactly to a d‑separation in the underlying DAG—underpins constraint‑based algorithms such as PC and FCI. While prior work established that for linear Gaussian and discrete Bayesian networks the set of unfaithful parameter values has Lebesgue measure zero, no comparable results existed for broader parametric families or non‑parametric models.

The authors first consider the most general setting: all Bayesian networks that are Markov with respect to a fixed DAG G. They introduce a metric d_TV on the space of joint distributions, induced by total variation distance, and prove that the set of faithful distributions is both dense and open with respect to this metric. Consequently, the complement (the unfaithful distributions) is nowhere dense, a stronger notion of atypicality than mere measure‑zero. This topological “typicality” holds for any standard Borel sample spaces, covering continuous, discrete, and mixed data.

Next, the paper narrows focus to two practically relevant subclasses.

1. Conditional Exponential Family Parameterisations – Under regularity conditions (smooth sufficient statistics, open parameter space, etc.), the authors show that if at least one faithful parameter exists, then the faithful parameters form a dense, open subset of the Euclidean parameter space, and the unfaithful parameters have Lebesgue measure zero. Moreover, the induced observational distributions are dense and open in the weak topology. This generalises the classic Spirtes‑Meek results to arbitrary conditional exponential families.

2. Non‑Parametric Models with Uniformly Equicontinuous and Bounded Conditional Densities – For this class, they prove analogous density and openness results both for the network kernels (with respect to the d∘T_V metric) and for the induced observational distributions (with respect to the weak topology). Because total variation convergence coincides with weak convergence for these uniformly bounded densities, the results are particularly strong: convergence in the chosen metric guarantees preservation of conditional independences.

The authors also extend all results to latent‑variable models, provided faithfulness is required only with respect to the latent projection of the DAG. This ensures that algorithms like FCI, which operate under latent confounding, inherit the same typicality guarantees.

A crucial bridge to statistical practice is established through recent work on consistent conditional independence testing (Genin & Kelly, Lauritzen). The paper shows that for the two subclasses above, a uniformly consistent test exists, and therefore any sound constraint‑based algorithm that relies on such a test (e.g., PC, FCI) is statistically consistent on an open and dense set of Bayesian networks. This consistency holds when the sample spaces are separable complete metric spaces.

In summary, the contributions are fourfold:

• Topological Typicality: Faithful distributions are dense and open in total variation topology for unrestricted Bayesian networks.
• Parametric Extension: The same typicality holds for any conditional exponential family, with Lebesgue‑measure‑zero unfaithful parameters.
• Non‑Parametric Extension: Uniformly equicontinuous, bounded conditional densities also enjoy dense‑open faithful sets, both in kernel and distribution spaces.
• Algorithmic Implications: Consistent conditional independence tests together with the above topological properties guarantee that PC, FCI, and similar algorithms are consistent on a “typical” (open, dense) subset of Bayesian networks, even with latent variables.

Thus, the paper rigorously demonstrates that faithfulness is not a fragile or merely convenient assumption; rather, it is a robust, generic property across a wide spectrum of Bayesian network models.