Causal models have no complete axiomatic characterization

Markov networks and Bayesian networks are effective graphic representations of the dependencies embedded in probabilistic models. It is well known that independencies captured by Markov networks (called graph-isomorphs) have a finite axiomatic characterization. This paper, however, shows that independencies captured by Bayesian networks (called causal models) have no axiomatization by using even countably many Horn or disjunctive clauses. This is because a sub-independency model of a causal model may be not causal, while graph-isomorphs are closed under sub-models.

💡 Research Summary

The paper investigates the axiomatic characterizability of conditional independence (CI) structures that arise from two popular graphical formalisms: undirected Markov networks (graph‑isomorphs) and directed acyclic graphs (DAGs) underlying Bayesian networks, often called causal models. While it is well‑known that CI relations captured by undirected graphs admit a finite axiomatic basis (the semi‑graphoid axioms) and consequently can be described by a finite or countable set of Horn or disjunctive clauses, the authors prove that no such complete axiomatization exists for causal models.

The authors first formalize a propositional fragment of first‑order logic called independence logic (I L), whose atoms are statements of the form I(X, Z, Y) meaning “X is independent of Y given Z”. They define Horn clauses (at most one positive literal) and disjunctive clauses (more than one positive literal) and show that any axiomatization can be reduced to a set of such clauses. A family of independence models is said to be completely characterized by a set of formulas if a model belongs to the family exactly when it satisfies every formula in the set.

The paper then examines the “heredity” property: whether a class of models is closed under taking sub‑models (restriction to a subset of variables). For graph‑isomorphs and for arbitrary CI relations, this property holds. The authors give a constructive proof that restricting an undirected graph to a subset V yields another undirected graph whose separation properties coincide with the restricted CI model. Consequently, graph‑isomorphs can be axiomatized.

The crucial contribution is the demonstration that causal models are not closed under sub‑models. The authors present a concrete DAG D (shown in Figure 1) over variables {0,1,2,3,4} and consider the variable subset V = {1,2,3,4}. The original causal model M induced by D satisfies certain d‑separation statements. When restricted to V, the resulting independence model M|V exhibits three pairs of variables that are never d‑separated (denoted D(α,β)) together with several independencies such as I(1,∅,3) and I(2,∅,4). The authors prove that no DAG on V can simultaneously realize all these constraints: the required edges would force both 2 → 3 and 2 ← 3 to appear, which contradicts the acyclicity of a DAG. Hence M|V is not a causal model.

Because any axiomatization must be preserved under taking sub‑models (if a model satisfies all axioms, any restriction must also satisfy them), the lack of closure implies that no set of Horn or disjunctive clauses—finite or countably infinite—can completely characterize the class of causal models. This settles a conjecture originally posed by Pearl and Paz and later hinted at by Studený, providing a rigorous proof.

The paper concludes by discussing the implications for causal inference. Since causal models cannot be captured by a purely logical axiom system, reasoning about conditional independencies in Bayesian networks inevitably requires reference to underlying probability distributions or algorithmic procedures (e.g., d‑separation tests). Moreover, the fact that sub‑models of causal structures may fall outside the class warns against naïve variable elimination or marginalization without checking whether the resulting independence structure remains representable by a DAG.

In summary, the work establishes a fundamental distinction between undirected graphical models and directed causal models: the former admit a complete, countable axiomatization, while the latter do not. This negative result deepens our understanding of the expressive limits of Bayesian networks and highlights the intrinsic complexity of causal reasoning.