Directed information and Pearls causal calculus

Probabilistic graphical models are a fundamental tool in statistics, machine learning, signal processing, and control. When such a model is defined on a directed acyclic graph (DAG), one can assign a partial ordering to the events occurring in the corresponding stochastic system. Based on the work of Judea Pearl and others, these DAG-based “causal factorizations” of joint probability measures have been used for characterization and inference of functional dependencies (causal links). This mostly expository paper focuses on several connections between Pearl’s formalism (and in particular his notion of “intervention”) and information-theoretic notions of causality and feedback (such as causal conditioning, directed stochastic kernels, and directed information). As an application, we show how conditional directed information can be used to develop an information-theoretic version of Pearl’s “back-door” criterion for identifiability of causal effects from passive observations. This suggests that the back-door criterion can be thought of as a causal analog of statistical sufficiency.

💡 Research Summary

The paper “Directed Information and Pearl’s Causal Calculus” establishes a rigorous bridge between Pearl’s intervention‑based causal framework on directed acyclic graphs (DAGs) and information‑theoretic notions of directed information, causal conditioning, and directed stochastic kernels. The authors begin by emphasizing that traditional information measures such as mutual information, conditional entropy, or KL‑divergence capture only statistical dependence and lack any intrinsic directionality. Consequently, they cannot distinguish cause from effect. To address this limitation, the paper revisits Massey’s directed information, later extended by Kramer, Tatikonda, and Mitter, which inherently respects temporal ordering and causal direction.

A concrete communication‑system example illustrates the difference between statistical and causal dependence. A message (W) is encoded, transmitted through a stochastic channel, and decoded into (\tilde W). Although the joint distribution of ((W,\tilde W)) can be factored in either order, only by “hard‑assigning’’ (intervening on) (W) does one affect downstream variables (the channel input, output, and decoded message). Conversely, fixing (\tilde W) leaves upstream variables untouched, revealing the asymmetry that pure statistical measures miss. This example motivates Pearl’s notion of an intervention as a surgical modification of the underlying structural equations.

The authors then formalize a class of Markovian dynamical systems described by sequential equations \