A Non-Parametric Bayesian Method for Inferring Hidden Causes

We present a non-parametric Bayesian approach to structure learning with hidden causes. Previous Bayesian treatments of this problem define a prior over the number of hidden causes and use algorithms such as reversible jump Markov chain Monte Carlo to move between solutions. In contrast, we assume that the number of hidden causes is unbounded, but only a finite number influence observable variables. This makes it possible to use a Gibbs sampler to approximate the distribution over causal structures. We evaluate the performance of both approaches in discovering hidden causes in simulated data, and use our non-parametric approach to discover hidden causes in a real medical dataset.

💡 Research Summary

The paper introduces a non‑parametric Bayesian framework for learning causal structures that involve hidden causes. Instead of fixing the number of latent factors and employing reversible‑jump Markov chain Monte Carlo (RJMCMC) to traverse models of different dimensionality, the authors assume an unbounded pool of possible causes while guaranteeing that only a finite subset actually influences the observed variables. This assumption is encoded through an Indian Buffet Process (IBP) prior over the binary cause‑to‑observable adjacency matrix. Because the IBP naturally generates a sparse, finite set of active causes for any finite dataset, the model’s dimensionality can change implicitly during inference, eliminating the need for explicit birth‑death proposals.

Inference is performed with a Gibbs sampler that iteratively updates each entry of the adjacency matrix conditioned on the current state of all other entries, the observed data, and the IBP hyper‑parameter α. New causes are introduced with probability proportional to α and the likelihood improvement they provide; unused causes are automatically pruned. The likelihood is modeled with a logistic link for binary observations, though the authors note that Gaussian or other exponential‑family forms can be substituted without altering the core algorithm. α itself is treated hierarchically and sampled from its posterior, allowing the model to adapt the expected number of active causes to the data.

The authors evaluate the method on two fronts. First, synthetic datasets with varying numbers of hidden causes, observed variables, and sample sizes are generated. The non‑parametric Gibbs sampler consistently matches or exceeds the structure‑recovery accuracy of RJMCMC while converging in far fewer iterations—often three to five times faster. Second, a real‑world medical dataset containing patient symptoms and diagnostic outcomes is analyzed. The discovered latent causes correspond closely to clinically recognized risk factors, demonstrating that the approach can uncover meaningful, interpretable structure in noisy, high‑dimensional domains.

Key contributions include: (1) a principled non‑parametric prior that removes the need for reversible‑jump moves, dramatically simplifying implementation; (2) an efficient Gibbs‑sampling inference scheme that scales to realistic problem sizes; (3) empirical evidence that the method outperforms traditional RJMCMC in both accuracy and computational cost; and (4) a compelling application to medical data that validates the practical utility of the approach. The paper concludes by suggesting extensions to continuous observations, temporal dynamics, and multimodal data, indicating a broad horizon for future research in non‑parametric hidden‑cause discovery.