Variational Probabilistic Inference and the QMR-DT Network

We describe a variational approximation method for efficient inference in large-scale probabilistic models. Variational methods are deterministic procedures that provide approximations to marginal and conditional probabilities of interest. They provide alternatives to approximate inference methods based on stochastic sampling or search. We describe a variational approach to the problem of diagnostic inference in the `Quick Medical Reference’ (QMR) network. The QMR network is a large-scale probabilistic graphical model built on statistical and expert knowledge. Exact probabilistic inference is infeasible in this model for all but a small set of cases. We evaluate our variational inference algorithm on a large set of diagnostic test cases, comparing the algorithm to a state-of-the-art stochastic sampling method.

💡 Research Summary

The paper “Variational Probabilistic Inference and the QMR‑DT Network” tackles the problem of performing diagnostic inference in the Quick Medical Reference Decision‑Theoretic (QMR‑DT) model, a massive bipartite Bayesian network that encodes roughly 600 diseases and 4,000 clinical findings. The network uses noisy‑OR gates to model the causal influence of diseases on findings, assumes marginal independence among diseases, and conditional independence of findings given the disease states. While these structural assumptions allow a compact factorization of the joint distribution, exact inference quickly becomes intractable because the size of the maximal clique in the moralized graph is on the order of 150 nodes, leading to exponential time and space requirements. Existing exact algorithms such as Quickscore, despite exploiting the noisy‑OR form, still require exponential time in the number of positive findings and would take decades on contemporary hardware for typical Clinical Pathology Conference (CPC) cases.

The authors review several families of approximate inference techniques that have been applied to QMR‑DT: (1) partial evaluation methods (e.g., bounded conditioning, cutset conditioning), which attempt to sum over a subset of variable instantiations but still face astronomically large cut‑set spaces; (2) search‑based methods (incremental SPI, mini‑buckets) that explore high‑probability hypotheses but struggle when the number of positive findings is large; and (3) stochastic sampling methods, notably likelihood‑weighted sampling (LWS) introduced by Shwe and Cooper, which can produce reasonable estimates for a few difficult cases but suffer from high variance and long runtimes on the full CPC corpus.

To overcome these limitations, the paper introduces a deterministic variational approximation tailored to the QMR‑DT’s noisy‑OR structure. The key idea is to replace intractable summations with analytically tractable bounds derived from large‑number‑of‑terms averaging principles. The noisy‑OR parameters (q_{ij}=P(f_i=1|d_j=1)) are re‑parameterized as (\theta_{ij}=-\log(1-q_{ij})). For each positive finding (f_i), the log‑likelihood (\log P(f_i=1|d)) is bounded above and below using Jensen’s inequality and a first‑order Taylor (or Laplace) approximation, introducing a variational parameter (\lambda_i) that controls the tightness of the bound. The overall variational objective becomes

\