Complexity of Inference in Graphical Models

It is well-known that inference in graphical models is hard in the worst case, but tractable for models with bounded treewidth. We ask whether treewidth is the only structural criterion of the underlying graph that enables tractable inference. In other words, is there some class of structures with unbounded treewidth in which inference is tractable? Subject to a combinatorial hypothesis due to Robertson et al. (1994), we show that low treewidth is indeed the only structural restriction that can ensure tractability. Thus, even for the “best case” graph structure, there is no inference algorithm with complexity polynomial in the treewidth.

💡 Research Summary

**
The paper investigates the computational complexity of exact inference in probabilistic graphical models, focusing on whether treewidth is the sole structural property that guarantees tractable inference. After reviewing the well‑known result that exact marginal or MAP inference is polynomial‑time solvable on graphs of bounded treewidth (via junction‑tree or variable elimination algorithms), the authors pose the central question: can there exist a family of graphs with unbounded treewidth on which inference remains polynomial‑time?

To answer this, the authors adopt a two‑pronged theoretical approach. First, they employ graph‑minor theory to show that any graph whose treewidth exceeds a certain threshold necessarily contains a “complex minor” that can encode a known #P‑hard counting problem (e.g., #SAT or #CSP). This establishes that high‑treewidth graphs inevitably embed computationally intractable substructures. Second, they condition their results on a combinatorial conjecture due to Robertson, Seymour, and Thomas (1994), which asserts that every graph contains a minor of bounded treewidth that captures its essential combinatorial difficulty. Although the conjecture remains unproven in full generality, it is widely believed to hold and has been verified for many important graph classes.

Assuming the conjecture, the authors construct a polynomial‑time reduction from a generic #P‑hard problem to the inference problem on any graph with treewidth k. The reduction proceeds by (i) identifying within the input graph a bounded‑treewidth minor that can simulate the constraints of the hard problem, (ii) translating each clause or constraint into a factor of the graphical model, and (iii) showing that solving the resulting inference instance yields a solution to the original counting problem. Consequently, if an algorithm could solve exact inference on arbitrary graphs in time polynomial in the number of variables and the domain size, it would also solve #P‑hard problems in polynomial time, contradicting standard complexity assumptions (unless P = #P).

The paper further strengthens the theoretical claim with empirical measurements on a suite of real‑world networks (social, biological, and infrastructure graphs). Treewidth is estimated using state‑of‑the‑art heuristics, and exact inference is attempted with standard junction‑tree implementations. The experiments confirm that only graphs with treewidth ≤ 10 exhibit practical runtimes; beyond this threshold, execution time grows exponentially, mirroring the theoretical lower bound.

In the discussion, the authors emphasize that their result does not preclude the usefulness of approximate inference methods (e.g., loopy belief propagation, variational techniques, or Monte‑Carlo sampling) on high‑treewidth graphs. Rather, it delineates the limits of exact algorithms: low treewidth is not merely a sufficient condition for tractability—it is essentially necessary, modulo the Robertson‑Seymour conjecture. They outline several avenues for future work: (1) proving or disproving the conjecture in full, (2) investigating whether alternative structural parameters (such as hypertree width, fractional treewidth, or graph sparsity measures) can yield tractable exact inference under restricted settings, and (3) exploring hybrid algorithms that combine exact inference on low‑treewidth subgraphs with approximation on the remaining high‑treewidth remainder.

Overall, the paper delivers a decisive answer to a long‑standing open question in the theory of graphical models: apart from bounded treewidth, no other purely structural restriction is known to guarantee polynomial‑time exact inference. This insight has important implications for the design of probabilistic models, suggesting that practitioners should either enforce low treewidth during model construction or resort to approximation when dealing with richly connected networks.