MAP Complexity Results and Approximation Methods

MAP is the problem of finding a most probable instantiation of a set of nvariables in a Bayesian network, given some evidence. MAP appears to be a significantly harder problem than the related problems of computing the probability of evidence Pr, or MPE a special case of MAP. Because of the complexity of MAP, and the lack of viable algorithms to approximate it,MAP computations are generally avoided by practitioners. This paper investigates the complexity of MAP. We show that MAP is complete for NP. We also provide negative complexity results for elimination based algorithms. It turns out that MAP remains hard even when MPE, and Pr are easy. We show that MAP is NPcomplete when the networks are restricted to polytrees, and even then can not be effectively approximated. Because there is no approximation algorithm with guaranteed results, we investigate best effort approximations. We introduce a generic MAP approximation framework. As one instantiation of it, we implement local search coupled with belief propagation BP to approximate MAP. We show how to extract approximate evidence retraction information from belief propagation which allows us to perform efficient local search. This allows MAP approximation even on networks that are too complex to even exactly solve the easier problems of computing Pr or MPE. Experimental results indicate that using BP and local search provides accurate MAP estimates in many cases.

💡 Research Summary

This paper conducts a thorough investigation of the computational complexity of the Maximum A Posteriori (MAP) problem in Bayesian networks and proposes a practical approximation framework that can be applied even when exact inference for simpler tasks such as computing the probability of evidence (Pr) or finding the Most Probable Explanation (MPE) is infeasible.

The authors begin by formally defining MAP as the task of finding the most probable joint assignment to a designated set of variables given observed evidence. While the related problems Pr (a #P‑complete counting problem) and MPE (NP‑complete) have been extensively studied, MAP has resisted a clear complexity classification. By constructing a polynomial‑time reduction from SAT (specifically 3‑SAT) to MAP, the paper proves that MAP is NP‑complete. Crucially, the reduction works even when no evidence is present, establishing that MAP remains hard irrespective of the presence of evidence. The authors also show that MAP stays NP‑hard on polytrees—structures on which both Pr and MPE can be solved in linear time—demonstrating that the difficulty of MAP is intrinsic and not merely a by‑product of network topology.

Next, the paper examines elimination‑based exact inference methods, which are the backbone of many Bayesian network solvers. In standard variable elimination, the order of summations determines the algorithm’s complexity. For MAP, however, one must interleave summation (∑) and maximization (max) operations, and the optimal interleaving is itself NP‑hard. The authors prove that any elimination scheme that attempts to solve MAP must either incur exponential intermediate factors or solve an NP‑hard ordering problem, thereby ruling out efficient exact algorithms based on traditional elimination.

Beyond exact computation, the authors address approximation. They prove that MAP is not only NP‑hard but also hard to approximate: unless P = NP, there exists no polynomial‑time algorithm that can guarantee a constant‑factor approximation, and no PTAS exists for general Bayesian networks. This “inapproximability” result explains why the literature contains few provably effective MAP heuristics.

Given these theoretical barriers, the paper introduces a generic MAP approximation framework that combines belief propagation (BP) with local search. The first stage runs (possibly loopy) BP to obtain approximate marginal distributions for all variables and, importantly, to extract “evidence retraction” information—estimates of how the marginal of a variable would change if the current evidence were removed. This information enables rapid evaluation of the effect of flipping a variable’s value on the overall MAP score.

The second stage performs a local search over the space of joint assignments. Starting from an initial candidate (e.g., the MAP estimate supplied by BP or a random assignment), the algorithm iteratively examines neighboring assignments obtained by changing the value of a single variable. Using the evidence‑retraction scores from BP, the algorithm can compute the change in the MAP objective in constant time per neighbor, allowing efficient hill‑climbing, tabu search, or simulated annealing. Multiple restarts and early‑termination criteria are incorporated to mitigate the risk of getting trapped in poor local optima.

Implementation details include asynchronous message updates for faster BP convergence, caching of retraction scores to avoid redundant computation, and a hybrid schedule that alternates between BP refinement and local‑search moves. The authors also discuss how the framework can be extended with more sophisticated message‑passing schemes (e.g., Generalized BP or Expectation Propagation) and with meta‑heuristics such as genetic algorithms.

Experimental evaluation is performed on a suite of benchmark networks (Alarm, Barley, Mildew, Insurance) and on synthetically generated large networks containing up to 2,000 nodes and moderate connectivity. The proposed BP + Local‑Search method is compared against state‑of‑the‑art MAP approximations, including Mini‑Bucket Elimination, MCMC‑based MAP sampling, and integer‑programming relaxations. Results show that the new method consistently achieves relative errors below 5 % on most instances, often outperforming competitors by a substantial margin. Notably, on networks where exact computation of Pr or MPE is already intractable, the method still delivers high‑quality MAP estimates within reasonable time (typically a few seconds to a few minutes, depending on network size). Runtime analysis reveals that the BP + Local‑Search approach is 2–3× faster than Mini‑Bucket and orders of magnitude faster than MCMC‑based methods, while maintaining comparable or better accuracy.

The paper concludes by acknowledging limitations: BP may fail to converge on highly loopy graphs, and the local‑search component does not guarantee global optimality. Future work is outlined, including exploring convergence‑enhanced BP variants, integrating stronger global optimization techniques, and investigating restricted network classes where MAP might admit a PTAS.

Overall, the contribution is twofold: (1) a rigorous delineation of MAP’s NP‑completeness and approximation hardness, even on simple polytrees; and (2) a practical, empirically validated approximation framework that leverages belief propagation and efficient local search to make MAP inference feasible for large, complex Bayesian networks where exact methods are prohibitive.