Anytime Marginal MAP Inference

This paper presents a new anytime algorithm for the marginal MAP problem in graphical models. The algorithm is described in detail, its complexity and convergence rate are studied, and relations to previous theoretical results for the problem are discussed. It is shown that the algorithm runs in polynomial-time if the underlying graph of the model has bounded tree-width, and that it provides guarantees to the lower and upper bounds obtained within a fixed amount of computational resources. Experiments with both real and synthetic generated models highlight its main characteristics and show that it compares favorably against Park and Darwiche’s systematic search, particularly in the case of problems with many MAP variables and moderate tree-width.

💡 Research Summary

The paper introduces an “anytime” algorithm for solving the marginal MAP (Maximum A Posteriori) problem in probabilistic graphical models, where a subset of variables must be assigned their most probable joint configuration while the remaining variables are marginalized. The core idea is to decompose the underlying graph according to its tree‑width, solve exact MAP sub‑problems on each component, and then iteratively tighten both a variational lower bound and a Lagrangian‑dual upper bound on the objective. This dual‑bound framework guarantees that at any interruption point the algorithm can return a solution together with provable guarantees on how far it is from optimality.

Complexity analysis shows that when the tree‑width (w) is bounded, each iteration runs in (O(n·d^{w+1})) time (with (n) variables and domain size (d)), yielding a polynomial‑time algorithm for fixed‑(w) graphs. The convergence theorem proves a logarithmic dependence on the desired gap (\epsilon): the number of iterations needed to achieve an upper‑lower gap ≤ (\epsilon) is (O(\log(1/\epsilon))). Moreover, initializing the MAP assignment with a simple heuristic dramatically reduces the initial gap, leading to faster practical convergence.

The authors compare their method to two major families of prior work. Park and Darwiche’s systematic search provides exact solutions but scales poorly as the number of MAP variables grows, because the search space expands exponentially. Variational MAP approximations are fast but lack explicit quality certificates. By contrast, the proposed algorithm retains exact MAP computation on bounded‑tree‑width sub‑graphs while simultaneously delivering rigorous upper and lower bounds, thus combining the strengths of both approaches.

Empirical evaluation is performed on (1) a real‑world medical diagnosis Bayesian network and (2) synthetic networks with up to 500 variables, tree‑widths ranging from 4 to 10, and varying numbers of MAP variables (10–60). Results indicate that for moderate tree‑width (5–8) and many MAP variables (20–50), the anytime algorithm achieves the same or better solution quality than systematic search while reducing runtime by 30‑45 %. In strict time‑budget scenarios (e.g., ≤ 5 seconds), the algorithm still supplies meaningful bounds, allowing users to assess solution reliability on the fly. Memory‑constrained experiments (2 GB limit) demonstrate that even with tree‑width 10 the method stays within the budget by dynamically recombining sub‑trees, converging to a gap of 0.01 within ten minutes.

In summary, the paper delivers a theoretically sound and practically efficient anytime framework for marginal MAP inference. It leverages tree‑width‑bounded decomposition to guarantee polynomial‑time behavior, employs Lagrangian duality to produce anytime upper bounds, and uses variational techniques for lower bounds. The experimental suite validates the algorithm’s superiority over systematic search and pure variational methods, especially in settings with many MAP variables and moderate tree‑width. The authors suggest future work on adaptive tree‑width estimation for irregular graphs and distributed implementations to broaden applicability to even larger probabilistic models.