On Properties of the Minimum Entropy Sub-tree to Compute Lower Bounds on the Partition Function

Computing the partition function and the marginals of a global probability distribution are two important issues in any probabilistic inference problem. In a previous work, we presented sub-tree based upper and lower bounds on the partition function of a given probabilistic inference problem. Using the entropies of the sub-trees we proved an inequality that compares the lower bounds obtained from different sub-trees. In this paper we investigate the properties of one specific lower bound, namely the lower bound computed by the minimum entropy sub-tree. We also investigate the relationship between the minimum entropy sub-tree and the sub-tree that gives the best lower bound.

💡 Research Summary

The paper investigates how to obtain reliable lower bounds on the partition function of a probabilistic graphical model by exploiting sub‑trees of the underlying junction graph. Starting from the factorized representation of a joint distribution p(x)=Z⁻¹∏_{R∈𝓡}α_R(x_R), the authors consider any sub‑set of factors that admits a junction‑tree representation (a “sub‑tree”). For a chosen sub‑tree T they define the induced distribution q_T(x) and its own partition constant Z_T. By taking logarithms of the factorization, weighting by q_T, and rearranging, they derive the identity

‑D(q_T‖p)=ln(Z_T Z)+E_{q_T}