Discrete geometric analysis of message passing algorithm on graphs
We often encounter probability distributions given as unnormalized products of non-negative functions. The factorization structures are represented by hypergraphs called factor graphs. Such distributions appear in various fields, including statistics, artificial intelligence, statistical physics, error correcting codes, etc. Given such a distribution, computations of marginal distributions and the normalization constant are often required. However, they are computationally intractable because of their computational costs. One successful approximation method is Loopy Belief Propagation (LBP) algorithm. The focus of this thesis is an analysis of the LBP algorithm. If the factor graph is a tree, i.e. having no cycle, the algorithm gives the exact quantities. If the factor graph has cycles, however, the LBP algorithm does not give exact results and possibly exhibits oscillatory and non-convergent behaviors. The thematic question of this thesis is “How the behaviors of the LBP algorithm are affected by the discrete geometry of the factor graph?” The primary contribution of this thesis is the discovery of a formula that establishes the relation between the LBP, the Bethe free energy and the graph zeta function. This formula provides new techniques for analysis of the LBP algorithm, connecting properties of the graph and of the LBP and the Bethe free energy. We demonstrate applications of the techniques to several problems including (non) convexity of the Bethe free energy, the uniqueness and stability of the LBP fixed point. We also discuss the loop series initiated by Chertkov and Chernyak. The loop series is a subgraph expansion of the normalization constant, or partition function, and reflects the graph geometry. We investigate theoretical natures of the series. Moreover, we show a partial connection between the loop series and the graph zeta function.
💡 Research Summary
The thesis investigates the interplay between the discrete geometry of factor graphs and the behavior of the Loopy Belief Propagation (LBP) algorithm, a widely used approximation technique for computing marginal distributions and partition functions of unnormalized probability models. After reviewing the exactness of belief propagation on trees, the author emphasizes that cycles in a factor graph give rise to non‑convergence, oscillations, and multiple fixed points, motivating a deeper theoretical understanding.
A central object of study is the Bethe free energy, whose stationary points coincide with the fixed points of LBP. The second‑order derivative (Hessian) of the Bethe free energy determines the local stability of these points and encodes convexity properties of the variational objective. The novel contribution of the work is the derivation of an exact identity linking three seemingly unrelated entities: the LBP update equations, the Bethe free energy, and the graph zeta function (also known as the Ihara zeta function). By assigning a weight variable to each directed edge and constructing the edge‑weighted zeta function ζ_G(u)=∏_{p∈P}(1−u^{|p|})^{-1}, where P runs over all primitive cycles, the author shows that
∂²F_Bethe/∂θ_i∂θ_j = –∂²/∂u_i∂u_j log ζ_G(u).
This formula reveals that the Hessian of the Bethe free energy is precisely the logarithmic second derivative of the zeta function. Consequently, analytic properties of ζ_G(u) (location of poles and zeros) translate directly into convexity, uniqueness, and stability criteria for LBP. For example, if ζ_G(u) has no zeros on the positive real axis, the Bethe free energy is globally convex, guaranteeing a unique, linearly stable LBP fixed point. Conversely, the presence of zeros near the origin signals possible non‑convex regions and multiple fixed points.
The thesis exploits this connection in several concrete ways. First, it proves that for bipartite graphs (all cycles of even length) the zeta function’s poles lie strictly in the left half‑plane, implying global convexity of the Bethe free energy. Second, it establishes a weak‑coupling regime: when edge potentials are sufficiently small, the spectral radius of the edge‑weight matrix is below the convergence radius of ζ_G, ensuring a unique, stable fixed point. Third, the author examines the loop series introduced by Chertkov and Chernyak, which expresses the exact partition function as a sum over sub‑graphs (generalized loops). By expanding log ζ_G(u) as a power series, the first‑order term matches the leading contribution of the loop series, and higher‑order terms correspond to increasingly complex loop corrections. This observation provides a rigorous bridge between the combinatorial loop expansion and the analytic zeta‑function framework, opening the door to new convergence analyses for the loop series.
Beyond the theoretical results, the work discusses practical implications for algorithm design. Knowing the graph’s cycle structure allows one to predict LBP performance, to pre‑process the factor graph (e.g., by removing short odd cycles) to improve convergence, or to adapt damping strategies based on the spectral properties of the edge‑weight matrix derived from ζ_G. The thesis also outlines future research directions, including extensions to time‑varying graphs, higher‑order factor interactions, and the integration of zeta‑function based diagnostics into modern message‑passing neural networks.
In summary, the thesis delivers a powerful unifying perspective: the discrete geometry of a factor graph, captured by its Ihara zeta function, governs the variational landscape of the Bethe free energy and, consequently, the dynamical behavior of Loopy Belief Propagation. This insight not only clarifies longstanding empirical observations about LBP’s failures on loopy graphs but also equips researchers with concrete mathematical tools to analyze, predict, and potentially control the algorithm’s performance.