Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ``planted'' solution; thus, we obtain the first rigorous result on BP for graph coloring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how Belief Propagation breaks the symmetry between the $3!$ possible permutations of the color classes.
Deep Dive into A Spectral Approach to Analyzing Belief Propagation for 3-Coloring.
Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ``planted’’ solution; thus, we obtain the first rigorous result on BP for graph coloring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how Belief Propagation breaks the symmetry between the $3!$ possible permutations of the color classes.
This paper deals with a rigorous analysis of the Belief Propagation ("BP" for short) algorithm on certain instances of the 3-coloring problem. Originally BP was introduced by Pearl [14] as a message passing algorithm to compute the marginals at the vertices of a probability distribution described by an acyclic "graphical model", i.e., a representation of the distribution's dependency structure as an acyclic graph. Although in the worst case BP will fail if the graphical representation features cycles, various version of BP are in common use as heuristics in artificial intelligence and statistics, where they frequently perform well empirically as long as the underlying model does at least not contain (many) "short" cycles. However, there is currently no general theory that could explain the empirical success of BP (with the notable exception of the use of BP in LDPC decoding [11,12,15]).
A striking recent application of BP is to instances of NP-hard constraint satisfaction problems such as 3-SAT or 3-coloring; this is the type of problems that we are dealing with in the present work. In this case the primary objective is not to compute the marginals of some distribution, but to construct a solution to the constraint satisfaction problem. For example, BP can be used to (attempt to) compute a proper 3-coloring of a given graph. Indeed, empirically BP (and its sibling Survey Propagation “SP”) seems to perform well on problem instances that are notoriously “hard” for other current algorithmic approaches, including the case of sparse random graphs.
For instance, let G(n, p) be the random graph with vertex set V = {1, . . . , n} that is obtained by including each possible edge with probability 0 < p = p(n) < 1 independently. Thus, the expected degree of any vertex in G(n, p) is (n -1)p ∼ np. Then there exists a threshold τ = τ (n) such that for any ǫ > 0 the random graph G(n, p) is 3-colorable with probability 1 -o(1) if np < (1 -ǫ)τ , whereas G(n, p) is not 3-colorable if np > (1 + ǫ)τ [1]. In fact, random graphs G(n, p) with average degree np just below τ were considered the example of “hard” instances of the 3-coloring problem, until statistical physicists discovered that BP/SP can solve these graph problems efficiently in a regime considered “hard” for any previously known algorithms (possibly right up to the threshold density) [4,6]. While there are exciting and deep arguments from statistical physics that provide a plausible explanation of why these message passing algorithms succeed, these arguments are non-rigorous, and indeed no mathematically rigorous analysis is currently known.
The difficulty in understanding the performance of BP/SP on G(n, p) actually lies in two aspects. The first aspect is the combinatorial structure of the random graph G(n, p) with respect to the 3-coloring problem, which is not very well understood. In fact, even the basic problem of obtaining the precise value of the threshold τ is one of the current challenges in the theory of random graphs. Furthermore, we lack a rigorous understanding of the “solution space geometry”, i.e., the structure of the set of all proper 3-colorings of a typical random graph G(n, p) (e.g., how many proper 3-colorings are there typically, and what is the typical Hamming distance between any two). But according to the statistical physics analysis, the solution space geometry affects the behavior of BP significantly.
The second aspect, which we focus on in the present work, is the actual BP algorithm: given a graph G, how/why does the BP algorithm “construct” a 3-coloring? Thus far there has been no rigorous analysis of BP that applies to graph coloring instances except for graphs that are globally tree-like (such as trees or forests). However, it seems empirically that BP performs well on many graphs that are just locally tree like (i.e., do not contain “short” cycles). Therefore, in the present paper our goal is to analyze BP rigorously on a class of graphs that may have a complex combinatorial structure globally, but that have a very simple solution space geometry. More precisely, we shall relate the success of BP to spectral properties of the adjacency matrix of the input graph. In addition, we point out that the analysis comprises a natural random graph model (namely, a “planted solution” model).
The main contribution of this paper is a rigorous analysis of BP for 3-coloring. We basically show that if a certain (simple) spectral heuristic for 3-coloring succeeds, then so does BP. Thus, the result does not refer to a specific random graph model, but to a special class of graphs -namely graphs that satisfy a certain spectral condition. More precisely, we say that a graph G = (V, E) on n vertices is (d, ǫ)-regular if there exists a 3-coloring of G with color classes V 1 , V 2 , V 3 such that the following is true. Let 1 Vi ∈ R V be the vector whose entries equal 1 on coordinates v ∈ V i , and 0 on all other coordinates; then R1. for all 1 < i < j
…(Full text truncated)…
This content is AI-processed based on ArXiv data.
