Belief Propagation and Loop Series on Planar Graphs

Belief Propagation and Lo op Series on Planar Graphs Mic hael Chertk o v 1 , Vladimir Y. Chern y ak 2 and Razv an T eo dorescu 1 1 Theoretical Division and Cen ter for Nonlinear Studies, Los Alamos National Lab oratory , Los Alamos, NM 87545 2 Departmen t of Chemistry , W ayne State Universit y , 5101 Cass Ave,Detroit, MI 48202 E-mail: chertkov@lanl.gov , chernyak@chem.wayne.edu , razvan@lanl.gov Abstract. W e discuss a generic mo del of Ba y esian inference with binary v ariables defined on edges of a planar graph. The Lo op Calculus approach of [1, 2] is used to ev aluate the resulting series expansion for the partition function. W e show that, for planar graphs, truncating the series at single-connected loops reduces, via a map reminiscen t of the Fisher transformation [3], to ev aluating the partition function of the dimer matc hing mo del on an auxiliary planar graph. Th us, the truncated series can b e easily re-summed, using the Pfaffian form ula of Kasteleyn [4]. This allows to identify a big class of computationally tractable planar mo dels reducible to a dimer mo del via the Belief Propagation (gauge) transformation. The Pfaffian represen tation can also b e extended to the full Lo op Series, in which case the expansion b ecomes a sum of Pfaffian con tributions, each asso ciated with dimer matchings on an extension to a subgraph of the original graph. Algorithmic consequences of the Pfaffian representation, as well as relations to quan tum and non-planar mo dels, are discussed. P A CS n umbers: 02.50.Tt, 64.60.Cn, 05.50.+q Submitted to: Journal of Statistic al Me chanics Belief Pr op agation and L o op Series on Planar Gr aphs 2 1. In tro duction Bayesian Infer enc e can b e seen b oth as a sub-field of Information Theory and of general Statistical Inference [5]. A typical problem in this field is: giv en observed noisy data and kno wn statistical mo del of a noisy communication channel (transition probability), as w ell as a prior distribution for the input (a pre-image), find the most lik ely pre-image, or compute the a p osteriori marginal probabilit y for some part of the pre-image. This field is also deeply related to Combinatorial Optimization , which is a branch of optimization in Computer Science, related to op erations research, algorithm theory and complexity theory [6]. A t ypical problem in Combinatorial Optimization is: solv e, appro ximate or coun t (exactly or appro ximately) instances of problems b y exploring the exp onen tially large space of solutions. In man y emerging applications (in magnetic and optical recording, micro-fabrication, chip design, computer vision, net w ork routing and logistics), the data are structured in a t wo-dimensional grid (array). Moreo ver, data asso ciated with an element of the grid are often binary and correlations imp osed by the problem are lo cal, so that only nearest neigh b ors on the grid are correlated. Suc h problems are t ypically stated in terms of binary statistical mo dels on planar graphs. In this pap er, we discuss a generic problem of Bay esian inference defined on a planar graph. W e fo cus on the problem of weigh ted counting, or (from the p ersp ectiv e of statistical ph ysics) w e aim to calculate the partition function of an underlying statistical mo del. As the seminal w ork of Onsager [7] on the tw o-dimensional Ising mo del and its com binatorial interpretation b y Kac and W ard [8] hav e sho wn, the planarity constraint dramatically simplifies statistical calculations. By contrast, three-dimensional statistical mo dels are m uc h more c hallenging, and no exact results are known. Building on the w ork of ph ysicists, sp ecifically on results of Fisher [3, 9] and Kasteleyn [4, 10], Barahona [11] has sho wn that calculating the partition function of the spin glass Ising mo del on an arbitrary planar graph is e asy , as the num b er of op erations required to ev aluate the partition function scales algebraically , O ( N 3 ), with the size of the system. T o prov e this, the partition function of the spin-glass Ising mo del w as reduced to a dimer mo del on an auxiliary graph, and the partition function was expressed as the Pfaffian of a sk ew-symmetric matrix defined on the graph. The p olynomial algorithm was later used in simulations of spin glasses [12]. How ever, Barahona also added a grain of salt to the exciting p ositive result, showing that generic planar binary problem is difficult [11, 13]. Sp ecifically , ev aluating t w o-dimensional spin glass Ising mo del in a magnetic field is NP-hard, i.e. it is a task of lik ely exp onen tial complexit y . When an exact computational algorithm of p olynomial complexit y is not a v ailable, efficien t appro ximations b ecome relev ant. Typically , the appro ximation is built around a tractable case. One such approximate algorithm built around the Fisher-Kasteleyn Pfaffian form ula w as recen tly suggested b y Glob erson and Jaakkola in [14]. Although this appro ximation (coined “planar-graph decomp osition”) gives a pro v able upp er b ound for the partition function for some sp ecial graphical models, it constitutes just heuristics, i.e. it suffers from lack of error-con trol and the inabilit y of gradual error-reduction. Belief Pr op agation and L o op Series on Planar Gr aphs 3 Con trolling errors in approximate ev aluations of the partition function of a graphical mo del is generally difficult. Ho w ever, one recen t approac h, developed b y t w o of us and called Lo op Calculus [1, 2], offers a new metho d. Lo op Calculus allo ws to express explicitly the partition function of a general statistical inference problem via an expansion (the Lo op Series), where each term is explicitly expressed via a solution of the Belief Propagation [15, 16, 17], or Bethe-Peierls [18, 19, 20] (BP) equations. This brough t new significance to the BP concept, whic h previously w as seen as just heuristics. The BP equations are tractable for any graph; generally , the n um b er of terms in the Lo op Series is exp onen tially large, so direct re-summation is not feasible. How ever, since any individual term in the series can b e ev aluated explicitly (once the BP solution is kno wn), the Lo op Series representation offers a p ossibilit y for correcting the bare BP appro ximation p erturbatively , accounting for lo op con tributions one after another sequen tially . This scheme was shown to w ork well in impro ving BP deco ding of Low- Densit y Parit y Chec k co des in the error-flo or regime, where the num b er of imp ortant lo op contributions to the Lo op Series is (exp erimen tally) small, and the most imp ortan t lo op contributions (comparable b y absolute v alue to the bare BP one) ha v e a simple, single-connected structure [21, 22]. In spite of this progress, the question remained: what to do with other truly difficult cases when the n umber of imp ortant lo op corrections is not small, and when the imp ortan t corrections are not necessarily single-connected? In general, w e still do not know how to answ er these questions, while a partial answer for the imp ortan t class of planar mo dels is pro vided in this pap er. 1.1. Brief Description of Our R esults In this manuscript w e show that, for any graph (planar or not), the partial sum of the lo op series o v er single-connected loops reduces to ev aluation of the full partition function of an auxiliary dimer-matching mo del on an extended, regular degree-3 graph. W eights of dimers calculated on the extended graph are expressed explicitly via solution of the resp ectiv e BP equations. The dimer w eights can b e p ositive or negativ e. In general, summing the single-connected partition is not tractable. Ho wev er, in the planar case, it reduces (through manipulations reminiscent of the Fisher-Kasteleyn transformations) to a Pfaffian defined on the extended graph, whic h is also planar by construction. Thus, w e find a big class of planar graphical mo dels whic h are computationally tractable b y reduction (via a BP/gauge transformation) to a lo op series including only single- connected lo ops, and summable into a Pfaffian. Moreo ver, we find that the partition function of the en tire Lo op Series is generally reducible to a weigh ted Pfaffians series, where eac h higher-order Pfaffian is asso ciated with a sum of dimer configurations on a mo dified subgraph of the original graph. Eac h term in the Pfaffian series is computationally tractable via the Belief Propagation solution on the original graph. The material in the man uscript is organized as follo ws. A formal definition of the mo del is giv en in Section 1.2 and a brief description of Lo op Calculus [1, 2] forms Section 1.3. Some introductory material on the graphical transformations is also giv en Belief Pr op agation and L o op Series on Planar Gr aphs 4 in App endix A. Section 2 is dev oted to re-summation of the single-connected lo ops in the Lo op Series (we called it single connected partition). Section 2.1 introduces graphical transformation from the original graph G to the extended graph G e , reminiscent the Fisher transformation [3, 9]. This allo ws to restate the single-connected lo op partition of the Lo op Series on the original graph in terms of a sum o ver dimer configurations on the extended graph. Subsection 2.2 adapts the Kasteleyn transformation [4, 10] to our case, thus expressing the partition function of the single-connected series as a Pfaffian of a matrix defined on the extended graph. Section 3 describ es a set of graphical mo dels reducible under Belief Propagation gauge (transformation) to a Lo op Series whic h is computationally tractable. Section 4 describ es the representation of the Lo op Series for planar graphs in terms of the Pfaffian Series, where each Pfaffian sums dimer matc hings on a graph extended from a subgraph of G , with the later corresp ondent to exclusion of an even set of v ertices from G . Grassmann representations, as well as fermionic mo dels are discussed in Section 5: a general set of Grassmann mo dels on sup er-spaces is given in Section 5.1, while Section 5.2 addresses the relation b et ween binary mo dels and in tegrable hierarc hies. A brief list of future researc h topics is giv en in Section 6. 1.2. V ertex-function Mo del W e introduce an undirected graph G = ( V , E ) consisting of vertices V = ( a = 1 , · · · , N ) and edges E . This study focuses mainly on planar graphs, lik e those emerging in comm unication or logistics netw orks connecting or relating nearest neighbors on a 2d mesh or terrain. Ho w ev er, the material discussed in the present and the follo wing Subsections is general, and applies to any graph, planar or not. A binary v ariable, σ ab = ± 1, whic h we will also b e calling a spin, is asso ciated with an y edge ( a, b ) ∈ E . The graphical mo del is defined in terms of the probability function p ( ~ σ ) = Z − 1 Y a ∈V f a ( ~ σ a ) , (1) for a spin configuration ~ σ ≡ { σ ab = ± 1 |∀ ( a, b ) ∈ E } . In (1), ~ σ a = ( σ ab |∀ b, s.t. ( a, b ) ∈ E ) is the vector built from all edge v ariables asso ciated with the given v ertex a . f a ’s are p ositiv e and otherwise we will assume no restrictions on the factor functions. Z is the normalization factor, the so-called partition function of the graphical mo del. W e refer to (1) as “v ertex-function” models, according to statistical physics notation [18]. In the information theory , they are kno wn as F orney-style graphical models [23, 24]. W e will assume in the following that the degree of connectivity of any v ertex in the graph is three. Note that this is not a restrictive condition, as the n -th order vertices, corresp onden t to n -spin in teractions with n > 3, can alwa ys b e represented in terms of a pro duct of triplet terms. Then the n -th degree v ertex can b e transformed into a planar graph consisting of degree three vertices. W e discuss transformations to the triplets, in general but also on some examples (Ising Mo del and Parit y Check Deco ding of a linear co de), in App endix A. Belief Pr op agation and L o op Series on Planar Gr aphs 5 1.3. L o op Calculus Lo op Calculus [1, 2] giv es an explicit expression for Z through the Lo op Series: Z = Z 0 · z , z ≡ 1 + X C Y a ∈ C µ a, ¯ a C ! , µ a, ¯ a C ≡ ˜ µ a, ¯ a C ( a,b ) ∈ C Q b ∈ C p 1 − m ab ( C ) (2) m ab = X σ ab σ ab b ab ( σ ab ) , ˜ µ a, ¯ a C = X ~ σ a Y b ∈ ¯ a C ( σ ab − m ab ) b a ( ~ σ a ) , (3) where C can b e any allo w ed generalized lo op on the graph G , i.e. C is a subgraph of G which do es not contain an y v ertices of degree one; ¯ a C is a set of v ertices of graph G whic h are also contained in the generalized lo op C (by construction ¯ a consists of tw o or three elemen ts); and b a ( ~ σ a ) and b ab ( σ ab ) are b eliefs asso ciated with v ertex a and edge ( ab ). The b eliefs are defined via message v ariables η ab 6 = η ba ∀ ( a, b ) ∈ E : b ab ( σ ab ) = exp (( η ab + η ba ) σ ab ) 2 cosh ( η ab + η ba ) , (4) ∀ a ∈ V : b a ( ~ σ a ) = f a ( ~ σ a ) exp  P ( a,b ) ∈E b η ab σ ab  P ~ σ a f a ( ~ σ a ) exp  P ( a,c ) ∈E c η ac σ ac  , (5) solving the follo wing system of the Belief Propagation (BP) equations ∀ ( a, b ) ∈ E : X ~ σ a f a ( ~ σ a ) exp   ( a,b ) ∈E X b η ab σ ab   ( σ ab − tanh ( η ab + η ba )) = 0 . (6) The bare (BP) partition function Z 0 in Eq. (2) has the following expression in terms of the message v ariables: Z 0 = Q a P ~ σ a ∈V f a ( ~ σ a ) exp  P ( a,b ) ∈E η ab σ ab  Q ( a,b ) ∈E [2 cosh ( η ab + η ba )] . (7) BP equations (6) are interpreted as conditions on the gauge transformations, lea ving the partition function of the mo del in v arian t. These equations ma y allow multiple solutions, related to eac h other via resp ective gauge transformations. The multiple solutions corresp ond to m ultiple extrema of the Bethe F ree Energy and Lo op Series can b e constructed around an y of the BP solutions. ‡ 2. Re-summation of the Single-connected P artition In the following we will show how to re-sum a part of the Lo op Series accounting for all the single-connected lo ops, i.e. subgraphs of G with all vertices of degree t wo Z s = Z 0 · z s , z s = 1 + ∀ a ∈ C , | δ ( a ) | C =2 X C ∈G r C , (8) ‡ See [1, 2, 21] for a detailed discussion of this and other related features of BP equations as gauge fixing conditions. Belief Pr op agation and L o op Series on Planar Gr aphs 6 Figure 1. Left panel: T ransformation from a vertex of G to resp ectiv e three-v ertex of the extended graph G e . Right panel: maps from the colorings of a v ertex of G to coloring of the respective 3-v ertex of G e . Notice, that the coloring of the external edges of G e are rev ersed in comparison with the coloring of original edges on G . where | δ ( a ) | C stands for the num b er of neigh b ors of a within C . The ev aluation will consist of the follo wing t w o steps: A) Sho w that z s is equal to the partition function of the dimer-matching mo del on an auxiliary graph, G e . The graph will b e constructed from the original G by a transformation reminiscen t of the Fisher’s tric k, introduced in [3, 9, 11] to streamline reduction of Ising mo del to the dimer-matc hing mo del; B) Use the Pfaffian form ula of Kasteleyn [4, 10, 11] to reduce z s to a Pfaffian of a sk ew- symmetric matrix defined on G e . Note that complexity of the Pfaffian ev aluation is N 3 , where N is the size of G . Note: while A) is v alid for any graphical mo del, B) applies only to the planar case. 2.1. T r ansformation to Dimer Matching Pr oblem F ollowing the construction of Fisher [3, 9], w e expand eac h vertex of G into a three- v ertex of the extended graph G e , according to the sc heme sho wn in the left panel of Figure 1. Consider a vertex a of G and assume that b, c, d are three neighbors of a on G . F or each v ertex a , there are three µ a ;¯ a C con tributions of degree tw o within a generalized lo op C , i.e. with | δ ¯ a C | = 2, which can p ossibly con tribute to the single- connected partition r s : µ a ; bc , µ a ; bd , µ a ; cd . W e asso ciate the three w eights with in ternal edges of the resp ectiv e three-v ertex of G e , while the w eigh ts of all the external edges of the three-v ertex are equal to unity . Then any coloring of the original graph, marking a single connected lo op of G , is in the one-to-one corresp ondence to a dimer-matching (whic h we also call coloring) of G e . The weigh ts and coloring assignments are illustrated on an example at the left panel of Figure 1. An example of transformation mapping a single-connected-lo op on G resp ective dimer on G e is sho wn in Figure 2. This map from the single-connected lo ops to dimers leads to the following Belief Pr op agation and L o op Series on Planar Gr aphs 7 Figure 2. Example of G (upp er left) to G e (lo wer right) map. Single connected lo op of G (shown in red) is in one-to-one corresp ondence with a v alid dimer matching of G e , where dimers are also sho wn in red. Figure 3. ~ p orien tation (left panel) and resp ective dimer (matc hing) configurations (righ t panel) corresp ondent to example of G e describ ed b y Eq. (10). represen tation for the single-connected partition z s z s = X ~ π Y ( a,b ) ∈G e ( µ ab ) π ab Y a δ   ( a,b ) ∈G e X b π ab , 1   , (9) where the dimer-weigh ts on G e are defined according to the simple rules explained in the previous paragraph. One finds that the righ t hand side of (9) is nothing but the partition function of a dimer-matching problem on G e . 2.2. Pfaffian Expr ession for the Partition F unction Kasteleyn has shown in [4, 10] (see also [11]) that z s is equal to a Pfaffian (the square ro ot of determinan t) of a skew-symmetric matrix ˆ A = − ˆ A t of size N a × N a , where N a is the n um b er of vertices in G a . Each element of the matrix with a > b (ordering is arbitrary , but it is fixed once and forever) is A ab = p ab z ab , where p ab = ± 1. There are many p ossible choices of ~ p = ( π ab = ± 1 | ( a, b ) ∈ G e ) which guarantee the Pfaffian relation: z s = p det ˆ A . A simple constructiv e wa y of choosing suc h a v alid ~ p is to relate it to orien tation of edges in a directed version of G a , built according to the follo wing Belief Pr op agation and L o op Series on Planar Gr aphs 8 “o dd-face” rule: n umber of clo c kwise-orien ted segmen ts of an y in ternal face of G e should b e negative. § Example of a v alid orien tation is shown in Figure 2 and the resp ective expressions are z s ; example = µ 12 µ 34 + µ 14 µ 23 = p Det ˆ A, ˆ A =      0 − µ 12 0 − µ 14 µ 12 0 µ 23 − µ 24 0 − µ 23 0 µ 34 µ 14 µ 24 − µ 34 0      (10) Since calculating the determinant requires ∼ N 3 a op erations, one finds that re- summation of all the single-connected lo ops in the Loop Series expression for the partition function of a planar graphical mo del can b e done efficiently in O ( N 3 a ) steps. 3. T ractable Problems Reducible to Single-Connected P artition In the case of a general v ertex-function graphical mo del, the BP-gauge transformations, describ ed b y the set of BP equations (6), result in exact cancelation in the Lo op Series of all the subgraphs con taining at least one vertex of degree one within the subgraph. Thus, for the graph with all vertices of degree three, an y v ertex con tributing a generalized lo op (subgraph) should b e of degree t wo or three within the subgraph. As sho wn in the previous Section, if one ignores generalized lo ops with vertices of degree three and the original graph is planar, the resulting sub-series (single-connected partition) is computationally tractable, i.e. the num ber of op erations required to ev aluate the single-connected partition is cubic in the system size (not exp onential !). In this Section we discuss the class of planar mo dels whose Lo op Series do not con tain any generalized lo ops with vertices of degree three. According to Section 2, these mo dels are tractable. Indeed, it is known that BP Eqs. (6) ha v e at least one solution for the set of messages { η } on any graph and for any factor functions defined on the vertices of the graph. The aforementioned requiremen t for the generalized lo op not to contain an y v ertex of degree three translates into the following set of additional equations ∀ a ∈ G : X ~ σ a f a ( ~ σ a ) ( a,b ) ∈E Y b (exp ( η ab σ ab ) ( σ ab − tanh ( η ab + η ba ))) = 0 . (11) Considered together, the set of Eqs. (6,11) is o verdefined, i.e. it cannot b e solv ed in terms of η v ariables for an y v alues of the factor functions. Ho w ever, if one allows flexibility in the factor functions, and, in fact, considers Eqs. (6,11) as a set of conditions on b oth the messages { η } and the factor functions { f } , one arrives at a big set of p ossible solutions. Therefore, Eqs. (6,11) define a big set of mo dels reducible via BP transformations to a tractable Loop Series consisting only of single connected lo ops. Moreo v er, the relations w e established ma y b e reversed. One may start from an arbitrary Lo op Series consisting of only single connected loops, apply an arbitrary § Except, p ossibly , the external face. Belief Pr op agation and L o op Series on Planar Gr aphs 9 gauge transformation lea ving the Lo op Series inv arian t (these transformations are not necessarily of BP t yp e), and arrive at a graphical mo del with some set of factor functions. A t first sight, the resulting graphical mo del might not lo ok tractable, but it actually is, b y construction. 4. Lo op Series as a Pfaffian Series Let us notice that the general planar problem (e.g. spin glass in a magnetic field) is NP- hard [11], and it is thus not surprising that full re-summation do es not allow expression in terms of a single Pfaffian (or a determinant). On the other hand, w e already found that a part of the Lo op Series, sp ecifically its single-connected partition, reduces to a computationally tractable Pfaffian. This suggests to represent the full Lo op Series as a sum ov er terms, each representing a set of triplets (fully colored vertices of degree tree on G ): z = X Ψ z Ψ | ¯ a | =3 Y a ∈ Ψ µ a ;¯ a , (12) where Ψ is either the empt y set or an y set of even no des on G ; µ a ;¯ a = µ a ; bcd are the weigh ts from Eq. (2) asso ciated with the triplet ( a ; b, c, d ), such that ( a, b ) , ( a, c ) , ( a, d ) ∈ E ; and z Ψ is the sum o v er all generalized lo ops (prop er Lo op Series colorings, i.e. subgraphs) of G such that all no des of Ψ are fully colored (all edges adjusted to the no des b elong to the generalized lo op), while any other v ertices of G are not colored or only partially colored. Thus, the first term in Eq. (12), where Ψ is the empt y set, represen ts the single-connected partition, z s . W e show here that not only the first term in Eq. (12), asso ciated with Ψ = ∅ , but any term z Ψ in Eq. (12) is computationally tractable, b eing equal to a Pfaffian of a matrix defined on G e . Indeed, it is straightforw ard to verify that the generalized lo ops asso ciated with the given set of triplets (fully colored v ertices) from the set Ψ are in one-to-one corresp ondence with the set of dimer matchings on G e ;Ψ , which is a subgraph of G e with all 3-vertices corresp ondent to Ψ, and external edges connected to the vertices, completely remov ed. Notice that some v ertices of G e ;Ψ are of degree t wo. (These are v ertices neigh b oring the remo ved triplets of Ψ.) An example of a G e ;Ψ construction is given in Figure 4. One asso ciates weigh ts to the edges of G e ;Ψ in exactly the same wa y as for the single-connected partition: the w eigh ts of all the external edges of 3-v ertices of G e ;Ψ are equal to unit y , while the in ternal edges are asso ciated with the respective v alues µ a ; bc , defined in Eq. (3). F or any of G e ;Ψ one constructs the skew-symmetric ˆ A Ψ matrix according to the Kasteleyn rule for the dimer-matching mo del describ ed in Section 2.1. As b efore, the dimensionalit y of the matrix is |G e ;Ψ | × |G e ;Ψ | and eac h element of the matrix is the pro duct of the resp ective dimer weigh t and orien tation sign. Notice that the c hoice of signs for the elements of ˆ A Ψ dep ends on the set of “excluded” triplets Ψ, and th us ˆ A Ψ Belief Pr op agation and L o op Series on Planar Gr aphs 10 Figure 4. Two generalized loops (shown on the top) of an exemplary G corresp ondent to the same configuration of triplets G , | G | = 2, and their resp ective dimer configurations on G e ;Ψ (sho wn on the b ottom). is not simply a minor of the original matrix ˆ A , the one corresp onding to the single- connected partition (without exclusion). Th us, z Ψ = Pf  ˆ A Ψ  = r Det  ˆ A Ψ  . (13) Eqs. (12,13) describ e the Pfaffian series representation for the Lo op Series of the planar problem. 5. F ermion Represen tation and Mo dels An y Pfaffian in Eq.(13) allo ws a compact represen tation in terms of Grassmann v ariables [25]. Indeed, let us asso ciate a Grassmann (anti-comm uting or fermionic) v ariable θ a with eac h v ertex of G e . The Grassmann v ariables satisfy ∀ ( a, b ) ∈ G e : θ a θ b + θ b θ a = 0 , (14) and comm ute with ordinary c -n um b ers. One also introduces the Berezin in tegration rules o v er the Grassmann v ariables Z dθ = 0 , Z θ dθ = 1 . (15) This translates into the following rule of Gaussian in tegration o v er the Grassmann v ariables: Z exp  − 1 2 ~ θ t ˆ A ~ θ  d ~ θ = Pf( ˆ A ) = q det( ˆ A ) , (16) where ~ θ is the vector of the Grassmann v ariables ov er the entire graph, ~ θ = ( θ i | i ∈ G a ) and ˆ A is an arbitrary skew-symmetric matrix on the graph. F or example, applying this form ula to the first term of the Pfaffian series (12) one derives z ~ 0 = Z exp  − 1 2 ~ θ t ˆ A ~ θ  d ~ θ . (17) Belief Pr op agation and L o op Series on Planar Gr aphs 11 In general, any term in the Pfaffian series of Eq. (12) can be represented as a Gaussian Grassmann integrable, how ev er with different Gaussian kernels, not reducible simply to minors of ˆ A . 5.1. Gr aphic al Mo dels on Sup er-Sp ac es In this Subsection we first consider graphical mo dels on spaces generalizing the 2-point (binary) set to sup er-spaces containing commuting and anti-comm uting parts. The mo dels will b e defined on arbitrary (non necessarily planar) graphs. Then, we return to the simple example (17) of pure dimer mo del with the Grassmann (anticomm uting) v ariables defined on vertices of G e , to see that the mo del can b e restated as the vertex- function Grassmann mo del on the original graph G . The general class of vertex-function mo dels can b e in tro duced as follo ws. F or our graph, G , consider a set of spaces { M aα | a ∈ α } , i.e., w e asso ciate a space with any edge, α , together with a v ertex, a , that b elongs to the edge. F or simplicity w e assume the spaces to b e iden tical, i.e., M aα ∼ = M for all a ∈ α . The basic v ariables are σ aα ∈ M aα . W e also introduce the notation (all pro ducts b elo w are cartesian) M a = Y α 3 a M aα , M α = Y a ∈ α M aα , M = a ∈ α Y aα M aα = Y a M a = Y α M α ; (18) ~ σ a ∈ M a , ~ σ α ∈ M α , ~ σ ∈ M . (19) Note that an y M α is a t w o-comp onent cartesian pro duct. The vertex-function mo del is determined by a set of v ertex functions f a ( ~ σ a ) defined on M a and a set of integration measures dµ α ( ~ σ α ) on M α . The mo del partition function is Z = Z M Y α dµ α ( ~ σ α ) Y a f a ( ~ σ a ) . (20) F or the particular case when measures ha ve supp orts restricted to the diagonals M ∼ = ∆ α ⊂ M α ∼ = M × M , i.e. supp µ α ⊂ ∆ α , w e can consider the basic v ariables that b elong to the diagonals. This corresp onds to a more conv en tional formulation of the vertex-function mo dels with the v ariables residing on edges. Note that the mo dels in tro duced allo w for lo op-to w er calculus [26], form ulated in terms of fixing a prop er gauge. The BP gauge fixing for a general vertex-function model describ ed by Eq. (20) is nothing more than c ho osing basis sets in the vector spaces (maybe infinite-dimensional) of functions in M aα . A standard binary mo del, defined in Eq. (1), corresp onds to the c hoice M = { 0 , 1 } of the basic space to b e a 2-p oint set. V ertex mo dels with q -ary alphab et, e.g. discussed in [26], are describ ed b y M = { 0 , 1 , . . . , q − 1 } . Contin uous mo dels are obtained if M is chosen to b e a manifold of dimension m . The con tin uous case can b e extended to the c hoice of M to b e a sup ermanifold M of dimension ( m + , m − ) that con tains m − Grassmann (anticomm uting) co ordinates and whose substrate ¯ M ⊂ M is an m + -dimensional manifold. Note that a manifold can be considered as a sup ermanifold with zero o dd dimension m − = 0. In the remainder of this Subsection w e will be dealing Belief Pr op agation and L o op Series on Planar Gr aphs 12 with an opp osite case of the zero ev en dimension m + = 0, sp ecifically with the purely Grassmann case of the (0 , 1) supermanifold. Eq. (17) is the partition function of a mo del stated in terms of Grassmann v ariables defined on the vertices of G e . The extended graph G e is constructed from the original graph G so that a v ertex of G extends into a triangle with three vertices of degree three (see the left panel of Figure 1). Therefore, the three Grassmann v ariables in (17) are asso ciated with a v ertex of G . Then, Eq. (17) defined on G e allo ws an ob vious reform ulation in the vertex-function form (20) on G , where ~ σ a represen ts the three Grassmann v ariables that reside on the vertices of G e , obtained b y expanding the v ertex a of the original graph. The dimer weigh ts for the three edges of G e asso ciated with the extended v ertex of G are enco ded in the Gaussian function f a ( ~ σ a ). The dimer weigh t asso ciated with an edge of G e that represents and edge α of the original graph G is enco ded in the in tegration measure dµ α ( ~ σ α ). Also notice that the vertex-function Grassmann mo del on a planar graph G can b e restated as a mo del on the triangulated graph, dual to G , with complex fermion (Grassmann) v ariables asso ciated with the edges of the dual graph and functions asso ciated with a face (elementary triangle) of the dual graph (Figure A3 illustrates the duality transformation). One interesting conclusion here is that the sequence of transformations discussed ab ov e leads us from a sp ecial binary mo del on a planar graph G to a Gaussian fermion (Grassmann) mo del on the dual graph, thus represen ting an instance of the disorder op erator approac h of Kadanoff-Cev a [27] dev elop ed originally for the Ising model on a square lattice. 5.2. Comments on R elation to Quantum A lgorithms and Inte gr able Hier ar chies A mapping of a classical inference problem on to finding an expectation v alue in a corresp onding quantum model tak es on a natural in terpretation as a quantum algorithm. This can b e tried b y using the theory of the infinite Kadom tsev-P etviashvilii (KP) hierarc h y , sp ecifically its fermionic form ulation [28]. Consider 1 D lattice fermions ψ k , ψ ∗ k with k ∈ Z and in tro duce the p opulation b n k = ψ ∗ k ψ k and shift op erators b H k = P j ∈ Z ψ ∗ k + j ψ j . Let | 0 i denote the standard many-particle v acuum state where all single-fermion orbitals with k ≤ 0 are o ccupied, and | W i is some uncorrelated (i.e. represen ted by a single Slater determinan t) man y-particle state, which is sufficiently close to | 0 i . Introducing t = t 1 , t 2 , . . . , ¯ t = ¯ t 0 , ¯ t 1 , ¯ t 2 , . . . , and ξ = . . . , ξ − 1 , ξ 0 , ξ 1 , . . . w e consider an exp ectation v alue Z W ( t , ¯ t , ξ ) = h 0 | e P k> 0 t k b H k e P k ∈ Z ξ k b n k e P k ≤ 0 ¯ t − k b H k | W i (21) The approach is based on mapping the partition function of a classical inference problem on a graph onto a calculation of an exp ectation v alue represented by Eq. (21). W e hav e established such a mapping for some simple Grassmannian mo dels on planar graphs [29], where all the details on the suggested approach will b e presented. Note that in the case ξ = 0 and ¯ t = 0 the exp ectation v alue Z W ( t , 0 , 0) = τ W ( t ) is related to the τ -function of the KP in tegrable hierarc h y . Belief Pr op agation and L o op Series on Planar Gr aphs 13 6. F uture Challenges W e conclude with a brief and incomplete discussion of future c hallenges and opp ortunities raised b y this study . • W e plan to extend the study lo oking at new appro ximate sc hemes for intractable planar problems. One new direction, suggested in Section 3, consists of exploring the vicinit y of the computationally tractable mo dels reducible via the BP-gauge transformation to the series of single-connected lo ops. It is also of great in terest to explore the vicinit y of in tegrable tractable models mentioned in 5.2. • P erturbativ e exploration of a larger set of in tractable non-planar problems whic h are close, in some sense, to planar problems, constitutes another in teresting extension of the research. Here, one w ould aim to blend the aforemen tioned planar techniques with planar (or similar) decomp osition techniques, e.g. these of the t yp e discussed in [14]. • One imp ortant comp onent of our analysis consisted in the Pfaffian re-summation of the single-connected lo op (dimer) contributions, which is a sp ecial feature of the graph planarity . On the other hand, it is kno wn that the planarity is equiv alent to the graph b eing minor-excluded with resp ect to K 5 and K 3 , 3 subgraphs. Therefore, one wonders if there exists a generalization of the Pfaffian reduction to partition functions of mo dels from other and/or broader graph-minor classes defined within the graph-minor theory [30]?Likewise, comparing with previous studies of the non- planar/non-spherical cases, based on the dimer approach [31, 32, 33]. • Extending the Lo op Series analysis of the binary planar problem to the q-ary case seems feasible via the Lo op T ow er construction of [26]. This research should b e of a sp ecial in terest in the con text of recently prop osed polynomial quantum algorithm for calculating partition function of the Potts mo del [34]. Besides, recent progress [35, 36] sho ws that a Kasteleyn-t yp e approac h is extendable to a q -ary case, leading to the concept of “heaps of dimers”, and (in the con tin uum limit) to fascinating connections with sp ecial, highly symmetric complex surfaces, known as Harnac k curv es. • One w ould also b e interested to study how (and if ) phase transitions in the disorder- a v eraged planar ensembles, e.g. analyzed in [37, 38, 39, 40, 41, 42, 43, 44, 45], are related to distribution of parameters c haracterizing computational tractabilit y (complexit y) of the mo dels. • In [46], the problem of finding all pseudo-co dew ords in a finite cycle co de (corresp onding to the type of graphical mo del discussed in this pap er), was addressed by constructing a generating function known as graph zeta function [47]. The interesting fact discov ered in [46] is that this generating function of pseudo- co dew ords has a determinant formulation, based on a discrete graph op erator. Hence, one ma y an ticipate an existence of yet uncov ered relation b etw een the graph zeta function and a Pfaffian-Lo op resummation of related graphical mo dels. Belief Pr op agation and L o op Series on Planar Gr aphs 14 7. Ac kno wledgmen ts Researc h of M.C. and R.T. w as carried out under the auspices of the National Nuclear Securit y Administration of the U.S. Department of Energy at Los Alamos National Lab oratory under Con tract No. DE C52-06NA25396, and sp ecifically the LDRD Directed Researc h gran t on Physics of A lgorithms . M.C. also ackno wledges supp ort of the W eston Visiting Professorship program at the W eizmann Institute of Science, where he started to work on the man uscript. V.Y.C. ac kno wledges supp ort through the start-up funds from W a yne State Univ ersity . App endix A. Graphical T ransformations In this Appendix we discuss graphical transformations reducing an y binary problem to the vertex-function mo del describ ed by Eq. (1), where all vertices are of degree three. Our main fo cus here is on the planar graphs, and on the graphical transformations preserving planarit y . How ever some of the transformations and considerations discussed b elo w apply to an arbitrary graph. Often the original binary mo del is not represented in the vertex-function form. Some or all binary v ariables describing a problem ma y actually b e assigned to v ertices of a graph, then resp ective functions are asso ciated with edges and not vertices. Ob viously , one can also reform ulate the mo del reducing it to the v ertex (canonical for our purp oses) form. The transformation is illustrated in Figure A1. Algebraic form of the transformation sho wn in the Figure reads, P σ 1 f 12 ( σ 1 , σ 2 ) f 13 ( σ 1 , σ 3 ) f 14 ( σ 1 , σ 4 ) = P σ 12 ,σ 13 ,σ 14 χ ( σ 12 , σ 13 , σ 14 ) f 12 ( σ 12 , σ 2 ) f 13 ( σ 13 , σ 3 ) f 14 ( σ 14 , σ 4 ), where χ ( σ 12 , σ 13 , σ 14 ) is the c haracteristic function equal to unit y if all v ariables σ 12 , σ 13 , σ 14 are equal each other and equal to zero otherwise. Next, let us notice that, giv en a v ertex-function mo del (1) with the degree of connectivit y higher than three, one can alw ays p erform a sequence of transformations reducing the degree of connectivity of all the no des in the resulting graphical mo del to Figure A1. T ransformation from binary v ariable on a vertex, σ 1 , to set of v ariables, σ 12 , σ 13 , σ 14 on resp ectiv e edges. Belief Pr op agation and L o op Series on Planar Gr aphs 15 Figure A2. T ransformation which allows reduction of an N -v ertex to tw o ( N − 1)- v ertices. It is assumed that (1) n umber of no des in the gray area is not large, i.e. O ( N ), (2) the new graph (on the right) is planar, (3) ordering (sa y clo ckwise) of the external no des is preserved. The num ber of parameters characterizing the N -v ertex is 2 N or smaller, thus the num ber of parameters characterizing the tw o ( N − 1) vertices and v ertices from the gra y area is sufficien t, i.e. > 2 · 2 N − 1 , to parameterize the original v ertex. Figure A3. Planar triangulated graph (black) and its dual (red). three. An elemen tary graphical transformation of the kind is illustrated in Figure A2. It is assumed that the transformation is applied sequen tially to v ertices of degree larger than three till none of these are left. The end result is that: (a) there are no vertices of degree larger than three left within the graph; (b) the increase in the total n umber of vertices is p olynomial; (c) if the original graph is planar the resulting graph is also planar. The set of transformations just describ ed is general, and thus often inefficient, in the sense that kno wing sp ecific form of the factor functions one can practically alw a ys do a more efficient, customized and simpler reduction. Below w e will illustrate this p oint on examples. Belief Pr op agation and L o op Series on Planar Gr aphs 16 App endix A.1. Ising Mo del The spin glass Ising mo del is usually defined in terms of σ i = ± 1 v ariables asso ciated with v ertices of the graph p ( σ ) = Z − 1 exp   X ( i,j ) J ij σ i σ j   , (A.1) where summation under the exp onen tial on the r.h.s. go es o ver all edges of the graph, and J ij asso ciated with an edge can b e p ositiv e or negative. Ob viously one can apply the vertex-to-edges transformation, explained in Figure A1, to restate the spin glass Ising mo del as a vertex-funct ion mo del. How ev er, in this case one can also do a simpler transformation to the dual graph. Let us consider a planar triangulated graph Γ sho wn in blac k in Figure A3. All vertices of the resp ective dual graph, Γ d , sho wn in red in Figure A3, ha v e degree of connectivit y three. W e assume that the spin glass Ising model is defined on the planar triangulated graph Γ. Defining a new v ariable σ ab on an edge of Γ d as the pro duct of t wo v ariables of the original graph σ ab ≡ σ i σ j connected by an edge ( i, j ) of Γ crossing the edge ( a, b of Γ d , one finds that the sum on the r.h.s. of Eq. (A.1), rewritten in terms of the new v ariables, b ecomes, P ( a,b ) ∈ Γ d J ab σ ab . Ho w ev er, the new v ariables, σ ab are not indep endent, but rather related to eac h other via a set of lo cal constraints, ∀ a ∈ Γ d : Q ( a,b ) ∈ Γ d b σ ab = 1. Then, Eq. (A.1) restated in terms of the new v ariables on the dual graph gets the following compact vertex-st yle form p ( σ d ) = Z − 1 exp   X ( a,b ) ∈ Γ d J ab σ ab   Y a ∈ Γ δ   ( a,b ) ∈ Γ d Y b σ ab , 1   . (A.2) One in teresting observ ation is that the allo w ed configurations of σ d ≡ ( σ ab | ( a, b ) ∈ Γ d ) on the dual graph corresp ond exactly to the single-connected lo ops on Γ d , where the lo ops are built from the excited, σ ab = − 1, edges. Therefore, and in accordance with discussion of Section 2, calculation of the partition function for the spin glass Ising is reduced to ev aluation of the resp ectiv e Pfaffian, whic h is the task of a p olynomial complexit y . Notice also that adding a magnetic field (linear in σ ) term in the expression under the exp onen t on the r.h.s. of Eq. (A.2) will raise the complexit y lev el to exp onen tial. App endix A.2. Parity-Che ck Base d Err or-Corr e ction Consider a linear co de with the co de-b o ok defined in terms of the bi-partite T anner graph, G = ( V b , V c , E ) consisting of N = | V b | bits and M = | V c | parit y c hecks, and the set of edges E relating bits to chec ks and c hecks to bits. Then a message ~ σ = ( σ i = 0 , 1 | i = 1 , · · · N ) is a co deword of the co de if it satisfies all the parit y c hec ks, i.e. ∀ α = 1 , · · · M : Q ( i,α ) ∈E i σ i = +1. Assuming that all the co dew ords are equally probable originally , and that the white c hannel transform a bit σ of the original co dew ord into the signal x with the probabilit y p ( x | σ ), one finds that the probability Belief Pr op agation and L o op Series on Planar Gr aphs 17 Figure A4. An illustrative example of a T anner graph (left), as w ell as chec k-vertex (A) and bit-v ertex transformations. for ~ σ to b e a co dew ord resulted in the measuremen t ~ x is p ( ~ σ | ~ x ) = 1 Z e P i ∈ V b σ i h i Y α ∈ V c δ   ( i,α ) ∈E Y i σ i , +1   , h i ≡ 1 2 ln p ( x i | + 1) p ( x i | − 1) , (A.3) where, as usual, the partition function Z is fixed b y the normalization condition, P ~ σ p ( ~ σ | ~ x ) = 1. Eq. (A.3) represents an example of a mixed graphical mo del, with v ariables σ i defined on bit-vertices, the parit y-chec k functions defined on chec k-v ertices and the c hannel functions (carrying the dep endencies on the log-likelihoo ds h i ) also asso ciated with the bit-v ertices. In this case transformation to the v ertex-style mo del is done by direct application of the vertex-to-edges pro cedure of Figure A1 to all the bit-vertices of G . Then, the verte x-style version of Eq. (A.3) b ecomes p ( ~ σ | ~ x ) = Z − 1 Y α ∈ V c f α ( ~ σ ) Y i ∈ V b f i ( ~ σ i ) , (A.4) ∀ i : ~ σ i ≡ ( σ iα = ± 1 | ( i, α ) ∈ E ) , (A.5) f i ( ~ σ i ) = ( exp( h i σ iα ) , ∀ α, β s.t. ( i, α ) , ( i, β ) ∈ E : σ iα = σ iβ , 0 , otherwise , (A.6) ∀ α : ~ σ α ≡ ( σ iα | ( i, α ) ∈ E ) , f α ( ~ σ α ) = δ   ( i,α ) ∈E Y i σ iα , +1   . (A.7) In general, degree of connectivity of bit-vertices and c heck-v ertices ma y be arbitrary . Direct application of the general pro cedure explained ab ov e (see Figure A1 and discussion therein) allo ws to reduce all the higher-degree no des to a larger set of no des of degree three. How ever, a simpler dendro-reduction is possible b oth for the bit-v ertices and c hec k-vertices. The dendro tric k (e.g. discussed in [48] for complexity reduction of a Linear Programming deco ding of LDPC co des) is sc hematically illustrated in the tw o righ t panels of Figure A4, where resp ectiv e algebraic relations are (A) : δ 6 Y i =1 σ i , +1 ! = (A.8) Belief Pr op agation and L o op Series on Planar Gr aphs 18 X σ 12 ,σ 34 ,σ 56 = ± 1 δ ( σ 1 σ 2 σ 12 , +1) δ ( σ 3 σ 4 σ 34 , +1) δ ( σ 5 σ 6 σ 56 , +1) δ ( σ 12 σ 34 σ 56 , +1) , (B) : δ ( σ 1 , · · · , σ 6 ) = (A.9) X σ 12 ,σ 34 ,σ 56 = ± 1 δ ( σ 1 , σ 2 , σ 12 ) δ ( σ 3 , σ 4 , σ 34 ) δ ( σ 5 , σ 6 , σ 56 ) δ ( σ 12 , σ 34 , σ 56 ) , and δ ( σ 1 , · · · , σ 6 ) is equal to unit y if all arguments are the same, and it is zero otherwise. Bibliograph y [1] Chertko v M and Cherny ak V Y, L o op Calculus in Statistic al Physics and Information Scienc e , 2006 Ph ys. Rev. E 73 065102(R) [cond-mat/0601487] [2] Chertko v M and Chern y ak V Y, L o op series for discr ete statistic al mo dels on gr aphs , 2006 J. Stat. Mec h. P06009 [cond-mat/0603189] [3] Fisher M E, Statistic al Me chanics on a Plane L attic e , 1961 Phys. Rev 124 1664 [4] Kasteleyn P W, The statistics of dimers on a lattic e , 1961 Physics 27 1209 [5] MacKay D, Information The ory, Infer enc e, and L e arning Algorithms , 2003 Cam bridge Univ. Press [6] Papadimitriou C H and Steiglitz K, Combinatorial Optimization: Algorithms and Complexity , 1998 Do ver [7] Onsager L, Crystal Statistics , 1944 Phys. Rev. 65 117 [8] Kac M and W ard J C, A c ombinatorial solution of the Two-dimensional Ising Mo del , 1952 Phys. Rev. 88 1332 [9] Fisher M E, On the dimer solution of planar Ising mo dels , 1966 J. Math. Phys. 7 1776 [10] Kasteleyn P W, Dimer Statistics and Phase T r ansitions , 1963 J. Math. Phys. 4 287 [11] Barahona F, On the c omputational c omplexity of Ising spin glass mo dels , 1982 J.Phys. A 15 3241 [12] Saul L and Kardar M, Exact inte ger algorithm for the two-dimensional ± J Ising spin glass , 1993 Ph ys. Rev. E 48 R3221 [13] Jerrum M, Two-dimensional monomer-dimer systems ar e c omputational ly intr actable , 1987 J. Stat. Ph ysics 48 121134 [14] Glob erson A and Jaakk ola T, Appr oximate infer enc e using planar gr aph de c omp osition , in Pr o c e e dings of A dvanc es in Neur al Information Pr o c essing Systems , 2006 20 [15] Gallager R G, L ow density p arity che ck c o des , 1963 MIT Press Cambridge MA [16] Gallager R G, Information The ory and R eliable Communic ation , 1968 J. Wiley New Y ork [17] Pearl J, Pr ob abilistic r e asoning in intel ligent systems: network of plausible infer enc e , 1988 Kaufmann San F rancisco [18] Baxter R J, Exactly Solve d Mo dels in Statistic al Me chanics , 1982 Academic Press [19] Bethe H A, 1935 Pro c. Roy . So c. London A 150 552 [20] Peierls R, Ising’s mo del of ferr omagnetism , 1936 Pro c. Camb. Phil. So c. 32 477 [21] Chertko v M and Chern y ak V Y, L o op Calculus Helps to Impr ove Belief Pr op agation and Line ar Pr o gr amming De c o dings of L ow-Density-Parity-Che ck Co des , invited talk, 44th Allerton Conference 2006 [ arXiv:cs.IT/0609154] [22] Chertko v M, R e ducing the Err or Flo or , invited talk at the Information Theory W orkshop on “F ron tiers in Co ding” 2007 [h [23] F orney G D, Co des on Gr aphs: Normal R e alizations , 2001 IEEE IT 47 520-548 [24] Lo eliger H A, An Intr o duction to F actor Gr aphs , 2001 IEEE Signal Pro cessing Magazine 28 [25] Berezin F, Intr o duction to Sup erAnalysis , 1987 Springer [26] Cherny ak V and Chertk ov M, L o op Calculus and Belief Pr op agation for q-ary Alphab et: L o op T ower , 2007 Pro ceedings of ISIT [cs.IT/0701086] [27] Kadanoff L P and Cev a H, Determination of an Op er ator Algebr a for the Two-Dimensional Ising Mo del , 1971 Phys. Rev. B 3 3918 Belief Pr op agation and L o op Series on Planar Gr aphs 19 [28] Sato M, Miwa T and Jim b o M 1977 Pro c. Japan Acad. 53A 6; 1978 Publ. Res. Int. Math. Sci. 14 223; 1979 15 201 577 871; 1980 16 531 [29] T eo dorescu R, Cherny ak V and Chertko v M, in pr ep ar ation [30] Lov´ asz L, Gr aph Minor The ory , 2005 Bulletin of the American Mathematical So ciety 43 75 [31] Regge T and Zecchina R, Exact solution of the Ising mo del on gr oup lattic es of genus g > 1, 1996 J. Math. Ph ys. 37 2796-2814 [32] Regge T and Zecchina R, Combinatorial and top olo gic al appr o ach to the 3D Ising mo del , 2000 J. Ph ys. A: Math.Gen. 33 741 [33] Galluccio A, Lo ebl M and V ondrak J, New algorithm for the Ising pr oblem: Partition function for finite lattic e gr aphs , 2000 Phys. Rev. Lett. 84 5924-5927 [34] Aharonov D, Arad I, Eban E and Landau Z, Polynomial Quantum Algorithms for A dditive appr oximations of the Potts mo del and other Points of the T utte Plane , QIP 2007 [35] Di F rancesco P and Guitter E, Inte gr ability of gr aph c ombinatorics via r andom walks and he aps of dimers , [arXiv:math/0506542v1] [36] Keny on R and Okounk ov A, Planar dimers and Harnack curves , [arXiv:math/0311062v2] [37] Sherrington D and Kirkpatrick S, 1975 Phys. Rev. Lett. 35 [38] Dotsenko V S and Dotsenk o V S, Critic al b ehaviour of the phase tr ansition in the 2D Ising Mo del with impurities , 1983 Adv ances in Physics 32 129-172 [39] Honeck er A, Picco M and Pujol P 2001 Phys. Rev. Lett. 87 047201 [40] Amoruso C, Hartmann A K, Hastings M B and Mo ore M A 2006 Phys. Rev. Lett. 97 267202 [41] Bhaseen B J, Caux J S, Kogan I I and Tsvelik A M 2001 Nucl.Phys. B 618 465 [42] Efetov K, Sup ersymmetry in Disor der and Chaos , 1997 Cambridge Universit y Press [43] Edwards S F and Anderson P W 1975 J. Phys. F 5 965 [44] Fisher D S and Huse D A 1986 Phys. Rev. Lett. 56 1601 [45] M´ ezard M, Parisi G and Virasoro M A, Spin Glass The ory and Beyond , 1987 W orld Scientific [46] Ko etter R, Li W-C. W, V on tob el P O and W alk er J L, Pseudo-c o dewor ds of cycle c o des via zeta functions 2004 Pro c. IEEE Inform. Theory W orkshop p. 6 [47] Horton M D, Stark H M and T erras A A, What ar e zeta functions of gr aphs and what ar e they go o d for? 2006 Con temp orary Mathematics 415 173 [48] Chertko v M and Stepanov M, Pseudo-c o dewor d L andsc ap e , 2007 Pro c. of ISIT [cs.IT/0701084]