Fastest Distributed Consensus Problem on Branches of an Arbitrary Connected Sensor Network

ISTRIBUTED consensus has appeared as one of the important and primary problems in the context of distributed computation (see, for example, [1] for early work). Some of its applications include distributed agreement, synchronization problems, [2] and load balancing in processor networks [3,4]. A problem that has received renewed interest recently is distributed consensus averaging algorithms in sensor networks. The main purpose of distributed consensus averaging algorithm on a sensor network is to compute the average of the initial node values, via a distributed algorithm, in which the nodes only communicate with their immediate neighbors. One of main research directions in this issue is the computation of the optimal weights that yield the fastest convergence rate to the asymptotic solution [5,6] known as Fastest Distributed Consensus (FDC) averaging algorithm. Moreover algorithms for distributed consensus find applications in, e.g., multi-agent distributed coordination and flocking [7,8,9], distributed data fusion in sensor networks [10,11,6], fastest mixing Markov chain problem [12], gossip algorithms [13,14], and distributed estimation and detection for decentralized sensor networks [15,16,17,18,19]. In previous works determining optimal weights and convergence rate of FDC averaging algorithm over a sensor S. Jafarizadeh and A. Jamalipour are with the School of Electrical and Information Engineering, University of Sydney, Sydney NSW 2006, Australia (e-mail: saber.jafarizadeh@sydney.edu.au & abbas.jamalipour@sydney.edu.au). network demanded full knowledge about network's topology. But here in this work we have determined the optimal weights for edges of five different branches connected to an arbitrary network without requiring full knowledge about network's topology. The branches considered in this paper are path, lollipop, semi-complete, ladder and palm branches. Stratification [20,21] and semidefinite programming are the key methods used for evaluating optimal weights. We have proved that the obtained weights are optimal and independent of rest of network. Several examples and simulation results are provided to confirm the validity of the obtained results. By numerical simulations we have compared the branches in terms of asymptotic and per step convergence rates. Also it is shown that the obtained results hold true in serial combination of these branches too. The organization of the paper is as follows. In section II we briefly review prior works on the agreement and consensus algorithms. Section III is an overview of the materials used in the development of the paper, including relevant concepts from distributed consensus averaging algorithm, graph symmetry, stratification and semidefinite programming. Section IV contains the main results of the paper where four different branches namely path, lollipop, semi-complete, ladder and palm branches are introduced together with the obtained optimal weights. In section V some examples are presented. Section VI is devoted to the proof of main results of paper. Section VII presents simulations and section VII concludes the paper. Tsitsiklis [1] gave a systematic study of agreement algorithms of the general type in an asynchronous distributed environment. Their recent work [9] summarizes the key results and establishes some new extensions. In [8], a continuous time state update model was adopted for consensus and the results were extended to situations involving switching sensor network topology and delayed communication. In [12] it has been shown that the convergence rate of FDC averaging algorithm is determined by the second largest eigenvalue modulus (SLEM) of the weight matrix furthermore FDC averaging problem is formulated as a convex optimization problem, in particular a semidefinite program. A well-studied method for choosing weights is the nearest neighbor rule. This method of choosing weights was analyzed in detail in [7]. In [5], the problem of designing the optimal weights was addressed for a fixed sensor network topology. The authors deal with the FDC averaging problem by numerical convex optimization methods but no closed-form solution for finding the optimal weights were proposed. In [6] the authors introduce the Metropolis weights over the timevarying communication graphs. They show that Metropolis weights preserve the average of node values and converge to the average of node values, provided the infinitely occurring communication graphs are jointly connected. In [20] the authors show how to exploit symmetries of a graph to efficiently compute the optimal weights of the fastest distributed consensus averaging algorithm. Many works dealt with reaching a quantized consensus (see [22,23] and references therein). In [24], Carli et al. consider the problem of reaching a consensus using quantized channels. They propose to round each node value at each step to the nearest integer. The proposed quantization method by [24] conserves the average of the states but the nodes do not necessarily reach a consensus and they converge to different values. In [25], Aysal et al. propose a different method for quantization called -probabilistic quantization‖ scheme. Using their method nodes converge to a consensus but the average of node values is not conserved. This section introduces the notation used in the paper and reviews relevant concepts from distributed consensus averaging algorithm, stratification, graph symmetry and semidefinite programming. We consider a network with the associated graph consisting of a set of nodes and a set of edges where each edge is an unordered pair of distinct nodes. The main purpose of distributed consensus averaging is to compute the average of the initial node values , through the distributed linear iterations . is the vector of initial node values on the network. is the weight matrix with the same sparsity pattern as the adjacency matrix of the network's associated graph and is the discrete time index (Here denotes the column vector with all coefficients one). In [5] it has been shown that the necessary and sufficient conditions for the convergence of linear iteration mentioned above is that one is a simple eigenvalue of associated with the eigenvector , and all other eigenvalues are strictly less than one in magnitude. Moreover in [5] FDC averaging problem has been formulated as the following minimization problem where are eigenvalues of arranged in decreasing order and is the Second Largest Eigenvalue Modulus (SLEM) of , and the main problem can be formulated in the semidefinite programming form as [5]: (1) We refer to problem (1) as the Fastest Distributed Consensus (FDC) averaging problem. An automorphism of a graph is a permutation of such that if and only if , the set of all such permutations, with composition as the group operation, is called the automorphism group of the graph and denoted by . For a vertex , the set of all images , as varies through a subgroup , is called the orbit of under the action of . The vertex set can be written as disjoint union of distinct orbits. Stratifying a graph into its orbits is called stratification [20,21] (in [20] this method is entitled block-diagonalization) and each orbit is called a stratum. In [20], it has been shown that the weights on the edges within an orbit are the same. For the sake of clarity, consider the wheel graph; (see Fig. 1). The automorphism group is isomorphic to the 6-element cyclic group , and corresponds to flips and 60degree rotations of the graph. The orbits of acting on the vertices are and there are two orbits of edges (2) (3) Fig. 1. a wheel graph . SDP is a particular type of convex optimization problem [26]. An SDP problem requires minimizing a linear function subject to a linear matrix inequality constraint [27]: where is a given vector, , and , for some fixed hermitian matrices . The inequality sign in means that is positive semi-definite. This problem is called the primal problem. Vectors whose components are the variables of the problem and satisfy the constraint are called primal feasible points, and if they satisfy , they are called strictly feasible points. The minimal objective value is by convention denoted by and is called the primal optimal value. Due to the convexity of the set of feasible points, SDP has a nice duality structure, with the associated dual program being: Here the variable is the real symmetric (or Hermitian) positive matrix , and the data , are the same as in the primal problem. Correspondingly, matrix satisfying the constraints is called dual feasible (or strictly dual feasible if ). The maximal objective value of , i.e. the dual optimal value is denoted by . The objective value of a primal (dual) feasible point is an upper (lower) bound on . The main reason why one is interested in the dual problem is that one can prove that , and under relatively mild assumptions, we can have . If the equality holds, one can prove the following optimality condition on . A primal feasible and a dual feasible are optimal, which is denoted by and , if and only if (2) which is called the complementary slackness condition. In one way or another, numerical methods for solving SDP problems always exploit the inequality , where and are the objective values for any dual feasible point and primal feasible point, respectively. The difference is called the duality gap. If the equality holds, i.e. the optimal duality gap is zero, then we say that strong duality holds. Here we have introduced five different branches namely Path branch, Lollipop branch, Semi-complete branch, Ladder branch and palm branch along with their corresponding evaluated optimal weights. Proofs and more detailed discussion are deferred to Section VI. The simplest form of branch is Path branch where a path graph consisting of nodes is connected to an arbitrary graph by a bridge as shown in Fig. 2. Using the same procedure as done in section VI for semicomplete branch we can state that the optimal weights for the edges of a path branch of an unknown network, equals , independent of the rest of network, except for the last edge (weighted by in Fig. 2) which connects path branch to the rest of the network. The -lollipop graph is the graph obtained by joining a complete graph to a path graph . We define the lollipop branch as a lollipop graph which is connected to an arbitrary graph by sharing the end node of path part with the arbitrary graph as shown in Fig. 3 for , . In section VI-A we have proved that in a Lollipop branch of an arbitrary network, the optimal weights for the edges on the complete graph and path graph equal and , respectively, independent of the rest of network, except for the last edge which connects Lollipop branch to the rest of the network (weighted by in Fig. 3). The Rest of Graph The Semi-Complete branch is a path branch with a semi-complete graph inside. A semi-complete graph is a complete graph without the edge between two nodes connected to path links. The whole branch consists of two path graphs and with and nodes, respectively, where each one of path graphs are connected to semi-complete graph by means of a bridge, as shown in Fig. 4 for . In section VI-B it has been proved that in a semi-complete branch of an arbitrary network, the optimal weights on the edges connecting inner nodes together and the edges connecting outer nodes to inner nodes in the semi-complete graph equal and , respectively. Outer nodes of semi-complete graph are the two nodes connected to path graphs by a bridge and the remaining nodes of semi-complete graph are the inner nodes of semi-complete graph. The optimal weights on the edges of path graphs and equal independent of the rest of network, except the last edge which connects the semi-complete branch to the rest of the network (weighted by in Fig. 4). The Rest of Graph The Ladder branch consists of two path branches and connected by four bridges to a complete ladder graph. A complete ladder graph is a ladder graph with nodes including a complete graph between every 4 neighboring nodes, or in other words a complete ladder graph consists of complete graphs where everyone of these complete graphs are sharing two nodes with the neighboring ones. A Ladder branch is depicted in Fig. 5. In section VI-C it has been proved that in a Ladder branch of an arbitrary network, the optimal weights of the edges connecting two nodes from one level of ladder to the nodes on the neighboring levels equal where the nodes on one level of complete ladder graph are the nodes on the same vertical position in Fig. 5. The complete ladder graph includes levels of nodes. The optimal weights of four bridges connecting complete ladder graph to two path graphs and equal . The optimal weights of the edges of path graphs and equal , independent of the rest of network, except the last edge which connects the semicomplete branch to the rest of the network (weighted by in Fig. 5). Also in section VI-C it has been shown that the SLEM of network is independent of the weights of edges connecting two nodes of complete ladder graph on the same level to each other (weighted by for in Fig. 5) as long as these weights satisfy (20). The palm branch of order is obtained by joining a path graph of length to a symmetric star graph with branches of length as shown in Fig. 6 for , . In section VI-D we have proved that in a palm branch of an arbitrary network, the optimal weights for the edges on path graph and star graph equal and , respectively, independent of the rest of network, except for the last edge which connects Lollipop branch to the rest of the network (weighted by in Fig. 6). w Fig. 6. An arbitrary graph with a palm branch for , . V. EXAMPLES In this section several examples for five types of branches introduced in previous section are presented. The simplest example is a sensor network with path topology. In a path network, the paths from every middle node to both ends of network can be considered as a path formed branch with the optimal weights equal which agrees with those of [28]. As another example, in [29,30] three types of networks have been studied, namely, Complete Cored Star, Star and Two Fused Star networks where in these networks path formed branches are connected to a complete graph and a central node respectively. In [29,30] it has been proved that the optimal weights for the edges on path formed branches are except for the last edge which connects the branch to the rest of network which confirms the results obtained in section VI. As an example for the Lollipop branch we consider the extended Barbell topology which is a network obtained by connecting two networks with complete graph topology , by a path bridge as shown in Fig. 7 for and . Dividing an extended Barbell network from every node on path bridge splits the network into two Lollipop branches and the results obtained in section VI-B imply that the optimal weights equal for the edges of path bridge and , for the edges of each one of complete graphs and , respectively. According to subsection III-B, the stratification of Barbell network reduces it to a path graph with nodes and disconnected single nodes where is the number of edges on the path bridge. Using the results obtained in [28] and section VI-B, one can deduce that the weight matrix of extended Barbell network has zero eigenvalues and non zero eigenvalues, which are for with and one eigenvalue equal to . Thus the SLEM of Barbell network equals . As a simple example for the semi-complete branch introduced in section IV-B we can consider a semi-complete graph with two path branches and as depicted in Fig. 8 for and . Dividing the network depicted in Fig. 8 from every node on both path branches splits it into a semi-complete and path branch. The optimal weights of the edges of path branches and equal and the optimal weights of the edges connecting inner nodes to each other equal and the edges connecting outer nodes to inner nodes in the semi-complete graph equal , respectively. According to subsection III-B, the stratification of the network reduces it to a path graph with nodes and disconnected single nodes. Using the results obtained in [28] and subsection III-B, one can deduce that the semi-complete graph with two path branches has zero eigenvalues and non zero eigenvalues, which are for with and one eigenvalue equal to . Thus the SLEM equals . As a simple example for the ladder branch introduced in section IV-D we can consider a Ladder graph with nodes connected to two path branches and by four bridges as depicted in Fig. 5 for and . In the network depicted in Fig. 9 each node on one of path branches splits the network into a ladder branch and path branch. The results obtained in section VI-C imply that the optimal weights equal for the edges of path branches ( and ) and for four bridges connecting path graphs and to complete ladder graph and for the edges of complete ladder graph as depicted in Fig. 9 for and . In this section solution of fastest distributed consensus averaging problem and determination of optimal weights for semi-complete branch introduced in section IV is presented. Due to lack of space for Lollipop, Ladder and palm branches we have only presented the stratification of network's connectivity graph. , which we call them the known part of the network. We denote the set of nodes of path graph by and the nodes of complete graph by where is the node connected to path graph , (see Fig. 3 for , ). Automorphism of Lollipop branch is permutation of nodes of complete graph which are not connected to path graph. Hence according to subsection III-B it has orbits, acting on vertices which are and class of edge orbits on the known part of network. Thus it suffices to consider just weights (as labeled in Fig. 3 for , ). We associate with the node , the column vector (where is the total number of nodes of network) with in the -th position, and zero elsewhere. Introducing the new basis , for and for where , the weight matrix for the known part of network in the new basis can be defined as where is the identity matrix of dimension and is as follows: In this section we solve the Fastest Distributed Consensus (FDC) averaging problem for an arbitrary network with a semi-complete branch, using stratification and Semidefinite Programming (SDP). We consider a network with the undirected associated connectivity graph , where is the set of nodes and is the set of edges. The whole branch is called the known part of network and consists of two path links with and edges and one semi-complete graph with nodes. We denote the set of nodes on the whole branch by (see Fig. 4 for ). Automorphism of semi-complete branch is permutation of nodes on semi-complete graph which are not connected to path graphs. According to subsection III-B it has class of edge orbits. Thus it suffices to consider just weights namely (as labeled in Fig. 4 for ). Therefore the weight matrix of semi-complete branch can be written as: We assign with node the column vector (where is the total number of nodes on network) with in the -th position and zero elsewhere, then the edge can be written as . In the new basis defined as with , the weight matrix for semi-complete branch takes the form as The off diagonal elements of first rows and columns of are zero and the diagonal entries equal which are the eigenvalues as well and by considering the fact that SLEM is the second largest eigenvalue in magnitude we should have which is definitely satisfied by (3) Based on subsection III-A, one can express FDC problem for semi-complete branch in the form of semidefinite programming as: (4) where is a column vector defined as: which is eigenvector of corresponding to the eigenvalue one. The weight matrix can be written as (5) where includes the weights on the unknown part of network and for are column vectors defined as In order to formulate problem (4) in the form of standard semidefinite programming described in section II-C, we define and as below: Using the constraints we have To have the strong duality we set , hence we have . Considering the linear independence of for , we can expand and in terms of as (9) with the coordinates and , to be determined. Using (5) and the expansions (9), from equalizing the coordinates of for in the slackness conditions (7), we have (10) where (10) hold for . Considering (8-b), (8-c), (8-d) and (8-e) we obtain for , or equivalently (11) for and for and , we have (12) where is the Gram matrices, defined as , or equivalently Substituting (12) in (10) we have b) and (14-b) hold for . Now we can determine the optimal weights in an inductive manner as follows: In the first stage, from comparing equations (13-a) and (14a) and considering the relation (11), we can conclude that , which results in and , where the latter is not acceptable. Assuming and substituting in (13-a) and (14-a), we have Continuing the above procedure inductively, up to stages, and assuming in the -th stage, we get the following equations from comparison of equations (13-b) and (14-b), (15-a) (15-b) while considering relation (11) we can conclude that which results in (16) Substituting in (15), we have (17) where ( 16) and ( 17) are true for and in the -th stage, in the same way as in previous stages from equations (13-c) and (14-c) we have In the -th stage, from equations (13-d) and (14-d) and using relation (11), we can conclude that where The relation above results in and , where the latter is not acceptable and In the -th stage, from equations (13-e) and (14-e) and using relation (11), we can conclude that which results in and , where the latter is not acceptable and doing the same inductive procedure as explained in previous stages, from equations (13-b) and (14b) we have ( where ( 18) and ( 19) are true for . Using (3) we have A ladder branch consists of two path graphs with and edges connected by four bridges to a complete ladder graph with nodes. We call the whole branch, the known part of network and denote the set of nodes on the whole branch by (see Fig. 5). Automorphism of ladder branch is permutation of nodes on the same level of complete ladder graph. According to subsection III-B it has and class of edge orbits on path graphs and the complete ladder graph, respectively. Thus it suffices to consider just weights namely (as labeled in Fig. 5). We assign with node the column vector (where is the total number of nodes of network) with in the -th position and zero elsewhere, then the edge can be written as . In the new basis defined as the weight matrix of ladder branch can be defined as: The off diagonal elements of first rows and columns of are zero and the diagonal entries equal , for and which are the eigenvalues as well. Considering the fact that SLEM is the second largest eigenvalue in magnitude, we can conclude that the SLEM of network is independent of for as long as these weights satisfy the following relations. We denote the set of nodes of path graph by and the nodes of star graph by . is the central node of star graph connected to path graph , (see Fig. 6 for , ). Automorphism of Lollipop branch is permutation of branches of star graph which are not connected to path graph via central node. Hence according to subsection III-B it has orbits, acting on vertices which are and class of edge orbits on the known part of network. Thus it suffices to consider just weights (as labeled in Fig. 6 for , ). We associate with the node , the column vector (where is the total number of nodes of network) with in the -th position, and zero elsewhere. Introducing the new basis , for and for where , the weight matrix for the known part of network in the new basis can be defined as where is the identity matrix of dimension and is as follows: VII. CONVERGENCE RATES OF BRANCHES In this section we aim to compare five branches introduced in section IV in terms of asymptotic and per step convergence rates. Also we have compared the obtained optimal weights with other common weighting methods, namely maximum degree [5], Metropolis-Hasting [12] and best constant [2] weighting methods by evaluating SLEM and comparing convergence time improvements. For this purpose we consider a network with symmetric star topology where 8 similar branches are connected to a central node. In In Fig. 10 the total error in terms of number of iterations over a symmetric star network with 8 path branches of length 10 is presented. The weighting methods, used for achieving the results of Fig. 10 are optimal weights (given in section IV), Maximum degree, Metropolis-Hasting and Best constant weighting methods (as defined in Appendix A). We define the total error as the Euclidean distance of vector of node values from the mean of vector of initial node values . In figures 11, 12, 13 and 14 we have had the same comparison as in Fig. 10 but for other types of branches introduced in section IV. In Fig. 11 Lollipop branches of order , in Fig. 12 Semi-Complete branches of order , in Fig. 13 Ladder branches of order and in Fig. 14 1 for a fixed number of nodes in each branch by choosing semi-complete topology and its corresponding optimal weights (given in section IV), one can achieve the fastest asymptotic mixing rate. Not to mention that faster mixing rate comes with the cost of more edges and connections. palm branch has the minimum number of edges required for the network to remain connected and still mixes faster than path and Lollipop branches. To compare these branches in terms of per step convergence rate, in Fig. 15 total error in terms of number of iterations over a symmetric star network with 8 identical branches is presented. The type of branches considered for the results of Fig. 15 are the same as in table 1. From Fig. 15 it is obvious that Semi-complete and Ladder branches mix faster compared to path, Lollipop and palm branches. Fastest Distributed Consensus averaging Algorithm in sensor networks has received renewed interest recently. In most of the methods proposed so far either numerical or analytical, full knowledge about the network's topology is required. Here in this work, we have solved fastest distributed consensus averaging problem and determined the optimal weights for certain branches of an arbitrary connected network by means of stratification and semidefinite programming. We have shown that the obtained weights are independent of rest of the network and these weights can be used for branches of any connected sensor network. Our approach is based on fulfilling the slackness conditions, where the optimal weights are obtained by inductive comparing of the characteristic polynomials initiated by slackness conditions. Examples and numerical results presented in paper confirm the optimality of obtained weights over other weighting methods. Moreover the obtained weights are optimal for combination of five branches introduced in paper. We believe that the method used for determining optimal weights is powerful and lucid enough to be extended to other types of branches with more general topologies, which is the object of The Metropolis-Hastings weighting method is defined as: where and are the degrees of nodes and , respectively and is the set of immediate neighbors of node . The Maximum degree weighting method is defined as: The best constant weighting method is defined as: In [5] it has been shown that the optimum choice of for best constant weighting method is where denotes the -th largest eigenvalue of and is the Laplacian matrix defined as with as the adjacency matrix of the sensor network's connectivity graph.