Spectral Design of Dynamic Networks via Local Operations

1 Spectral Design of Dynamic Networks via Local Operations V ictor M. Preciado, Member , IEEE, Michael M. Za vlanos, Member , IEEE , and Ali Jadbabaie, Senior Member , IEEE Abstract Motiv ated by the relationship between the eigenv alue spectrum of a network and the behavior of dynamical processes ev olving in it, we propose a distributed iterativ e algorithm in which a group of n autonomous agents self-org anize the structure of their communication network in order to control the network’ s eigenv alue spectrum. In our algorithm, we assume that each agent has only access to a local (‘myopic’) vie w of the netw ork around it and that there is no centralized coordinator . In each iteration, agents of the network perform a decentralized decision process in which agents share limited information about their myopic vision of the network to find the most beneficial edge addition/deletion from a spectral point of view . W e base our approach on a novel distance function defined in the space of eigenv alue spectra that is written in terms of the spectral moments of the Laplacian matrix. In each iteration, agents in the network run a greedy algorithm to find the edge addition/deletion that minimized the spectral distance to the desired spectrum. The spectral distance presents interesting theoretical properties that allow an elegant and ef ficient distributed implementation of the greedy algorithm using distributed consensus. Our distributed algorithm is stable by construction, i.e., locally optimizes the network’ s eigen v alue spectrum, and is shown to perform very well in practice. V .M. Preciado and A. Jadbabaie are with the Department of Electrical and Systems Engineering at the Univ ersity of Pennsylvania, Philadelphia, P A 19104 USA. (e-mail: preciado@seas.upenn.edu; jadbabai@seas.upenn.edu). M.M. Zavlanos is with the Department of Mechanical Engineering at the Stevens Institute of T echnology , Hoboken, NJ 07030 USA. (e-mail: michael.zavlanos@ste v ens.edu). This work was supported by ONR MURI “Next Generation Network Science” and AFOSR “T opological and Geometric T ools for Analysis of Complex Networks”. October 26, 2021 DRAFT 2 I . I N T RO D U C T I O N A wide variety of complex networks composed of autonomous agents are able to display a remarkable lev el of self-organization despite the absence of a centralized coordinator [1], [2]. For example, the intricate structure of many biological, social and economic networks, emerges as the result of local interactions between agents aiming to optimize their local utilities [3]. In most real cases, these agents ha ve only access to myopic information about the structure of the network around them. Despite the limited information accessible to each agent, most of these “self-engineered” networks are able to efficiently satisfy their functional requirements. The beha vior of man y netw orked dynamical processes, such as information spreading, synchro- nization, or decentralized coordination, is directly related to the netw ork eigen v alue spectra [4]. In particular , the spectrum of the Laplacian matrix of a netw ork plays a ke y role in the analysis of synchronization in networks of nonlinear oscillators [5], [6], as well as in the behavior of many distributed algorithms [7], and decentralized control problems [8], [9]. Motiv ated by the relationship between a network’ s eigen v alue spectrum and the behavior of dynamical processes e v olving in it, we propose a distributed iterativ e algorithm in which a group of autonomous agents self-org anize the structure of their communication network in order to control the netw ork’ s eigen value spectrum. The ev olution of the graph is ruled by a decentralized decision process in which agents share limited information about their myopic vision of the network to decide which network adjustment is most beneficial globally . Optimization of network eigen values has been studied by sev eral authors in both centralized [10], [11], [12] and decentralized settings [13]. In these papers, the objecti ve is usually to find the weights associated to the edges of a gi ven network in order to optimize eigen values of particular rele v ance, such as the Laplacian spectral gap or spectral radius (i.e., the second smallest and largest eigenv alues of the Laplacian matrix, respecti vely). In contrast to existing techniques, we propose a distributed frame work where we control the so-called spectral moments of the Laplacian matrix by iterati vely modifying the structure of the network. W e sho w that the benefits of controlling the spectral moments, instead of indi vidual eigen values, lies in lo wer computational cost and elegant distributed implementation. The performance of our algorithm is illustrated in nontri vial computer simulations. The rest of this paper is org anized as follo ws. In Section II, we re vie w terminology and October 26, 2021 DRAFT 3 formulate the problem under consideration. In Section III, we introduce a decentralized algorithm to compute the spectral moments of the Laplacian matrix from myopic views of the network’ s structure. W e also introduce a nov el perturbation technique to efficiently compute the effect of adding or removing edges on the spectral moments. Based on these results, in Section IV, we propose a distributed algorithm in which a group of autonomous agents modify their network of interconnections to control of the spectral moments of a network to wards desired values. Finally , in Section V, we illustrate our approach with sev eral computer simulations. I I . P R E L I M I N A R I E S & P R O B L E M D E FI N I T I O N A. Eigen values of Graphs and their Spectral Moments Let G = ( V , E ) be an undirected graph, where V = { 1 , . . . , n } denotes a set of n nodes and E ⊆ V × V denotes a set of e undirected edges. If ( i, j ) ∈ E , we call nodes i and j adjacent (or first-neighbors), which we denote by i ∼ j . W e define the set of first-neighbors of a node i as N i = { j ∈ V : ( i, j ) ∈ E } . The de gree d i of a verte x i is the number of nodes adjacent to it, i.e., d i = |N i | . 1 An undirected graph is called simple if its edges are unweighted and it has no self-loops 2 . A graph is weighted if there is a real number associated with e very edge. More formally , a weighted graph H can be defined as the triad H = ( V , E , W ) , where V and E are the sets of nodes and edges in H , and W = { w ij ∈ R , for all ( i, j ) ∈ E } is the set of (possibly negati ve) weights. Graphs can be algebraically represented via matrices. The adjacency matrix of a simple graph G , denoted by A G = [ a ij ] , is an n × n symmetric matrix defined entry-wise as a ij = 1 if nodes i and j are adjacent, and a ij = 0 otherwise. Gi ven a weighted, undirected graph H , the weighted adjacency matrix is defined by W H = [ w ij ] , where w ij is the weight associated to edge ( i, j ) ∈ E and w ij = 0 if i is not adjacent to j . W e define the de gree matrix of a simple graph G as the diagonal matrix D G = diag ( d i ) . W e define the Laplacian matrix L G (also kno wn as combinatorial Laplacian, or Kirchhoff matrix) of a simple graph as L G = D G − A G . For simple graphs, L G is a symmetric, positiv e semidefinite matrix, which we denote by L G  0 [14]. Thus, L G has a full set of n real and orthogonal eigen vectors with real nonnegati ve eigen values 0 = λ 1 ≤ λ 2 ≤ ... ≤ λ n . 1 W e define by | X | the cardinality of the set X . 2 A self-loop is an edge of the type ( i, i ) . October 26, 2021 DRAFT 4 Furthermore, the tri vial eigen value λ 1 = 0 of L G always admits a corresponding eigen vector v 1 = (1 , 1 , ..., 1) T . The algebraic multiplicity of the trivial eigen value is equal to the number of connected components in G . The smallest and largest nontrivial eigen values of L G , λ 2 and λ n , are called the spectral gap and spectral radius of L G , respectiv ely . Gi ven an undirected (possibly weighted) graph G , we denote its Laplacian spectrum by S ( G ) = { λ 1 , ..., λ n } , and define the k -th Laplacian spectral moment of G as, [14]: m k ( G ) , 1 n n X i =1 λ k i . (1) The following theorem states that an eigen value spectrum is uniquely characterized by a finite sequence of moments: Theor em 2.1: Consider two undirected (possibly weighted) graphs G 1 and G 2 with Laplacian eigen value spectra S ( G 1 ) = { λ (1) 1 ≤ ... ≤ λ (1) n } and S 2 ( G 1 ) = { λ (2) 1 ≤ ... ≤ λ (2) n } . Then, λ (1) i = λ (2) i for all 1 ≤ i ≤ n if and only if m k ( G 1 ) = m k ( G 2 ) for 0 ≤ k ≤ n − 1 . Pr oof: In the Appendix. In the rest of this paper we will focus on the spectrum of the graph Laplacian matrix L G and its spectral moments, which we denote by m k ( L G ) . In this case, Theorem 2.1, implies that the Laplacian spectral moment of a graph on n nodes is uniquely characterize by the sequence of n − 1 spectral moments ( m k ( L G )) n − 1 k =1 . It is worth remarking that two nonisomorphic 3 graphs G 1 and G 2 can present the same eigen value spectrum [16], in which case we say that G 1 and G 2 are isospectral. In other words, the eigenv alue spectrum of a graph is not enough to characterize its structure. On the other hand, as we shall show in Section III, there are man y interesting connections between the structural features of a graph G and the spectral moments of its Laplacian matrix, m k ( L G ) . B. Local Structural Pr operties of Graphs In this section we define a collection of structural properties that are important in our deriv a- tions. A walk of length k from node i 1 to node i k +1 is an ordered sequence of nodes ( i 1 , i 2 , ..., i k +1 ) such that i j ∼ i j +1 for j = 1 , 2 , ..., k . One says that the walk touches each of the nodes that 3 T wo simple graphs G 1 and G 2 with adjacency matrices A G 1 and A G 2 are isomorphic if there exists a permutation matrix P n such that A G 1 = P n A G 2 P T n . October 26, 2021 DRAFT 5 comprises it. If i 1 = i k +1 , then the walk is closed. A closed walk with no repeated nodes (with the exception of the first and last nodes) is called a cycle . Giv en a walk p = ( i 1 , i 2 , ..., i k +1 ) in a weighted graph H with weighted adjacency matrix W H = [ w ij ] , we define the weight of the walk as, ω ( p ) = w i 1 i 2 w i 2 i 3 ...w i k i k +1 . W e no w define the concept of local neighborhood around a node. Let δ ( i, j ) denote the distance between two nodes i and j (i.e., the minimum length of a walk from i to j ). By con vention, we assume that δ ( i, i ) = 0 . W e define the r -th order neighborhood G i,r = ( N i,r , E i,r ) around a node i as the subgraph G i,r ⊆ G with node-set N i,r , { j ∈ V : δ ( i, j ) ≤ r } , and edge- set E i,r = { ( v , w ) ∈ E s.t. v , w ∈ N i,r } . Giv en a set of k nodes K ⊆ V , we define G K as the subgraph of G with node-set V ( G K ) = K and edge-set E ( G K ) = { ( i, j ) ∈ E s.t. i, j ∈ K } . W e define L G ( K ) as the k × k submatrix of L G formed by selecting the rows and columns of L G index ed by K . In particular , we define the Laplacian submatrix L i,r , L G ( N i,r ) . W e say that a structural measurement is local with a certain radius r if it can be computed from the set of local neighborhoods {G i,r , i = 1 , ..., n } . For example, the degree sequence of G is a local structural measurement (with radius 1 ), since we can compute the degree of each node i from the neighborhood N i, 1 . In contrast, the eigen value spectrum of the Laplacian matrix is not a local property , since we cannot compute the eigen values unless we know the complete graph structure. One of the main contrib utions of this paper is to propose a nov el methodology to extract global information regarding the Laplacian eigen value spectrum from the set of local neighborhoods. C. Spectral Metrics and Pr oblem Definition As discussed in Section I, our goal is to propose a distributed algorithm to control the eigen- v alue spectrum of a multi-agent network, via its spectral moments, by iterati vely adding/removing edges in the network; see Section II-B. For this, we define the follo wing spectral distance between two graphs G a and G b , with spectra S a = { λ ( a ) i } n i =1 and S b = { λ ( b ) i } n i =1 , as 4 d M ( S a , S b ) = n − 1 X k =1  m k ( G a ) 1 /k − m k ( G b ) 1 /k  2 . (2) 4 Note that d M is a distance in the space of eigen value spectra, but not in the space of graphs, since we can find nonisomorphic graphs that are isospectral. October 26, 2021 DRAFT 6 According to Theorem 2.1, two graphs are isospectral if their first n − 1 spectral moments coincide; thus, d M in (2) is in fact a distance function in the space of graph spectra. W e further define the spectral pseudometric 5 : d K ( S a , S b ) = K X k =1  m k ( G a ) 1 /k − m k ( G b ) 1 /k  2 , (3) for K < n − 1 . The benefit of using the spectral pseudodistance versus other spectral distances is due to the fact that, as we shall sho w in Section III, we can ef ficiently compute the first K spectral moments of the Laplacian matrix from the set of local Laplacian submatrices with radius b K / 2 c , i.e., { L i, b K/ 2 c , i ∈ V } . In other words, assuming that each agent has access to the Laplacian submatrix associated to its neighborhood with radius r , we shall sho w how to distributedly compute the first 2 r + 1 Laplacian moments of the complete graph L G . W ith the notation defined abov e, we can rigorously state the problem addressed in this paper as follows: Pr oblem 1: Giv en a desired spectrum S ∗ = { λ ∗ i } n i =1 , find a simple graph G ∗ such that its Laplacian eigen value spectrum, denoted by S ( G ∗ ) , minimizes d K ( S ( G ∗ ) , S ∗ ) . Finding a simple graph with a giv en (feasible 6 ) eigen v alue spectrum is, in general, a hard combinatorial problem, e ven in a centralized setting. In this paper , we propose a distributed approximation algorithm to find a graph with a spectrum ‘close to’ S ∗ in the d K pseudometric. In our algorithm, a group of agents located at the nodes of a network iterati vely add/remove edges to dri ve the network’ s eigen value spectrum towards the desired spectrum. In each iteration, the set of agents perform a decentralized decision process to find the most beneficial edge addition/deletion from the point of view of the global eigen v alue spectrum. T o formulate our algorithm, we first need to define the edit distance d E ( G a , G b ) between two graphs G a and G b , which is the minimum number of edge additions plus edge deletions to transform G a into a graph that is isomorphic to G b . T o approximately solve Problem 1 in a distributed way , we propose the following iteration to determine a sequence of graphs {G ( t ) } t ≥ 0 , 5 A pseudometric is a generalization of distance in which two distinct points (in our case, two distinct spectra) can have zero distance. 6 W e say that an eigen v alue spectrum is feasible if there is a simple graph whose Laplacian matrix presents that spectrum. October 26, 2021 DRAFT 7 starting from any graph G 0 : G ( t + 1) , arg min G d K ( S ( G ) , S ∗ ) s.t. d E ( G ( t ) , G ) = 1 , λ 2 ( G ) > 0 . (4) The resulting sequence of spectra { S ( G ( t )) } t ≥ 0 con ver ges to S ∗ as t gro ws. The constraint d E ( G ( t ) , G ( t + 1)) = 1 enforces only single edge additions or deletions at each iteration, while the requirement λ 2 ( G ( t )) > 0 enforces graph connecti vity at all times, which will be necessary for the distributed implementation in Section IV. Note that the Iteration (4) typically requires global knowledge of the network structure. In this paper , we propose a computationally ef ficient, distributed algorithm in which agents in the network solve (4) using only their local, myopic views of the network structure. In particular , we shall show how the set of agents can compute, in a distributed fashion, the ef fect of an edge addition/deletion on the first 2 r + 1 Laplacian moments. Furthermore, we shall also propose a distributed algorithm to find the edge addition/deletion that minimizes the resulting v alue of the spectral pseudodistance to S ∗ . Before we describe the implementation details in Section IV, we first provide the theoretical foundation for our approach in Section III. Remark 2.1 (Con ver gence): Sev eral remarks are in order . First, note that it is not always possible to find a simple graph that exactly match a giv en eigen value spectrum. Second, the spectral pseudometric d K ( S ( G ) , S ∗ ) may present multiple minima for a giv en S ∗ . These minima could correspond, for example, to se veral isospectral graphs matching the desired spectrum S ∗ [16]. Therefore, iteration (4) may con ver ge to different isospectral graphs depending on the initial condition G 0 . Third, iteration (4) finds the most beneficial edge addition/deletion in each time step, hence, this greedy approach may get trapped in a local minimum. In practice, we observe that in our numerical simulations the spectra of these local minima are remarkably close to those of the desired spectrum. I I I . M O M E N T - B A S E D A N A LY S I S O F T H E L A P L AC I A N M A T R I X In this section, we use tools from algebraic graph theory to compute the spectral moments of the Laplacian matrix of G when only the set of local Laplacian submatrices { L i,r , i ∈ V } is av ailable. As a result of our analysis, we propose a decentralized algorithm to compute a truncated sequence of Laplacian spectral moments via a single distributed averaging. Furthermore, we also October 26, 2021 DRAFT 8 present an efficient approach to compute the effect of adding or deleting an edge in the Laplacian spectral moments of the graph. P articularly useful in our deri vations will be the follo wing result from algebraic graph theory [14]: Lemma 3.1: Let H = ( V , E , W ) be a weighted graph with weighted adjacency matrix W H = [ w ij ] . Then  W k H  ii = X p ∈ P i,k ( H ) ω ( p ) , where P i,k ( H ) is the set of closed walks of length k starting and finishing at node i in the weighted graph H . A. Algebr aic Analysis of Structur ed Matrices Consider the symmetric Laplacian matrix L G of a simple graph G = ( V , E ) . W e denote by G i,r = ( N i,r , E i,r ) the neighborhood of radius r around node i and define the local Laplacian submatrix L i,r , as the submatrix L G ( N i,r ) , formed by selecting the rows and columns of L G index ed by the set of nodes N i,r . By con vention, we associate the first row and column of the submatrix L i,r with node i ∈ V , which can be done via a simple permutation of ro ws and columns. 7 For a simple graph G with Laplacian matrix L G , we define L ( G ) as the weighted graph whose adjacency matrix is equal to L G . In other words, L ( G ) has edges with weight − 1 for ( i, j ) ∈ E ( G ) , 0 for ( i, j ) 6∈ E ( G ) , and d i for all self-loops ( i, i ) , i ∈ V ( G ) . W e also define H i,r as the weighted subgraph of L ( G ) with node set N i,r , containing all the edges of L ( G ) connecting pairs of nodes in N i,r (including self-loops). Notice that, according to this definition, the weighted adjacency matrix of H i,r is equal to L i,r . In this paper , we assume that each agent in the network kno ws the structure of its local neighborhood G i,r , for a fixed r . Therefore, agent i has access to the local Laplacian submatrix L i,r . The follo wing results allo ws us aggregate information from the set of local Laplacian submatrices, { L i,r } i ∈V , to compute a sequence of spectral moments of the (global) Laplacian matrix L G . Theor em 3.2: Consider a simple graph G with Laplacian matrix L G . Then, for a gi ven radius 7 Notice that permuting the ro ws and columns of the Laplacian matrix does not change the topology of the underlying graph. October 26, 2021 DRAFT 9 r , the Laplacian spectral moments can be written as m k ( L G ) = 1 n n X i =1  L k i,r  11 , (5) for k ≤ K = 2 r + 1 . Pr oof: Since the trace of a matrix is the sum of its eigen v alues, we can expand the k -th spectral moment of the Laplacian matrix as follows: m k ( L G ) = 1 n T race  L k G  = 1 n n X i =1  L k G  ii Therefore, since L G is the weighted adjacency matrix of the Laplacian graph L ( G ) , we hav e (from Lemma 3.1) m k ( L G ) = 1 n n X i =1 X p ∈ P i,k ( L ( G )) ω ( p ) , (6) where the weights ω ( p ) are summed over the set of closed walks of length k starting at node i in the weighted graph L ( G ) . For a fixed value of k , closed walks of length k in L ( G ) starting at node i can only touch nodes within a certain distance r ( k ) of i , where r ( k ) is a function of k (see Fig. 1). In particular , for k e ven (resp. odd), a closed walk of length k starting at node i can only touch nodes at most k / 2 (resp. b k / 2 c ) hops away from i . Therefore, closed walks of length k starting at i are always contained within the neighborhood of radius b k / 2 c . In other words, the neighborhood G i,r of radius r contains all closed walks of length up to 2 r + 1 starting at node i . Therefore, for k ≤ 2 r + 1 , we have that X p ∈ P i,k ( L ) ω ( p ) = X p ∈ P 1 ,k ( H i,r ) ω ( p ) , where H i,r is the weighted graph whose adjacenc y matrix is equal to the local Laplacian submatrix L i,r (notice that, by con vention, we associate the first row and column of L i,r with node i ). Therefore, according to Lemma 3.1, we hav e X p ∈ P 1 ,k ( H ( L i,r )) ω ( p ) =  L k i,r  1 , 1 . (7) Then, substituting (7) into (6), we obtain the statement of our Theorem. October 26, 2021 DRAFT 10 Fig. 1. Cycles C 6 and C 7 , of lengths 6 and 7 , in a neighborhood of radius 3 around node i . Remark 3.1 (Distributed computation of spectral moments): Since ev ery node i has access to its local neighborhood G i,r , it is possible to compute the first 2 r + 1 moments via a simple distributed av eraging of the quantities {  L k i,r  1 , 1 } i ∈V , [7]. This av eraging ef ficiently aggregates local pieces of local structural information (described by the local Laplacian submatrices) to produce a truncated sequence of spectral moments of the (global) Laplacian matrix. This is an useful result for the analysis of complex networks for which retrieving the complete structure of the network can be very challenging (in many cases, not e ven possible). Based on Theorem 3.2, we propose a distrib uted algorithm to compute a sequence of 2 r + 1 spectral moments of L G from local submatrices L i,r , as described in Algorithm 1. Note that, computing the spectral moments via (5) is much more efficient than computing these moments via an explicit eigen v alue decomposition for many real-world networks. In most real applications, the Laplacian matrix representing the network structure is a sparse graph for which the number of nodes in the neighborhood N i,r is very small compared to n , for moderate values of r . B. Moment-Based P erturbation Analysis In this section, we use spectral graph theory to compute the effect of adding or deleting an edge on the spectral moments of the Laplacian matrix. T raditionally , the ef fect of a matrix October 26, 2021 DRAFT 11 Algorithm 1 Decentralized moment computation Require: Local Laplacian submatrices L i,r for all nodes i ∈ V ; 1: Each node i ∈ V computes a vector µ i ,  µ i, 1 , µ i, 2 , µ i, 3 , ..., µ i, 2 r +1  T , where µ i,k ,  L k i,r  1 , 1 ; 2: Using distributed av eraging, compute the following vector of av erages: m 2 r +1 ( L G ) , 1 n n X i =1 µ i = 1 n n X i =1  0 , µ i, 2 , µ i, 3 , ..., µ i, 2 r +1  T = ( m 1 ( L G ) , m 2 ( L G ) , ..., m 2 r +1 ( L G )) T . perturbation on the eigen v alue spectrum is analyzed using eigen value perturbation techniques [17]. In particular , the effect of adding a ‘small’ perturbation matrix δ W to an n × n symmetric matrix W with eigen value spectrum { σ k } n k =1 can be approximated, in the first-order , by [17] e σ k − σ k ≈ u T k δ W u k , where u k is the eigen vector of W associated with the eigen value σ k , and { e σ k } n k =1 is the eigen value spectrum of the perturbed matrix W + δ W . In the case of the Laplacian matrix, the perturbation matrix δ W corresponding to the addition of an edge ( i, j ) can be written as, δ W = ( e i − e j ) ( e i − e j ) T , where e i is the unit vector in the direction of the i -th coordinate. W e denote by G + ( i, j ) the graph resulting of adding edge ( i, j ) to G , and { e λ k } n k =1 is the Laplacian spectrum of G + ( i, j ) . Therefore, adding edge ( i, j ) perturbs the eigen values of the Laplacian matrix as follo ws: e λ k − λ k ≈ v T k ( e i − e j ) ( e i − e j ) T v k = ( v k,i − v k,j ) 2 , where v k is the eigen vector of L G associated to λ k , and v k,j is its j -th component. Hence, the resulting spectral radius can be approximated as e λ 1 ≈ λ 1 + ( v 1 ,i − v 1 ,j ) 2 . Therefore, computing the ef fect of an edge addition on the spectral radius using traditional perturbation techniques requires computation of the dominant eigen v alue and eigen vector of L G , October 26, 2021 DRAFT 12 which is computationally expensi ve for very large graphs. As an alternati ve to the traditional analysis, we propose a nov el approach, based on algebraic graph theory , to compute the effect of structural perturbation on the spectral moments of the Laplacian matrix L G without e xplicitly computing the eigen values or eigen vectors of L G . Furthermore, our approach can be ef ficiently implemented in a fully decentralized manner . In our deri v ations, we use the follo wing result from algebraic graph theory: Lemma 3.3: Let H = ( V , E , W ) be a weighted graph with weighted adjacency matrix W H = [ w ij ] . Then m k ( W H ) = 1 n X p ∈ P k ( H ) ω ( p ) , (8) where P k ( H ) is the set of closed walks of length k in the weighted graph H . Pr oof: This lemma is a consequence of Lemma 3.1. Specifically , we hav e that m k ( W ( H )) = 1 n T race  W ( H ) k  = 1 n X i ∈V h W ( H ) k i i,i = 1 n X i ∈V X p ∈ P i,k ( H ) ω ( p ) = 1 n X p ∈ P k ( H ) ω ( p ) , where P k ( H ) , ∪ i ∈V P i,k ( H ) is the set of all closed walks of length k in H (for any starting node i ∈ V ). C. P erturbation on the Spectral Moments Consider a simple graph G with Laplacian matrix L G . W e denote by G + ( i, j ) (resp. G − ( i, j ) ) the graph resulting from adding (resp. remo ving) edge ( i, j ) to (resp. from) G . Consider the sets of nodes N i,r and N j,r being within a radius r from node i and node j , respecti vely . Let us define the follo wing submatrices index ed by the set of nodes in N i,r ∪ N j,r : U r, ( i,j ) , L G ( N i,r ∪ N j,r ) , U + r, ( i,j ) , L G +( i,j ) ( N i,r ∪ N j,r ) , U − r, ( i,j ) , L G − ( i,j ) ( N i,r ∪ N j,r ) . October 26, 2021 DRAFT 13 The follo wing lemma allo ws us to ef ficiently compute the increment (resp. decrement) in the Laplacian spectral moments of G due to the addition (resp. removal) of edge ( i, j ) : Theor em 3.4: Gi ven a simple graph G with Laplacian matrix L G , the increment (decrement) in the k -th Laplacian spectral moment of a graph G due to the addition or deletion of an edge ( i, j ) can be written as m k  L G ± ( i,j )  − m k ( L G ) = 1 n  T race  U ± r, ( i,j )  k − T race  U r, ( i,j )  k  , (9) for k ≤ 2 r + 1 . Pr oof: Consider the weighted Laplacian graphs of L G , L G +( i,j ) , and L G − ( i,j ) , which we denote by H , L ( G ) , H + , L ( G + ( i, j )) and H − , L ( G − ( i, j )) , respecti vely . (By definition, the adjacenc y matrices of the Laplacian graphs are the Laplacian matrices of the graphs.) Then, according to Lemma 3.3, we have that the k -th spectral moments m k ( L G ) , m k  L G +( i,j )  and m k  L G − ( i,j )  can be written as weighted sums over the sets of all closed walks of length k in H , H + , and H − , as follo ws, m k ( L G ) = 1 n X p ∈ P k ( H ) ω ( p ) , m k  L G ± ( i,j )  = 1 n X p ∈ P k ( H ± ) ω ( p ) . W e define P ( i,j ) k,r ( H ) , P ( i,j ) k,r ( H + ) , and P ( i,j ) k,r ( H − ) as the sets of closed walks of length k in, respecti vely , H , H + , and H − visiting only nodes in the set N i,r ∪ N j,r . Then, we can split the summation in (8) for the Laplacian matrices, as follows: m k ( L G ) = 1 n X p ∈ P ( i,j ) k,r ( H ) ω ( p ) + 1 n X p ∈ P k \ P ( i,j ) k,r ( H ) ω ( p ) , (10) m k  L G ± ( i,j )  = 1 n X p ∈ P ( i,j ) k,r ( H ± ) ω ( p ) + 1 n X p ∈ P k \ P ( i,j ) k,r ( H ± ) ω ( p ) . (11) Notice that, as we illustrated in Fig. 1, none of the closed walk of length k ≤ 2 r + 1 touching node i (resp. node j ) can leav e the neighborhood N i,r (resp. N j,r ). Therefore, all closed walks of length k ≤ 2 r + 1 touching either node i or j (or both) are contained 8 in N i,r ∪ N j,r . As a consequence, none of the closed walks in P k \ P ( i,j ) k ( H ) or P k \ P ( i,j ) k ( H ± ) touches node i or 8 W e say that a walk is contained in a set of nodes N if it only touches nodes in N . October 26, 2021 DRAFT 14 j . Since addition/remov al of edge ( i, j ) does not influence those walks not touching i or j , we hav e that X p ∈ P k \ P ( i,j ) k,r ( H ) ω ( p ) = 1 n X p ∈ P k \ P ( i,j ) k,r ( H ± ) ω ( p ) . Thus, from (10) and (11) we hav e m k  L G ± ( i,j )  − m k ( L G ) = 1 n X p ∈ P ( i,j ) k,r ( H ± ) ω ( p ) − 1 n X p ∈ P ( i,j ) k,r ( H ) ω ( p ) . (12) Since P ( i,j ) k,r ( H ) is the set of all closed walks of length k in H visiting nodes in the set N i,r ∪ N j,r , we can apply Lemma 3.3 to obtain 1 n X p ∈ P ( i,j ) k,r ( H ) ω ( p ) = m k ( L G ( N i,r ∪ N j,r )) = 1 n T race  U r, ( i,j )  k . (13) Similarly , for P ( i,j ) k,r  H ±  , we obtain 1 n X p ∈ P ( i,j ) k,r ( H ± ) ω ( p ) = 1 n T race  U ± r, ( i,j )  k . (14) Finally , substituting (13) and (14) in (12) provides us with the statement of our theorem. Remark 3.2 (Computational cost): According to Lemma 3.4, we can compute the increment or decrement in the Laplacian spectral moments (up to order 2 r + 1 ) by computing T race ( U r, ( i,j ) ) k and T race ( U ± r, ( i,j ) ) k . Notice that the sizes of U r, ( i,j ) and U ± r, ( i,j ) are |N i,r ∪ N j,r | , which is usually small for large sparse graphs (and moderate r ). I V . D E C E N T R A L I Z E D C O N T R O L O F S P E C T R A L M O M E N T S In this section, we integrate the results dev eloped in Section III with a novel technique for distributed connectivity verification of edge additions or deletions in order to obtain a distributed solution to Problem 1 in the form of (4), as discussed in Section II-C. This relies on the assumption that an agent at node i is able to communicate at time slot t with all the agents in its first-order neighborhood N i, 1 ( t ) only . 9 Moreov er , we also assume that every agent has only a myopic view of the network structure. This means that at time slot t agent i ∈ V only kno ws the 9 Notice that, since G ( t ) is time-dependent, so are the neighborhoods G i,r ( t ) = ( N i,r ( t ) , E i,r ( t )) . October 26, 2021 DRAFT 15 topology of the neighborhood G i,r ( t ) , within a particular radius r . This limits the set of possible actions that ev ery agent i can take in ev ery step of the iteration (4), to be local edge additions of non-edges ( i, j ) 6∈ E ( t ) in G i,r ( t ) or local edge deletions of edges ( i, j ) 6∈ E ( t ) in G i, 1 ( t ) . In what follows, it will be useful to predetermine the master node for each edge ( i, j ) ∈ E ( t ) , which can be arbitrarily chosen from the set of nodes { i, j } . The notion of master node is useful to coordinate actions in our decentralized algorithm. The agent located at the master node of ( i, j ) is the only one with the authority to decide if edge ( i, j ) is deleted. W e denote by D i ( t ) the set of edges having node i as its master . In our simulations, we choose this set to be D i ( t ) , { ( i, j ) ∈ E ( t ) | i > j } . 10 Similarly , it is useful to predefine a master node for each nonedge 11 ( i, k ) 6∈ E ( t ) . The agent located at the master node of the nonedge is the only one with the authority to decide if edge ( i, k ) is added to the network. W e denote by A i ( t ) the set of nonedges having node i as its master . In our case, we define this set as A i ( t ) , { ( i, k ) 6∈ E ( t ) | k ∈ N i,r ( t ) and i > k } , where we limit node k to be in N i,r ( t ) , since we are only considering local edge additions. A. Connectivity-Pr eserving Edge Deletions In a centralized framework, network connectivity can be inferred from the number of trivial eigen values of the Laplacian matrix. Howe ver , when only local network information is av ailable, only sufficient conditions for connectivity can be verified. One such condition is the requirement that |N j ( t ) ∩ N i,r ( t ) | > 1 , which can be locally verified by agent i with knowledge of only G i,r . Since this condition is only suf ficient but not necessary for connectivity preservation, we need a mechanism to check connecti vity for those edges in the set C ( t ) = { ( i, j ) ∈ E ( t ) : |N j ( t ) ∩ N i,r ( t ) | = 1 } . of critically connected edges, for which the sufficient condition does not hold. The proposed mechanism relies on a the concept of a maximum consensus. In particular , consider a graph G ( t ) = ( V , E ( t )) at time t ≥ 0 and for any ( i, j ) ∈ C ( t ) associate a scalar v ariable x ( i,j ) k ( s ) ∈ R with ev ery node k ∈ V . Assume that the v ariables x ( i,j ) k ( s ) are randomly 10 Since the indices of all nodes in the network are distinct natural numbers, this definition results in a unique assignment. 11 A pair of nodes ( i, k ) is a nonedge of G if ( i, k ) 6∈ E ( G ) . October 26, 2021 DRAFT 16 initialized and run the follo wing maximum consensus update x ( i,j ) k ( s + 1) = max l ∈N k, 1 −{ i,j } { x ( i,j ) l ( s ) } (15) on the graph G ( t ) − ( i, j ) obtained by virtually disabling the link ( i, j ) via blocking commu- nication through it. Then, the network G ( t ) − ( i, j ) is almost surely connected if and only if the v ariables x ( i,j ) k ( s ) for all k ∈ V con ver ge to the common value max k x ( i,j ) k (0) . Note that con ver gence in this case takes place in finite time that is upper bounded by the diameter of the network [18]. This idea can be extended to simultaneous verification of multiple link deletions in C ( t ) . In fact, since ev ery edge is assigned a unique master agent, we can partition the set C ( t ) in to |V | disjoint subsets C ( t ) ∩ D i ( t ) for all i ∈ V . This allo ws us to define the sets P ki = { x ( i,j ) k ( s ) : ( i, j ) ∈ C ( t ) ∩ D i ( t ) } containing all v ariables of agent k that ha ve as a master agent i . A simple schematic of the proposed construction is shown in the following table: C ∩ D 1 C ∩ D 2 . . . 1 P 11 = { x (1 ,j ) 1 } P 12 = { x (2 ,j ) 1 } . . . 2 P 21 = { x (1 ,j ) 2 } P 22 = { x (2 ,j ) 1 } . . . . . . . . . . . . . . . Note that the second subscript i in the set P ki denotes that master agent for the variables contained in P ki . Therefore, agent k initializes only those variables in the set P kk . Finally , stack all variables in the set P ki in a vector x ki ( s ) ∈ R |C ( t ) ∩D i ( t ) | and denote by [ x ki ( s )] ( i,j ) the scalar state associated with edge ( i, j ) ∈ C ( t ) ∩ D i ( t ) . Using the notation defined abov e, we can simultaneously verify connectivity for all edges in C ( t ) by a high-dimensional consensus. For this, e very agent k initializes randomly all vectors x ki (0) ∈ R |C ( t ) ∩D i ( t ) | for all masters i ∈ V and updates the vectors x ki ( s ) ∈ R |C ( t ) ∩D i ( t ) | as follows: Case I : If k is not a neighbor of the master agent i , i.e., if k 6∈ N i , then it updates the vectors x ki ( s ) as x ki ( s + 1) := max l ∈N k ( t ) { x ki ( s ) , x li ( s ) } , (16) where the maximum is applied elementwise on the vectors. Case II : If k is a neighbor of the master agent i , i.e., if k ∈ N i , then it virtually remov es link ( k , i ) and updates the entry [ x ki ( s )] ( k,i ) as [ x ki ( s + 1)] ( k,i ) := max l ∈N k ( t ) \{ i }  [ x ki ( s )] ( k,i ) , [ x li ( s )] ( k,i )  , (17) October 26, 2021 DRAFT 17 Algorithm 2 Connecti vity verification Require: x ij (0) ∈ R |C ( t ) ∩D j ( t ) | for all i, j ∈ V ; 1: f or s = 1 : τ do 2: Update x ij ( s + 1) by (16)–(19); 3: end for 4: Compute S i ( t ) by (20); while for all other links ( j, i ) ∈ C ( t ) ∩ D i ( t ) with j 6 = k it updates the entries [ x ki ( s )] ( j,i ) as [ x ki ( s + 1)] ( j,i ) := max l ∈N k ( t )  [ x ki ( s )] ( j,i ) , [ x li ( s )] ( j,i )  . (18) Case III : For the v ariables x kk ( s ) for which k is the master , it virtually remov es the links ( k , j ) ∈ C ( t ) ∩ D k ( t ) and updates the entries [ x kk ( s )] ( k,j ) as [ x kk ( s + 1)] ( k,j ) := max l ∈N k ( t ) \{ j }  [ x kk ( s )] ( k,j ) , [ x lk ( s )] ( k,j )  . (19) The high-dimensional consensus defined by (16)–(19) con verges in a finite time τ > 0 [18]. When this happens, node k requests the entries [ x ik ( τ )] ( k,i ) from all its neighbors i ∈ N k ( t ) for which ( k , i ) ∈ C ( t ) ∩ D k ( t ) and compares them with [ x kk ( τ )] ( k,i ) . Since, violation of connectivity due to deletion of ( k , j ) would result in nodes k and i being in dif ferent connected components, if [ x kk ( s )] ( k,i ) = [ x ik ( s )] ( k,i ) then the network G ( t ) − ( k , i ) would still remain connected. Hence, we can define the set S k ( t ) ,  ( k , i ) ∈ C ( t ) ∩ D k ( t ) : [ x kk ( τ )] ( k,i ) = [ x ik ( τ )] ( k,i )  , (20) containing the edges in C ( t ) ∩ D k ( t ) whose remov al does not disconnect the network. B. Most Beneficial Local Action T o solve Problem 1 via the iterati ve algorithm proposed in (4), we need to add or delete an edge ( i, j ) that minimizes the spectral pseudometric d K ( S ( G ± ( i,j ) ( t )) , S ∗ ) at ev ery time step t . For this, let S D i ( t ) , d K ( S ( G ( t )) , S ∗ ) denote a local copy of the spectral distance of the graph G ( t ) that is av ailable to agent i , so that initially S D i (0) = S D (0) for all agents i ∈ V . The quantity S D (0) can be computed in a distributed way by means of distributed av eraging, according to Theorem 3.2. Then, the key idea is that every master agent i computes the spectral October 26, 2021 DRAFT 18 distance S D ± ( i,j ) ( s ) , d K  S  G ± ( i,j ) ( s )  , S ∗  resulting from adding a link ( i, j ) ∈ A i ( t ) or deleting a link ( i, j ) ∈ S i ( t ) . Computation of this distance relies on Theorem 3.4 and requires that agent i has knowledge of the structure of its neighborhoods G i,r only , for r = b K / 2 c . For all possible local edge additions or deletions, master agent i determines the most beneficial one ( i, j ∗ i ( t )) , argmin ( i,j ) ∈A i ( t ) ∪S i ( t )  S D ± ( i,j ) ( t ) − S D i ( t )  . Note that the minimization above may result in multiple edges having the same optimal value. Such ties can be broken via, e.g., a coin toss. Then, the largest decrease in the error associated with the most beneficial edge ( i, j ∗ i ( t )) becomes: S D i ( t ) ,    S D ± ( i,j ∗ i ) ( t ) , if min ( i,j ) ∈A i ( t ) ∪S i ( t ) { S D ± ( i,j ) ( t ) − S D i ( t ) } ≤ 0 D , otherwise . for a large constant D > 0 . In other words, S D i ( t ) is nontri vially defined only if the exists a link adjacent to node i that if added or deleted decreases the error function S D ( t ) . Otherwise, a large value D > 0 is assigned to S D i ( t ) to indicate that this action is not beneficial to the final objecti ve. Finally , for each node i , we initialize the state vector b i (0) , [ i j ∗ i ( t ) S D i ( t ) m ( i, j ∗ i ( t ))] T , containing the best local action ( i, j ∗ i ( t )) , the associated spectral pseudodistance S D i ( t ) , and the vector of resulting moments m ( i, j ∗ i ( t )) ,  m k  S  G ± ( i,j ∗ i ) ( t )  K k =1 . In the follo wing section, we discuss how to compare all local actions b i ( t ) for all nodes i ∈ V to find the best global action that minimizes the spectral pseudometric. C. F r om Local Information to Global Action In order to obtain the overall most beneficial action, all local actions need to be propagated in the network and compared against each other . For this, e very agent i communicates with its neighbors and updates its desired action b i ( s ) with the action b j ( s ) corresponding to the node j that contains the smallest distance to the target moments [ b j ( s )] 3 , S D i ( t ) , i.e., b i ( s + 1) = b j ( s ) , where j = argmin k ∈N i ( t ) { [ b i ( s )] 3 , [ b k ( s )] 3 } . October 26, 2021 DRAFT 19 Algorithm 3 Globally most beneficial action Require: b i (0) , [ i j ∗ i ( t ) S D i ( t ) m ( i, j ∗ i ( t ))] T ; 1: f or s = 1 : τ do 2: b i ( s + 1) := b j ( s ) , with j = max { argmin k ∈N i ( t ) { [ b i ( s )] 3 , [ b k ( s )] 3 } ; 3: end for 4: if [ b i ( τ )] 3 < D then 5: Update N i ( t + 1) , m i ( t + 1) and S D i ( t + 1) according to (21)–(24); 6: else if [ b i ( τ )] 3 = D then 7: No beneficial action. Algorithm has con ver ged; 8: end if In case of ties in the distances to the targets [ b j ( s )] 3 , then the node with the largest index is selected (line 2, Alg. 3). Note that line 2 of Alg. 3 is essentially a minimum consensus update on the entries [ b i ( s )] 3 and will con verge to a common outcome for all nodes in finite time τ > 0 , when they ha ve all been compared to each other . When the consensus has con ver ged, if there exists a node whose desired action decreases the distance to the target moments, i.e., if [ b i ( s )] 3 < D (line 4, Alg. 3), then Alg. 3 terminates with a greedy action and node i updates its set of neighbors N i ( t + 1) and vector of moments m i ( t + 1) (line 5, Alg. 3). If the optimal action is a link addition, i.e., if [ b i ( τ )] 2 6∈ N i ( t ) , then N i ( t + 1) := N i ( t ) ∪ { [ b i ( τ )] 2 } . (21) On the other hand, if the optimal action is a link deletion, i.e., if [ b i ( τ )] 2 ∈ N i ( t ) , then N i ( t + 1) := N i ( t ) \ { [ b i ( τ )] 2 } . (22) In all cases, the moments and error function are updated by m i ( t + 1) := [[ b i ( τ )] 4 . . . [ b i ( τ )] 4+ K ] T (23) and S D i ( t + 1) := [ b i ( τ )] 3 , (24) respecti vely . Finally , if all local desired actions increase the distance to the target moments, i.e., if [ b i ( τ )] 3 = D (line 6, Alg. 3), then no action is taken and the algorithm terminates with a October 26, 2021 DRAFT 20 network topology with almost the desired spectral properties. This is because no action exists that can further decrease the distance to the target moments. D. Synchr onization Communication time delays, packet losses, and the asymmetric network structure, may result in runs of the algorithm starting asynchronously , outdated information being used for future decisions, and consequently , nodes reaching different decisions for the same run. In the absence of a common global clock, the desired synchronization is ideally event trigger ed , where by a triggering ev ent we understand the time instant that messages are transmitted and receiv ed by the nodes. For an implementation of such a scheme see [19]. V . N U M E R I C A L S I M U L A T I O N S In the follo wing numerical examples, we illustrate the performance and limitations of our iter- ati ve graph process. The objectiv e of our simulations is to find a graph whose Laplacian spectral moments match those of a desired spectrum. In each example, we analyze the performance of our algorithm and study the spectral and structural properties of the resulting graph. Example 5.1 (Star vs. T wo-Star Networks): In our first two simulations, we try to find graphs that match the spectral moments of (i) a star graph and (ii) a two-star graph (Fig. 3). The Lapla- cian spectral moments of a star network with 10 nodes are: ( m k ) 5 k =1 = (1 . 8 , 10 . 8 , 100 . 8 , 1000 . 8 , 10000 . 8) . Starting with a random graph on 10 nodes, we run our distributed algorithm to iterati vely add and delete edges that minimize the spectral pseudodistance. W e observe, in Fig. 2, that the spectral pseudodistance ev olves to wards zero in 45 steps. W e also verify that, although we are only controlling the first fiv e spectral moments of the Laplacian matrix, the resulting network structure is e xactly the desired star topology . This indicates that a star graph is an e xtreme case in which the graph topology is uniquely defined by their first five Laplacian spectral moments. In our second simulation, we consider the two-star network with 20 nodes in Fig. 3 (a). The Laplacian spectral moments of this graph are ( m k ) 5 k =1 = (1 . 9 , 12 . 8 , 133 . 6 , 1480 , 16590) . W e observe in Fig. 2 how , after running our iterative algorithm for 94 iterations, our graph process stops in a graph topology with a spectral pseudodistance very close to zero (in particular , 5 . 2 e − 2 ). The resulting topology , represented in Fig. 3 (b), is very close to the desired two-star network. This topology is a local minima of our ev olution process because we could transform it into October 26, 2021 DRAFT 21 Fig. 2. Conv ergence of the spectral pseudodistance d k ( S t , S ∗ ) for the star graph (blue plot) and the two-stars graph (red plot), where S ∗ is the spectrum of the desired graph and S t is the spectrum of G t . our optimal two-star graph by two simple operations: (1) Adding an edge connecting nodes u and v (Fig. 3 (b)), and (2) removing edge ( u, w ) . On the other hand, one can verify that step (1) would increase the spectral pseudodistance; therefore, our greedy e v olution process does not follo w this two-steps path. Despite this limitation, our final topology is remarkably close to the two-star network and their eigen value spectra are very similar , as shown in Fig. 4. Example 5.2 (Chain vs. ring networks): In the next two simulations, we try to find graphs that match the spectral moments of (i) a ring graph and (ii) a chain graph. Starting from a random graph, we run our iterativ e algorithm to match the spectral moments of a chain graph with 20 nodes, ( m k ) 5 k =1 = (1 . 9 , 5 . 6 , 18 . 4 , 63 . 6 , 226 . 4) . In this case, the spectral pseudodistance con ver ges to zero in finite time and the final topology is exactly the desired chain graph. On the other hand, if we try to match the spectral moments of the ring graph in Fig. 5 (a), with ( m k ) 5 k =1 = (2 , 6 , 20 , 70 , 252) , an exact reconstruction is very dif ficult to achie ve. In Fig. 5 (b), we depict the graph returned by our algorithm, after 83 iterations. Note that since we are only allo wing local structural modifications in our graph process, it is hard for our algorithm to replicate long cycles in the graph. On the other hand, although the structure of the resulting October 26, 2021 DRAFT 22 Fig. 3. Structures of the two-stars network (a) and the network returned by our algorithm (b). Fig. 4. Empirical cumulati ve distribution functions for the eigen values of the two-stars graph (blue) and the graph returned by our algorithm (red). The subgraph in the lower right corner shows the CDF’ s around the origin. October 26, 2021 DRAFT 23 Fig. 5. In (a) we observe a ring graph with 20 nodes. The topology returned by our iterati ve algorithm can be observed in (b). Fig. 6. Empirical cumulativ e distribution of eigenv alues for the ring graph with 20 nodes (blue plot) and the topology returned by our algorithm in Fig. 5 (b) (red plot). network is not the desired ring graph, its eigen v alue spectrum is remarkably close to that of a ring, as we can see in Fig. 6. October 26, 2021 DRAFT 24 The above examples illustrate two limitations of our algorithm, namely , the existence of local minima in the graph e volution process and the inability of our algorithm to recov er long cycles. Despite these limitations, our algorithm is able to find graph topologies with eigen value spectra remarkably close to the desired ones by matching fi ve spectral moments only . Furthermore, the resulting topologies are structurally v ery similar to the desired ones, indicating that the spectral moments of the Laplacian matrix contains rich information about the structure of a network. In the next two examples, we show how our algorithm is also able to ef ficiently generate graphs matching the spectral properties of two popular synthetic network models: the Small-W orld [20] and the Scale-Free [21] networks. Example 5.3 (Small-W orlds): The small-world model was proposed by W atts and Strogatz [20] to generate networks with high clustering 12 coef ficients and small av erage distance. W e can generate a small-world network by following these steps: (1) take a ring graph with n nodes, (2) connect each node in the ring to all its neighborhoods within a distance k , and (3) add random edges with a probability p . In this example, we generate a small-world network with n = 40 , k = 2 , and p = 3 /n . The first three spectral moments of a random realization of this network are ( m k ) 3 k =1 = (6 . 55 , 51 . 9 , 457) . Then, we run our algorithm to generate a graph whose first three spectral moments are close to those of the small-world network. After running our algorithm for 78 iterations, we obtain a graph topology with a spectral pseudodistance very close to zero (in particular , 1 . 7 e − 3 ) and an eigen value spectrum remarkably similar to that of the small-world network, as shown in Fig. 7. Example 5.4 (P ower-Law): Another popular model in the ‘Network Science’ literature is the scale-free netw ork. This model was proposed by Barab ´ asi and Albert in [21] to e xplain the presence of hea vy-tailed degree distributions in many real-world networks. In this example, we generate a random power -law network with n = 50 nodes and m = 4 , where m is a parameter that characterizes the av erage degree of the resulting network (see [21] for more details about this model). A random realization of this network presents the following sequence of moments: ( m k ) 5 k =1 = (7 . 72 , 111 , 2 . 81 e 3 , 9 . 70 e 4 , 3 . 82 e 6) . Then, after running our algorithm for 98 iterations, we obtain a graph topology with a spectral pseudodistance very close to zero (in particular , 5 . 5 e − 2 ). The eigen value spectrum of the resulting topology is remarkably similar 12 The clustering coef ficient of a network is a measure of the number of triangles present in the network. October 26, 2021 DRAFT 25 Fig. 7. Empirical cumulativ e distribution for the eigen value spectrum of the small-world graph in Example 5.3 (blue) and the topology resulting from our algorithm (red). to that of the small-world network, as shown in Fig. 8. Furthermore, we can compare the degree sequences of the po wer-law network and the topology generated by our algorithm. W e compare these sequences, sorted in descending order , in Fig.9. W e observe how the degree sequence of the topology obtained in our algorithm is remarkably close to that of the po wer -law network. This indicates that the spectral properties of a network contains rich information about the network structure, in particular , the first fi ve spectral moments seems to highly constrain many rele v ant structural properties of the graph, such as the degree distribution. V I . C O N C L U S I O N S A N D F U T U R E R E S E A R C H In this paper , we have described a fully decentralized algorithm that iterativ ely modifies the structure of a network of agents with the objectiv e of controlling the spectral moments of the Laplacian matrix of the network. Although we assume that each agent has access to local information reg arding the graph structure, we show that the group is able to collectiv ely aggregate their local information to take a global optimal decision. This decision corresponds to the most beneficial link addition/deletion in order to minimize a distance function that in volves October 26, 2021 DRAFT 26 Fig. 8. Empirical cumulativ e distribution for the eigenv alue spectrum of the power-la w graph in Example 5.4 (blue) and the topology resulting from our algorithm (red). Fig. 9. Degree sequences (in descending order) of the power-la w graph in Example 5.4 (blue) and the topology resulting from our algorithm (red). October 26, 2021 DRAFT 27 the Laplacian spectral moments of the network. The aggregation of the local information is achie ved via gossip algorithms, which are also used to ensure network connectivity throughout the ev olution of the network. Future work in volves identifying sets of spectral moments that are reachable by our control algorithm. (W e say that a sequence of spectral moments is reachable if there exists a graph whose moments match the sequence of moments.) Furthermore, we observed that fitting a set of low- order moments does not guarantee a good fit of the complete distribution of eigen v alues. In fact, there are important spectral parameters, such as the algebraic connectivity , that are not captured by a small set of spectral moments. Nev ertheless, we observed in numerical simulations that fitting the first four moments of the eigenv alue spectrum often achie ves a good reconstruction of the complete spectrum. Hence, a natural question is to describe the set of graphs most of whose spectral information is contained in a relativ ely small set of low-order moments. A P P E N D I X Theor em A.1: Consider two undirected (possibly weighted) graphs G 1 and G 2 with (real) eigen value spectra S ( G 1 ) = { λ (1) 1 ≤ ... ≤ λ (1) n } and S 2 ( G 1 ) = { λ (2) 1 ≤ ... ≤ λ (2) n } . Then, λ (1) i = λ (2) i for all 1 ≤ i ≤ n if and only if m k ( G 1 ) = m k ( G 2 ) for 0 ≤ k ≤ n − 1 . Pr oof: The theorem states that the spectrum S ( A ) = { λ i } n i =1 of an y n × n symmetric matrix A is uniquely characterized by its first n − 1 spectral moments. First, we use Cayley- Hamilton theorem to prov e that the first n − 1 spectral moments of the spectrum S characterize the whole infinite sequence of moments ( m k ( S )) ∞ k =0 , as follo ws. Let φ ( λ ) , det ( λI n − A ) = λ n + α n − 1 λ n − 1 + ... + α 0 , be the characteristic equation of A . Then, from Cayley-Hamilton, we hav e φ ( A ) = 0 . Multiplying φ ( A ) by 1 n A t , and applying the trace operator , we hav e that, 1 n T race  A t φ ( A )  = 1 n T race  A t + n  + α n − 1 1 n T race  A t + n − 1  + ... + α 0 1 n T race  A t  = m t + n ( A ) + α n − 1 m t + n − 1 ( A ) + ... + α 0 m t ( A ) = 0 , for all t ∈ N . Therefore, giv en the sequence of moments ( m k ( A )) n − 1 k =0 , we can use the recursion m t + n ( A ) = − α n − 1 m t + n − 1 ( A ) − ... − α 1 m t +1 ( A ) − α 0 m t ( A ) , to uniquely characterize the infinite sequence of moments ( m k ( A )) ∞ k =0 . October 26, 2021 DRAFT 28 Second, we prove that the infinite sequence of moments ( m k ( A )) ∞ k =0 uniquely characterizes the eigen v alue spectrum. Let us define the spectral measur e of the matrix A with real eigenv alues λ 1 ≤ λ 2 ≤ ... ≤ λ n , as µ A ( x ) = n X i =1 δ ( x − λ i ) , where δ ( • ) is the Dirac delta function. In what follo ws, we pro ve that the spectral measure of A is uniquely characterized by its infinite sequence of spectral moments using Carleman’ s condition [22]. Since there is a tri vial bijection between the eigen value spectrum of A and its spectral measure, uniqueness of the spectral measure imply uniqueness of the eigen value spectrum. Carleman’ s condition states that a measure µ on R is uniquely characterized by its infinite sequence of moments ( M k ( µ )) ∞ k =1 if ( i ) M k ( µ ) < ∞ for all k ∈ N , and ( ii ) ∞ X s =1 ( M 2 s ( µ )) − 1 / 2 s = ∞ . In our case, the moments of the spectral measure µ A are M k ( µ A ) = Z + ∞ −∞ x k dµ A ( x ) = n X i =1 λ k i = n m k ( A ) . These moments satisfy: ( i ) M k ( µ A ) ≤ nλ k n < ∞ , for any finite matrix A , and ( ii ) ∞ X s =1 ( M 2 s ( µ A )) − 1 / 2 s = ∞ X s =1 n X i =1 λ 2 s i ! − 1 / 2 s ≥ ∞ X s =1  λ 2 s n  − 1 / 2 s = ∞ X s =1 λ − 1 n = ∞ , for any A 6 = 0 . As a consequence, the spectral measure of any finite matrix A 6 = 0 with real eigen values is uniquely characterized by ( M k ( µ A )) ∞ k =0 . Since, M k ( µ A ) = n m k ( A ) , we ha ve that the sequence of moments ( m k ( A )) n − 1 k =0 uniquely characterizes ( M k ( µ A )) ∞ k =0 . Therefore, the sequence of moments ( m k ( A )) n − 1 k =0 uniquely characterize the spectral measure µ A and the real eigen value spectrum S = { λ i } n i =1 . October 26, 2021 DRAFT 29 R E F E R E N C E S [1] N. Wiener , The Mathematics of Self-Or ganising Systems. Recent Developments in Information and Decision Pr ocesses , Macmillan, 1962. [2] H. Haken, Synergetics: An Intr oduction , 3 rd Edition, Springer-V erlag, 1983. [3] M.O. Jackson, Social and Economic Networks , Princeton Univ ersity Press, 2008. [4] V .M. Preciado, Spectral Analysis for Stochastic Models of Lar ge-Scale Complex Dynamical Networks , Ph.D. dissertation, Dept. Elect. Eng. Comput. Sci., MIT , Cambridge, MA, 2008. [5] L.M. Pecora and T .L. Carroll, “Master Stability Functions for Synchronized Coupled Systems, ” Physics Review Letters , vol. 80, pp. 2109-2112, 1998. [6] V .M. Preciado and G.C. V erghese, “Synchronization in Generalized Erd ¨ os-R ´ enyi Networks of Nonlinear Oscillators, ” Pr oc. of the 44th IEEE Conference on Decision and Contr ol , pp. 4628-4633, 2005. [7] N.A. L ynch, Distributed Algorithms , Morgan Kaufmann Publishers, 1997. [8] A. Fax and R. M. Murray , “Information Flow and Cooperati ve Control of V ehicle Formations, ” IEEE T ransactions on Automatic Contr ol , vol. 49, pp. 1465-1476, 2004. [9] R. Olfati-Saber and R. M. Murray , “Consensus Problems in Networks of Agents with Switching T opology and T ime- Delays, ” IEEE T ransactions on Automatic Contr ol , vol. 49, pp. 1520-1533, 2004. [10] R. Grone, R. Merris, and V .S. Sunder , “The Laplacian Spectrum of a Graph, ” SIAM Journal Matrix Analysis and Applications , vol. 11, pp. 218-238, 1990. [11] A. Ghosh and S. Boyd, “Growing W ell-Connected Graphs, ” Pr oc. of the 45th IEEE Conference on Decision and Contr ol , pp. 6605-6611, 2006. [12] Y . Kim and M. Mesbahi, “On Maximizing the Second-Smallest Eigen value of a State Dependent Graph Laplacian, ” IEEE T ransactions on Automatic Control , v ol. 51, pp. 116-120, 2006. [13] M.C. DeGennaro and A. Jadbabaie, “Decentralized Control of Connectivity for Multi-Agent Systems, ” Pr oc. of the 45th IEEE Conference on Decision and Contr ol , San Diego, CA, Dec. 2006, pp. 3628-3633. [14] N. Biggs, Algebraic Graph Theory , Cambridge University Press, 2 nd Edition, 1993. [15] V .M. Preciado and G.C. V erghese, “Lo w-Order Spectral Analysis of the Kirchhoff Matrices for a Probabilistic Graph with Prescribed Expected Degree Sequence, ” IEEE T ransactions on Cir cuits and Systems I , vol. 56, pp, 1231-1240, 2009. [16] D.M. Cvetkovi ´ c, M. Doob, and H. Sachs, Spectra of Graphs , 3 rd Edition, Wile y-VCH, 1998. [17] J. H. W ilkinson, The Algebraic Eigen value Pr oblem, Oxford University Press, 1965. [18] J. Cortes, “Distributed Algorithms for Reaching Consensus on General Functions, ” Automatica , vol. 44, pp. 726-737, 2008. [19] M. M. Zavlanos and G. J. Pappas, “Distributed Connectivity Control of Mobile Networks, ” IEEE T ransactions on Robotics , vol. 24, pp. 1416-1428, 2008. [20] D.J. W atts and S. Strogatz, “Collective Dynamics of Small W orld Networks, ” Natur e , vol. 393, pp. 440-42, 1998. [21] A. L. Barab ´ asi and R. Albert, “Emergence of Scaling in Random Networks, ” Science , vol. 285, pp. 509-512, 1999. [22] N.I. Akhiezer, The Classical Moment Pr oblem and Some Related Questions in Analysis , Oliver & Boyd, 1965. October 26, 2021 DRAFT