Sparsity Invariance for Convex Design of Distributed Controllers

Sparsit y In v ariance for Con v ex Design of Distribu ted Con trollers Luca F urieri, Y a ng Zheng, An tonis P apac hr isto doulou, and Maryam Kamgarp our ∗ July 14, 2020 Abstract W e address the problem of designing optimal linear t ime-in v ariant (L TI) sparse controlle rs for L TI systems, whic h corresponds to minimizing a norm of the closed-lo op system sub ject to sparsity constraints on the con troller structure. This problem is NP-hard in general and motiv ates the developmen t of tractable approximatio ns. W e chara cterize a class of conv ex restrictions based on a new notion of Sparsity Inv ariance (SI). The u nderlying idea of SI is to design sparsity patterns for transfer matrices Y ( s ) and X ( s ) suc h that any correspond ing control ler K ( s ) = Y ( s ) X ( s ) − 1 exhibits the desired sparsit y pattern. F or sparsity constraints, th e approach of S I goes beyond the notion of Qu adratic Inv ariance (QI) : 1) the S I approach alwa ys yields a conv ex restriction; 2) the solution via the S I app roac h is guaranteed to b e globally optimal when QI h olds and p erforms at least as well as considering a nearest QI subset. Moreo ver, the notion of SI naturally app lies to d esigning structu red static controllers, while QI is not utilizable. N umerical ex amples show that even for non- QI cases, SI can reco ver solutions that are 1) globally optimal and 2) strictly more performing than previous methods. 1 In tro duction The safe and efficien t op eration of several large- scale s ystems, such as the s mart grid [1], biolo gical net- works [2], and automa ted highw ays [3], relies on the decision ma k ing o f multiple interacting agents. Co or di- nating the decisions of these age nts is challenged by a lack of complete informa tion of the systems’ internal v ariables. Such limited informa tion arises due to priv acy co ncerns, geogra phic distance or the challenges of implemen ting a r eliable co mm unication netw ork. The celebra ted work [4] highlighted that lacking full infor mation can enormously co mplica te the design of optimal con tr ol inputs. Indeed, the optimal feedback control policies may not even b e linear for the Linear Quadratic Ga ussian (LQG) control problem without full output info r mation. The intractability inherent to lack of full information was inv estigated in the works [5, 6]. The co re challenges discussed there in motiv ated ident ifying sp ecial ca s es of optimal control pro blems with partial information for which efficient algorithms can b e used. ∗ This research wa s gratefully funded b y the Europ ean Uni on ERC Starting Grant CONENE. Anto nis Pap ac hristo doulou wa s supported in part b y the EPSR C pro j ect EP/M002454 /1. Luca F urieri and Marya m Kamgarpour are with the Aut omatic Con trol Lab oratory , Departmen t of Inform ation T ec hnology and Electrical Engineering, E T H Z ¨ urich, Switzerland. E-mails: { furieri l, mkamgar } @contro l.ee.ethz.ch . Y ang Zheng is with the School Of Engineering And Applied Sciences, Har v ard Cen ter for Green Buildings and Cities, Harv ard Universit y . E-mail: zhengy@g.harva rd.edu . A n tonis Papac hris todoulou is with the Departmen t of Engineering Science, Universit y of Oxford, United Kingdom. Email: antonis@eng.o x.ac.uk . 1 Optimally controlling a linear time-inv ariant system ( L TI) with distributed sensor measurements amounts to computing a linear controller that has a desired spa rsity pattern and minimizes a nor m of the closed- lo op system. F or this g enerally in tractable pro blem, the notion of Quadr a tic Inv a riance (QI) was shown to be sufficient [7] a nd necessary [8] for an exa c t conv ex reformulation. A rela ted pro blem of s ensor-a c tuator architecture co -design was addre s sed in [9, 1 0] by exploiting QI and using spa rsity-inducing no rm p enalties. 1.1 Previous work on non-QI cases Given the importance and in tricacy of computing optimal distributed controllers, a v ariety of approximation metho ds hav e been prop os ed for general sys tems a nd information structures that ar e not QI. F or example, the author s in [11] develope d semidefinite prog rams that are r elaxations of this generally NP-har d problem. How ever, these r elaxations might fail to r e cov er a spars e controller that is stabilizing, as confirmed exp eri- men tally in [12]. T o a ddr ess this issue, po lynomial optimization has b een used in [12] to obta in a sequence of co n vex relaxa tions which co n verges to a stabilizing distributed controller. Nevertheless, p erformance of the re c ov er e d solution is not dir e ctly address ed in [1 2]. F or the finite-hor izon control problem, the a uthors in [13] der ived conv ex upp er b ounds to the non-co nv ex cost function to o btain conser v ative feasible solu- tions. Ho wev er , the theoretical sub-optimality b ounds w ere shown to be loo se. A lternatively , the system level a pproach [1 4] pro po sed an implemen tation where controllers ar e req uired to share lo c ally estima ted disturbances in the sta te-feedback case a nd internal controller states in the output-feedback ca se. W e note that the class ical dis tr ibuted control only requires to shar e output measurements, but no intermediate com- putations, a mong subs y stems. The need to share this additiona l informatio n in [14] might r aise concer ns of system se c urity a nd vulner ability in safety critical applications [15], where ea ch subsys tem can only r ely on its own sens or measur emen ts. A differen t approach to sparse output-feedback co ntroller synthesis is to develop a c onvex r estriction : the unstructured problem is reformulated as an equiv a len t conv e x pro gram a nd conv ex constraints ar e added to guarantee the desired spa rsity pa ttern o f the rec overed co n trollers . Convex r estrictions ex hibit sp ecific a d- v antages: 1) their optimal s olutions can b e r eadily computed with standard conv ex optimiza tion techniques, and 2 ) all their feasible solutions a re structured and stabilizing b y design. A disadv antage is that a restriction may b e infeasible even when the o riginal problem is feasible. This motiv ates developing convex r estrictions that are as tig ht as p ossible for improv ed feasibility and pe r formance. In the literature, convex r estrictions hav e mostly b een develop ed for the sp ecial case o f computing static controllers [16–18]. Wit hin this setting, the problem of optimal sensor and actuator selection was addr essed in [19, 20] with an ADMM approa ch. F or the gener al case of dyna mic controllers given non-Q I information structures, the w o rk [21] suggested restricting the des ir ed sparsity pattern to a subse t tha t is QI to obtain upp er b ounds o n the minimum co st. How ever, to the b est of the authors’ k nowledge, a metho d for conv ex restrictions that can outpe rform [21] and go e s b eyond the notion of Q I for sparsity constra int s is not k nown. 1.2 Con t ributions This paper propo ses a generalized framework for t he conv ex design of optimal a nd near-optimal L TI dynamic output-feedback controllers with a pre-determined s parsity pattern. Our underlying idea is to identif y appropria te sparsit y patterns for tw o tra nsfer matrices Y ( s ) and X ( s ) such that any corresp onding feedbac k controller in the form K ( s ) = Y ( s ) X ( s ) − 1 exhibits the desired structure. This fundament al prop erty is denoted as Spa rsity Inv a riance (SI). 2 Our first contribution is to develop alg ebraic conditions on the binary matr ic es asso ciated with the sparsities of Y ( s ) and X ( s ) that are necessa ry and sufficient for SI. Among all such sparsities, w e sugge st a p olyno mial-time algor ithm to design s pa rsities that lead to b etter p erfor mance for the distributed control problem at hand. Second, we show that the SI no tion steps b eyond tha t of QI in several ways. Indeed, SI can b e applied to general systems sub ject to a r bitrary sparsity constr aints, regar dless of whether Q I holds. F urthermor e, SI recov ers a controller that is prov a bly g lobally optimal when QI holds a nd per forms at lea st as well as that obtained by cons idering a near est QI spar sity s ubs et [21] when QI do es not hold. Third, we provide examples to s how that, even if Q I do es not hold, co n trollers obtained throug h the SI appro a ch can be 1) globally optimal and 2 ) in gener al str ictly more p erforming than those obtained using the nearest QI subset appr oach of [21]. Finally , we remark tha t the SI c o ncept is applicable to distributed static controller design, as s tudied in our prelimina r y work [18], wherea s the Y oula parametriza tion and thus the QI notion is not utilizable. F or brevity , o ur theor etical discussion fo cuses on contin uo us -time systems, but o ur results also natur ally hold for discr ete-time systems with spa rsity constra ints, as we will discuss in the num erical results. The r est o f this pap er is structur e d as follows. Sec tio n 2 states necessar y background and presents the problem for m ulation. Section 3 in tro duces the class of conv ex restrictions under in vestigation and fully characterizes our notion of Spars ity In v ariance (SI). W e describ e how SI can b e utilized in an optimized way . In Section 4, w e show that 1) SI encompasses the previous approa ch es based on the QI notion, and 2) that strictly b e tter p erfor ming spar se controllers can be computed efficie n tly with the SI a pproach. W e present nu merical results in Sectio n 5 and conclude the pap er in Section 6. 2 Bac kground and Problem Statemen t Here, we fir st in tr o duce some notation o n spars it y structures and trans fer functions. Then, we state the problem of distributed optimal control, a nd introduce the neces sary background on the Y oula par ametrization of in ternally stabiliz ing controllers. 2.1 Notation and sparsit y structures W e use R , C and N to denote real num b ers , complex num b ers and p ositive integers, resp ectively . The ( i, j )- th element in a matrix Y ∈ R m × n is r eferred to as Y ij . W e use I n to denote the identit y matrix of size n × n , 0 m × n to denote the zero matrix o f s iz e m × n and 1 m × n to denote the matrix of size m × n with all entries set to 1. T r ansfer functions: W e denote the imaginar y a xis as j R := { z ∈ C | ℜ ( z ) = 0 } and consider contin uous- time transfer functions f : j R → C . A m × n t r ansfer matrix is the set of m × n matr ic e s whos e ent ries are tr a nsfer functions. W e deno te the se t of m × n caus a l transfer matr ic es a s R m × n c . A tr ansfer function is called pr op er (resp. strictly-pr op er ) if it is rational and the degree of the numerator p o lynomial do es not exceed (r esp. is strictly lower than) the deg ree of the deno minator p olynomia l. Similar to [7], w e denote by R m × n sp the s et of m × n strictly prop er transfer matrices. Finally , we let RH m × n ∞ be the se t of m × n causa l and stable tra nsfer matrices. Sparsity s tr uctures of transfer matrices can b e conv eniently r epresented by binary matrices . A binary matrix is a matrix with entries from the set { 0 , 1 } , and we us e { 0 , 1 } m × n to denote the set of m × n binary matrices. Given a binary matrix X ∈ { 0 , 1 } m × n , we de fine the asso cia ted sp arsity subsp ac e o f causal transfer 3 matrices as Sparse( X ) := { Y ∈ R m × n c | Y ij ( j ω ) = 0 f or all i, j such that X ij = 0 for almost all ω ∈ R } . Similarly , given a tr ansfer function Y ∈ R m × n c , w e define X = Struct( Y ) a s the binary matrix given by X ij :=    0 if Y ij ( j ω ) = 0 for almost all ω ∈ R , 1 otherwise . W e say that the transfer matrix X ∈ R n × n c is inv ertible if X ( j ω ) ∈ C n × n is inv ertible for almos t all ω ∈ R . Let X , ˆ X ∈ { 0 , 1 } m × n and Z ∈ { 0 , 1 } n × p be binary matrices. Throughout the paper , we adopt the following conv entions: X + ˆ X := Struct( X + ˆ X ), and X Z := Struct( X Z ). W e s ay X ≤ ˆ X if and only if X ij ≤ ˆ X ij ∀ i, j , and X < ˆ X if and only if X ≤ ˆ X and there exist indices i , j such that X ij < ˆ X ij . Also, we denote X  ˆ X if and only if there exist indices i, j s uc h that X ij > ˆ X ij . Given a binary matrix X ∈ { 0 , 1 } m × n we denote its ca rdinality , i.e. , the to tal num b er of nonzero ent ries, as k X k 0 := m X i =1 n X j =1 X ij . Considering the following binar y matr ices X 1 = " 0 1 0 1 1 1 # , X 2 = " 0 1 0 1 0 1 # , X 3 = " 1 1 0 1 0 1 # , we hav e X 2 < X 1 , X 3  X 1 and X 2 + X 1 = X 1 . Their cardinalities are k X 1 k 0 = 4 , k X 2 k 0 = 3 and k X 3 k 0 = 4, resp ectively . F or the following transfer matrix , Y = " 0 1 s +1 0 1 s +1 1 s +1 1 s +1 # ∈ RH 2 × 3 ∞ , if we consider the binary matrix X 1 in the exa mple ab ov e, we hav e Y ∈ Sparse( X 1 ) a nd X 1 = Struct( Y ). 2.2 Problem st atemen t W e consider L TI systems in contin uous-time ˙ x ( t ) = Ax ( t ) + B u ( t ) + H x w ( t ) , (1) y ( t ) = C y x ( t ) + H y w ( t ) , z ( t ) = C z x ( t ) + D z u ( t ) + H z w ( t ) , where x ( t ) ∈ R n , u ( t ) ∈ R m , y ( t ) ∈ R p , z ( t ) ∈ R q , a nd w ( t ) ∈ R r are the s tate, con trol input, observed output, a perfor mance s ignal defined bas ed on our control ob jectives, and additiv e disturbance at time t ∈ R , 4 ✲ ❄ ✲ v 1 ❡ ❡ ✛ ✲ ✛ v 2 ✛ ✛ y u w z K P 11 P 12 P 21 G Figure 1: In terconnection of P and K . resp ectively . The input-o utput tra nsfer function repr esentation for (1) can b e written as " z y # = P " w u # = " P 11 P 12 P 21 G # " w u # , with P 11 := C z ( sI n − A ) − 1 H x + H z , P 12 := C z ( sI n − A ) − 1 B + D z , P 21 := C y ( sI n − A ) − 1 H x + H y , G := C y ( sI n − A ) − 1 B , where s b elong s to j R . Notice that P 11 , P 12 , P 21 are pro per trans fer functions and G is strictly pro p er . Consider the in ter connection of Figure 1. A dynamic output-feedback c o nt roller u = K y with K ∈ R m × p c is said to b e int ernal ly stabilizing if and o nly if the nine tra nsfer matrices from w , ν 1 , ν 2 to z , y , u a re stable. W e deno te the set of all causal L TI internally stabilizing output-feedback controllers as C stab . W e say that P is stabiliza ble if only and if C stab 6 = ∅ a nd a ny K ∈ C stab stabilizes P . F urthermo r e, we say that a controller K stabilizes G if and only if the four transfer matrices from ν 1 , ν 2 to y , u ar e a ll sta ble. F or the r est of the pap er we make the following assumption. Assumption 1: The system P is stabilizable. A test for sta biliz a bilit y of P is offered in [22, Chapter 4 ]. I t is well-known [22, Chapter 4], [7] that under Assumption 1 a co n troller K stabilizes P if and only if it sta bilizes G . The co n trol pro blem is to compute a dynamic o utput-feedback controller K ∈ C stab which minimizes a g iven nor m k · k of f ( K ) = P 11 + P 12 K ( I p − GK ) − 1 P 21 , (2) which is the close d- lo op transfer function from w to z . In distr ibuted co n trol, it is common to a dd the req uir ement that K only uses partial output mea surements. This req uir ement can b e c aptured by adding the co ns traint K ∈ Sparse( S ) for a given binary matrix S ∈ { 0 , 1 } m × p , where S ij = 0 enco des the fact that the i -th sca lar control input cannot measure the j -th measurement output. W e for m ulate this distributed, sparsity-constrained c o nt rol problem as f ollows [7]: Problem P K minimize K ∈C stab k f ( K ) k sub ject to K ∈ Sparse( S ) , where k · k is a n y nor m of int erest. It w as shown that a nece s sary and sufficient condition for a feasible 5 solution to P K to exis t is that a ll the distributed fixed mo des asso ciated with S lie in the left ha lf o f the complex plane [23 ]. E ven if P K is feasible, directly co mputing its optimal so lution is intractable b eca use the set C stab is non-conv ex in g eneral. This can b e easily verified by chec king tha t whe n K 1 , K 2 ∈ C stab , the controller K = 1 2 ( K 1 + K 2 ) do es not lie in C stab in general. F urther more, the co st function k f ( K ) k is non-conv ex in K . 2.3 The Y oula parametrization of stabilizing con troller The first step to co n vexify pr oblem P K is to derive a conv ex formulation o f the set C stab and the function f ( K ). This is achiev e d b y using a doubly c oprime factorization of G . Lemma 1 (Chapter 4 of [ 22]) F or any G ∈ R p × m sp , ther e exist eight pr op er and stable tr ansfer m atric es defining a doubly c oprime factorization of G , that is, they satisfy G = N r M − 1 r = M − 1 l N l , " U l − V l − N l M l # " M r V r N r U r # = I m + p . (3) Then, the Y oula parametrization of all in ter nally stabilizing co n trollers [2 4] establishes the following equiv alence [22, C ha pter 4]: C stab = { ( V r − M r Q )( U r − N r Q ) − 1 | Q ∈ RH m × p ∞ } 1 . (4) F urthermor e, it was pr ov ed in [2 2, Chapter 4 ] that the set of a ll closed-lo op trans fer functions fro m w to z achiev a ble by K ∈ C stab is f ( C stab ) = { T 1 − T 2 QT 3 | Q ∈ RH m × p ∞ } , where f ( · ) is defined in (2) and T 1 = P 11 + P 12 V r M l P 21 , T 2 = P 12 M r and T 3 = M l P 21 . T o facilitate our problem fo r mulation, we define Y Q = ( V r − M r Q ) M l , (5) X Q = ( U r − N r Q ) M l . (6) It dir ectly follows fro m (4) that C stab = { Y Q X − 1 Q | (5) , (6) , Q ∈ RH m × p ∞ } . (7) W e notice that (3) implies U r = M − 1 l + GV r and (5) implies V r M l = Y Q + M r QM l . Hence, we hav e X Q = ( M − 1 l + GV r − N r Q ) M l = I p + G ( Y Q + M r QM l ) − N r QM l = I p + GY Q . (8) 1 Equiv alently , C stab = { ( U l − QN l ) − 1 ( V l − QM l ) | Q ∈ RH m × p ∞ } . 6 Now we can equiv alently reformulate P K int o the following o ptimiza tion proble m. Problem P Q minimize Q ∈RH m × p ∞ k T 1 − T 2 QT 3 k sub ject to (5) , (6) , Y Q X − 1 Q ∈ Sparse( S ) . Without t he sparsity constraint Spars e( S ), problem P Q would b e con vex, as (5) , (6) and the cost function are affine in Q . The primar y source of non-conv exity is the require men t that Y Q X − 1 Q ∈ Sparse( S ). W e conclude that the co mplexit y of distributed c ontrol is ultimately linked to the non-co nv ex sparsity requirement o n the Y oula parameter. 3 Sparsit y In v ariance One a pproach to remov e the non-conv ex spa rsity req uirement on the Y oula parameter is as follows: replace Y Q X − 1 Q ∈ Sparse( S ) with th e c onv ex constrain t that Y Q and X Q comply with appropriate s parsity patterns, in a wa y such that Y Q X − 1 Q is guar a nt eed to lie in Spar se( S ). In other words, we res trict our attention to distributed spa rse controllers K ∈ Spars e( S ) defined as the pro duct of t wo structur ed matrix facto rs. W e note that related idea s app ear ed for the sp ecific cas e of row-column spar s ities (e.g. [10, 2 0]), but the case of arbitrar y sparsities w as not addres sed. F ollowing the general idea ab ove, in this pap er w e in vestigate a no tion of Spar s it y Inv ar iance (SI) for conv ex design o f spa rse controllers. As will be thoroug hly discussed in Se c tion 4, SI leads to the larges t known class o f con vex restr ictions of P K for general systems sub ject to sparsity co ns traints on the controller. Definition 1 (Sparsit y Inv ariance (SI)) Given a binary m atrix S , the p air of binary m atr ic es T , R sat- isfies a pr op erty of sp arsity invarianc e (SI) with r esp e ct to S if Y ∈ Sp arse ( T ) and X ∈ Sp arse ( R ) ⇓ (9) YX − 1 ∈ Sp arse ( S ) . Motiv ated by the SI prop erty , conside r the following conv ex problem: Problem P T ,R minimize Q ∈RH m × p ∞ k T 1 − T 2 QT 3 k sub ject to (5) , (6) , Y Q Γ ∈ Sparse( T ) , X Q Γ ∈ Sparse( R ) , where T ∈ { 0 , 1 } m × p , R ∈ { 0 , 1 } p × p and Γ ∈ R p × p c , with Γ inv ertible, ar e parameters to b e designed b efore per forming the optimization. F or simplicity , o ne could select Γ = I p , but w e illustra te in E xample 1 of Section 4 that there are cases where a differen t c hoice of Γ migh t lead to impro ved and even globally-optimal per formance for non-QI problems. F or any choice o f T , R and Γ , the ab ov e progra m is conv ex. One fundamen tal question is when its feasible solutions lead to stabilizing c ontrollers K = ( Y Q Γ )( X Q Γ ) − 1 = 7 Y Q X − 1 Q lying in the desired sparsity subspa ce Spars e( S ). The notion of SI (9) defined ab ov e is a mathematical expression of this re quirement. In the next subsection we es ta blish necessar y and s ufficient co nditions on the binary ma trices T and R to sa tisfy the SI prop erty (9). Remark 1 Note that the notion of SI is an algebraic requirement for binar y matrice s R and T , giv en a binary matrix S . This is indep endent of the par a meterization o f internally sta bilizing con trollers. In addition to the Y oula parameter ization, w e recently obser ved that the SI idea (9 ) is equiv alently applicable within the system-level [1 4] (SLP) a nd input-output [25] (IOP ) para meterizations, in both contin uous - and discrete- time. W e r e fer to [26, Remar k 4] for details. F or br evity , in this pap er we will develop our theoretical re sults within the Y oula parameteriza tion, and note that they can be straig h tforwardly applied to the SLP and the IOP . Remark 2 W e assume that R ≥ I p . Since X Q = I p + GY Q ∈ Sparse( R ) and G is strictly prop er, the assumption is without lo s s of genera lity for Γ = I p . F o r convenience, in the definition of problem P T ,R we do not indica te Γ ex plicitly as a parameter. This is b e c ause the SI pro per t y (9 ) only dep ends on the binary matrices T and R . 3.1 Characterization of SI One immediate idea in des igning the binary matrices T and R to g uarantee K = ( Y Q Γ )( X Q Γ ) − 1 = Y Q X − 1 Q ∈ Spar se( S ) is to s imply select T = S and R = I p similar to [16, 17, 27]. How ever, man y other choices are av ailable that lead to improv e d conv ex restrictions. The next Theorem provides a full characterizatio n of the SI prop erty (9) in terms o f the bina ry ma trices T and R . Theorem 1 L et T ∈ { 0 , 1 } m × p and R ∈ { 0 , 1 } p × p b e su ch that R ≥ I p . The fol lowing two statemen t s ar e e quivalent: 1. T ≤ S and T R p − 1 ≤ S . 2. SI as p er (9) holds. The pro of o f Theorem 1 is r epo rted in App endix A.1. The r elev ance of The o rem 1 to ch aracter izing a class of conv ex r estrictions o f P K is stated in the following Co rollar y . Corollary 1 L et T ∈ { 0 , 1 } m × p and R ∈ { 0 , 1 } p × p b e su ch that R ≥ I p , T ≤ S and T R p − 1 ≤ S . Then, pr oblem P T ,R p − 1 is a c onvex r estriction of P K for any invertible tr ansfer mat rix Γ ∈ R p × p c . Pro of Pr oblem P T ,R p − 1 is o b viously c o nv ex. W e only need to s how that an y solution to P T ,R p − 1 corres p onds to a feas ible solution of P Q . First, g iven any inv ertible Γ ∈ R p × p c we have ( Y Q Γ )( X Q Γ ) − 1 = Y Q X − 1 Q . Let Y = Y Q Γ and X = X Q Γ in (9). Since (9) holds b y Theorem 1, b y definition YX − 1 = Y Q X − 1 Q ∈ Sparse( S ) and thus every solution of P T ,R is a so lution of P Q . 8 Second, since P Q is eq uiv alent to P K , we conclude that P T ,R is a restr ic tio n of P K for every inv ertible Γ ∈ R p × p c . Finally , since T R p − 1 ≤ S and R ≥ I p we hav e that T ( R p − 1 ) p − 1 ≤ S by transitive closure of the graph having R as its adjacency matrix. Hence, P T ,R p − 1 is a conv ex restr ic tion of P K for every inv ertible Γ ∈ R p × p c . In summary , the algebr aic conditions T ≤ S and T R p − 1 ≤ S , (10) are equiv alent to SI and yield a class o f conv e x r estrictions of P K . Cle a rly , our co ndition (10) includes the choice T = S and R is (blo ck)-diagonal as p er [16, 17, 27]. W e will further show in Section 4 that the conv ex restrictions develop e d in [21] are a particular case of (10). Therefore, our no tio n of SI natura lly encompasse s and extends pre v ious conv ex r e strictions of P K . Remark 3 F or ea ch T and R as p er (1 0), it is alwa ys prefera ble to solve the co n vex re s triction P T ,R p − 1 instead of P T ,R . Indeed, notice tha t s inc e T R p − 1 ≤ S and R ≥ I p , then T ( R p − 1 ) p − 1 ≤ S . Equiv ale n tly , when T and R sa tisfy spa r sity inv ariance (10), so do T a nd R p − 1 , and b oth P T ,R and P T ,R p − 1 are conv ex restrictions of P K . Since requir ing X Q ∈ Spar se( R ′ ) for some R ′ < R p − 1 may b e conse rv ative in the cas e Sparse( R ′ ) ⊂ Sparse( R p − 1 ), we will fo cus on the co n vex restriction P T ,R p − 1 to av oid this p oss ibilit y . After determining a ll the matr ices T and R for sparsity inv ar iance, a na tural follow-up q uestion aris es: how ca n we choose T and R as p er Theorem 1 to obtain a co nv ex restrictio n o f P K that is as tight as po ssible? 3.2 Optimized design of SI Here, we s tudy how to choose the binar y matric es T and R o ptimally for a fixed in vertible Γ ∈ R p × p c . In or de r to deter mine the be s t p erfo rming choice fo r T and R satisfying (10), one would need in genera l to solve P T ,R p − 1 with the chosen Γ for ea ch T and R such that (1 0) holds, and then selec t the problem minimizing the ob jective k T 1 − T 2 QT 3 k . Clear ly , this approach is not tractable in general, as o ne needs to solve a lar ge num b e r of co n vex progr ams that is exponential in m and p , that is, one conv ex pro gram for each bina r y matrices T a nd R s uc h that T R p − 1 ≤ S . Even if we simplify the search ab ov e b y fixing any T ≤ S and lo oking fo r the b e st p erforming choice o f R , we w ould still nee d to solve a large num b er of conv ex progr ams that is ex p o nential in p , that is, one conv ex pro g ram for e a ch binar y matr ix R such that T R p − 1 ≤ S . T o deal with the ab ove c hallenges, here we suggest a sub optimal, but computationally efficient algorithm that gener ates a lo cally optimized binary ma trix R tailored to any chosen T ≤ S . Spec ific a lly , our prop osed a pproach is to 1) select T ≤ S a nd then 2) co mpute that binary matrix R ⋆ T which is the least sparse among those sa tis fying T R p − 1 ≤ T . (11) Clearly , b oth 1) a nd 2) above a re simplificatio ns of the gene r al pro ble m of finding the g lobally tightest con vex restriction P T ,R of P K for a fixed inv ertible Γ ∈ R p × p c ; indeed, we do not optimize ov er T and we imp ose (11), a condition str onger than the SI r e quirement (1 0) . The gain is that R ⋆ T is unique and can b e computed efficiently as per Algor ithm 1, whic h has a p olynomial complexity of O ( mp 2 ). 9 Algorithm 1 Generation of R ⋆ T 1: Initialize R ⋆ T = 1 p × p 2: for each i = 1 , . . . , m , k = 1 , . . . , p do 3: if T ik == 0 then 4: for each j = 1 , . . . , p do 5: if T ij == 1 the n 6: ( R ⋆ T ) j k ← 0 7: end if 8: end for 9: end if 10: en d for The idea b ehind Alg orithm 1 is to only set an entry of R ⋆ T to 0 if the condition T R ⋆ T ≤ T w ould be violated. W e now formalize the main r e s ult a bo ut R ⋆ T . Theorem 2 Consider a binary matrix T ∈ { 0 , 1 } m × p , and define R T := { R ∈ { 0 , 1 } p × p | R ≥ I p , (11) holds } . Then, 1. Ther e exists a unique R ⋆ T ∈ R T such that R ⋆ T ≥ R p − 1 , ∀ R ∈ R T . 2. Such R ⋆ T c an b e c ompute d via Algorithm 1. Pro of Let R ⋆ T be the unique bina ry matrix gener ated by Alg o rithm 1. It is ea sy to chec k that T R ⋆ T ≤ T by co nstruction. Since R ⋆ T ≥ I p , it follows ( T R ⋆ T ) R ⋆ T ≤ T R ⋆ T ≤ T and T ( R ⋆ T ) p − 1 ≤ · · · ≤ T R ⋆ T ≤ T . W e conclude R ⋆ T ∈ R T . Next, consider any binary matrix R ∈ R T . By definition, we have that T R p − 1 ≤ T and so ( R p − 1 ) j k = 0 whenever T ij = 1 and T ik = 0 . Then, R p − 1 ≤ R ⋆ T since ( R ⋆ T ) j k is set to 0 by Algor ithm 1 if and only if T ik = 0 and T ij = 1. Therefore, we hav e R p − 1 ≤ R ⋆ T , ∀ R ∈ R T . The next cor o llary connects our result to characterizing tight convex restr ictions of P K . Corollary 2 Given a binary matrix T ≤ S , c ompute R ⋆ T as p er Algori thm 1. Then, for every fixe d invertible Γ ∈ R p × p c , P T ,R ⋆ T is the tightest c onvex r estriction of P K among those in the form P T ,R p − 1 with R ∈ R T . Pro of Fix a n inv ertible Γ ∈ R p × p c and co nsider the problems P T ,R p − 1 and P T ,R ⋆ T , where R ∈ R T and R ⋆ T is generated by Algorithm 1. By Theo rem 2, we ha ve R p − 1 ≤ R ⋆ T , meaning that Sparse( R p − 1 ) ⊂ Sparse( R ⋆ T ). The only difference b etw e en pr oblem P T ,R p − 1 and problem P T ,R ⋆ T is: P T ,R p − 1 requires X Q Γ ∈ Sparse( R p − 1 ) while P T ,R ⋆ T requires X Q Γ ∈ Sparse( R ⋆ T ). Therefor e , we conclude that P T ,R ⋆ T admits the largest feasible region a mong a ll P T ,R p − 1 with R ∈ R T . This completes our pro of. Our suggested pro cedure can find a tight con vex res triction for P K by using the co mputationally efficient Algorithm 1, which ma kes the approach practical for pr actitioners. How ever, optimally cho osing Γ and T is also a non- trivial task which w e leav e for future work. W e remark that in the lack of any further insight, one can alwa y s choose T = S and Γ = I p and still obtain spar se controllers w ith tight s ub-optimality gaps, a s will b e shown exp erimentally in Section 5. F urther mo re, a s shown in Section 4, the trivial choice T = S and Γ = I p combined with Algor ithm 1 for cho osing R is sufficient to rec over a nd e x tend the optimality re s ults of [7], [21] which ar e base d on the Quadr atic Inv ar iance (QI) notion. W e conclude this sec tion by providing an exa mple to illustrate the SI approach. 10 ❧ y 5 ❧ y 4 ❧ y 3 ❧ y 2 ❧ y 1 K 5 , : K 4 , : K 3 , : K 2 , : K 1 , : ✲ ❅ ❅ ❅ ❘ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ❇ ❇ ❇ ❇ ❇ ❇ ❇ ◆ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ✲ ❅ ❅ ❅ ❘ ✲ ✲ ✲ ✲ ✲ ✲ ❧ u 5 ❧ u 4 ❧ u 3 ❧ u 2 ❧ u 1 ❧ y 5 ❧ y 4 ❧ y 3 ❧ y 2 ❧ y 1 X − 1 5 , : X − 1 4 , : X − 1 3 , : X − 1 2 , : X − 1 1 , : ✲ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ❇ ❇ ❇ ❇ ❇ ❇ ❇ ◆ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ✲ ❅ ❅ ❅ ❘ ✲ ✲ ❅ ❅ ❅ ❘ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ❇ ❇ ❇ ❇ ❇ ❇ ❇ ◆ ✲ ❅ ❅ ❅ ❘ ❆ ❆ ❆ ❆ ❆ ❯ ✲ ❅ ❅ ❅ ❘ ✲ Y 5 , : Y 4 , : Y 3 , : Y 2 , : Y 1 , : ✲ ✲ ✲ ✲ ✲ ❧ u 5 ❧ u 4 ❧ u 3 ❧ u 2 ❧ u 1 Figure 2: In the figure, we den ote as K i, : , Y j, : , X − 1 k, : the i th, j th and k th row of K , Y Q and X − 1 Q resp ectively . F or ev er y non-zero en try of K i, : , Y j, : or X − 1 k, : the co rresp onding signal enters the blo ck wi th an arrow, t h us represen ting the inf ormation flo w from measured outputs to con trol signals. The scheme on the left represen ts the desired sparsity patt ern S for con troller K . The scheme on the righ t represen ts the sparsity pattern of con trollers that are feasible for P S,R ⋆ S , i.e. those in the form Y Q ( X Q ) − 1 with Y Q ∈ Sparse( S ) and X Q ∈ Sparse( R ⋆ S ). Example 1 Motiv ated by the numerical exa mple in [7], let us co ns ider the unstable plant G =         u ( σ ) 0 0 0 0 u ( σ ) v ( σ ) 0 0 0 u ( σ ) v ( σ ) u ( σ ) 0 0 u ( σ ) v ( σ ) u ( σ ) u ( σ ) 0 u ( σ ) v ( σ ) u ( σ ) u ( σ ) v ( σ )         , with u ( σ ) = u ( s ) = 1 s +1 , v ( σ ) = v ( s ) = 1 s − 1 in contin uo us -time or u ( σ ) = u ( z ) = 0 . 1 z − 0 . 5 , v ( σ ) = v ( z ) = 1 z − 2 in dis crete-time , and define P 11 = " G 0 5 × 5 0 5 × 5 0 5 × 5 # , P 12 = " G I 5 # , P 21 = h G I 5 i . Our go al is to design a stabilizing c ontr ol ler K which minimizes k f ( K ) k H 2 and satisfies the sp arsity p att ern b elow: S =         1 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         . This information struct u r e is depicte d in Figur e 2. Here, we apply the prop osed SI approa ch and Algor ithm 1 for spar sit y design in or der to obtain a conv e x restriction of P K . F or this instance, we choose to fix T = S and Γ = I p . Acco rding to Theorem 2 and Corollar y 2 , the tightest conv ex restric tio n of P K such that T R p − 1 = S R p − 1 ≤ S is P S,R ⋆ S , where R ⋆ S R ⋆ S =         1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         , 11 is generated via Algor ithm 1. Given a doubly coprime factorization o f G , an y solution of P S,R ⋆ S is in t he form K = Y Q ( X Q ) − 1 ∈ C stab ∩ Spar se( S ), wher e Y Q ∈ Sparse( S ), X Q ∈ Sparse( R ⋆ S ) and ( X Q ) − 1 ∈ Sparse( R ⋆ S ). R emark 4 (Performanc e i mpr ovement) The classic a l immediate idea would b e to req uire that X Q is diagonal as p er [1 6, 1 7, 27]; instead, SI a llows the off-diag onal entries of X Q = I p + GY Q to b e non-zer o through the optimized choice of R ⋆ S , thus removing unnecessary constra in ts on the entries o f ( GY Q ). This additional freedom can b e s een gr aphically on the right side of Figure 2; the information flow from outputs to control inputs remains the same as the one enco ded by S , but we allow for as man y arr ows as p ossible in the first stag e from outputs to the rows of X − 1 , thus maximizing the degr ees of freedom in the optimization. In Section 5 w e will numerically solve P S,R ⋆ S for this example a nd show that per formance improv emen t over the metho d of [21] is obtained. 4 Bey ond Quadratic In v ariance W e start by rec a lling the w ell-known notion of Quadra tic Inv ar iance (QI) [7] in Subsection 4.1, and its application to the design of g lo bally optimal [7] and sub-o ptimal [21] distributed dynamic output-feedback controllers in Subsection 4.2. In Subsectio ns 4.3, 4 .4 w e sho w that the s ug gested SI no tio n strictly go es beyond that o f QI for sparsity cons tr aints: 1) the controllers obtained using the SI no tion p erform at le a st as well a s those obtained b y [7] and [2 1]; 2) we show through ex amples that using the SI notion w e can recov er glo ba lly optimal controllers even when QI do es not hold, a nd tha t strict per formance improvemen ts ov er [21] ca n b e obtained in gener al. Last, in Subsection 4 .5, we discus s the applica bilit y o f SI to computing distributed static co ntrollers, wher eas the QI notion is not applica ble. 4.1 Quadratic In v ariance The celebrated work of [7 ] characterized conditions on G and Sparse( S ) under which P K admits an exa ct conv ex reformulation in the Y oula para meter Q , denoted a s quadr atic invarianc e (QI). Definition 2 (Quadratic i nv ariance [7]) A subsp ac e K ⊆ R m × p c is QI with r esp e ct t o G if K GK ∈ K , ∀ K ∈ K . F or the purp ose of this pa per , we will limit our fo cus to QI sparsity subspaces in the fo rm Sparse( S ). It is shown that giv en a con troller K nom ∈ Sparse( S ) that stabilizes G and is itse lf stable, there exists a parametriza tion such that K ∈ Sparse ( S ) ⇔ Q ∈ Sparse( S ) [7]. According ly , a con vex optimization pro blem equiv alent to P K is obtained. The req uir ement of a stable a nd stabilizing controller K nom was removed in [2 8]. One main result fro m [28] is as fo llows: Theorem 3 (Theorem IV.2 of [ 28]) Consider any doubly-c oprime factorization of G and let S p arse ( S ) b e QI with r esp e ct to G . Th en, the fol lowing two st atements hold: 1. If Q ∈ RH m × p ∞ is such that Y Q ∈ Sp arse ( S ) , then K = Y Q X − 1 Q is a stabilizing c ontr ol ler in S p arse ( S ) . 2. F or any K ∈ C stab ∩ S p arse ( S ) ther e exists Q ∈ RH m × p ∞ for which Y Q ∈ Sp arse ( S ) and K = Y Q X − 1 Q . 12 According to Theorem 3, if Sparse( S ) is QI with resp ect to G , then P K can b e eq uiv alently reformulated as minimize Q ∈RH m × p ∞ k T 1 − T 2 QT 3 k (12) sub ject to (5 ) , (6) , Y Q ∈ Sparse( S ) . The optimal solution Q ⋆ of (12) can b e used to r ecov er the glob al ly optimal solution K ⋆ of P K via K ⋆ = Y Q ⋆ X − 1 Q ⋆ . 4.2 Con vex restrictions for non-QI sparsity patterns When Spa rse( S ) is not QI w ith resp ect to G , the authors of [21] prop ose d finding a binary ma trix T QI < S such that Spars e( T QI ) is QI with r esp ect to G . Then, the constra int Y Q X − 1 Q ∈ Sparse( S ) of problem P Q can be replaced by Y Q ∈ Spars e( T QI ), a nd any feasible Q for this co n vex progra m will corr esp ond to a feasible controller K = Y Q X − 1 Q ∈ C stab ∩ Sparse( T QI ) ⊆ C stab ∩ Spar se( S ) . (13) This inclusion (13) directly follows from Theorem 3 and the fact that Spar se( T QI ) ⊂ Sparse( S ). A c hallenge of this approach is to compute T QI such that Sparse( T QI ) is QI and as close as p os sible to S in order to reduce conserv atism, in the sense that k S k 0 − k T QI k 0 is minim ized. In general, there might b e multiple choices of T QI with the same car dinality . F urthermore, the Q I condition T QI ∆ T QI ≤ T QI of [7, Theorem 26], where ∆ = Struct( G ), is nonlinear in T QI . F or these reaso ns, a pr o cedure to compute a closest QI subset o f S in p olyno mial time w as not provided in [21]. Instead, w e have shown that the po lynomial time Algo rithm 1 can b e combined with the SI notion to find a conv ex restric tio n for any given T ≤ S . In the next subsections, we show that the recovered c o nt roller s p erform at leas t as well as those based o n the notion of QI by choosing T ≤ S appropr iately , and can be stric tly more p erforming in ge ner al even with the trivial choice T = S . 4.3 Connections of SI with QI Here, we show that it is not necessary to c heck the QI prope r ty in order to obtain a g lobally optimal so lution. Note that c he cking the pr o p e r ty of Q I b efore solving P K was prop ose d in [7] and required in man y subsequent works. Indeed, the approa ch in [7] is guaranteed to yie ld fea sible solutions for P K only if QI holds. Instea d, our techn ique can b e direc tly applied given S without fir s t chec king QI. This result is summarized in the following theorem and cor ollary . Theorem 4 L et ∆ = Stru ct ( G ) and let R ⋆ S b e t he binary matrix gener ate d by Algo rithm 1 with T = S . The fol lowing statement s ar e e quivalent. i) Sp arse ( S ) is QI with r esp e ct t o G . ii) R ⋆ S ≥ I p + ∆ S , wher e R ⋆ S is gener ate d by Algori thm 1 with T = S . Pro of i) ⇒ ii): Supp ose that Spa rse( S ) is QI with resp ect to G . W e hav e that S ∆ S ≤ S by [7, Theo rem 26], implying that S ( I p + ∆ S ) ≤ S a nd ultimately S ( I p + ∆ S ) p − 1 ≤ S. 13 In addition, w e hav e that R ⋆ S ≥ I p and S R ⋆ S ≤ S by construction. It follo ws tha t S ( R ⋆ S ) p − 1 ≤ . . . ≤ S R ⋆ S ≤ S . Also, according to Theor em 2, we hav e R ⋆ S ≥ R , ∀ R ≥ I p such that S R p − 1 ≤ S . By p osing R = I p + ∆ S , we have shown ab ov e that S R p − 1 ≤ S . Hence, R ⋆ S ≥ R = I p + ∆ S . ii) ⇒ i): Supp ose that R ⋆ S ≥ I p + ∆ S , which implies ( R ⋆ S ) p − 1 ≥ ( I p + ∆ S ) p − 1 . By de finitio n of R ⋆ S , we hav e o bserved that S ( R ⋆ S ) p − 1 ≤ S . It follows tha t S ( I p + ∆ S ) p − 1 ≤ S ( R ⋆ S ) p − 1 ≤ S . (14) Combining (14) with the fact that ( I p + ∆ S ) ≥ I p , we hav e S ( I p + ∆ S ) ≤ S ( I p + ∆ S ) p − 1 ≤ S . This implies S ∆ S ≤ S which is equiv alent to QI by [7 , Theorem 26]. Corollary 3 The fol lowing statements ar e e quivalent. i) Sp arse ( S ) is QI with r esp e ct t o G . ii) P K is e quivalent to P S,R ⋆ S with Γ = I p , wher e R ⋆ S is the binary matrix gener ate d by Algorithm 1 with T = S . Pro of It is well-known [8, 28] that (12) is eq uiv alent to P K if and only if QI ho lds. It rema ins to show that P S,R ⋆ S is equiv alent to (12) if and only if Q I holds. W e first show that X Q lies in Spars e( I p + ∆ S ) for every Q ∈ RH m × p ∞ such that Y Q ∈ Sparse( S ). Indeed, by (8) we hav e X Q = I p + GY Q for e very Q ∈ RH m × p ∞ and thus X Q ∈ Sparse( I p + ∆ S ). W e hav e shown in Theorem 4 that QI is eq uiv alent to R ⋆ S ≥ I p + ∆ S , where R ⋆ S is g enerated by Algorithm 1. It follows that the constr a int Y Q Γ = Y Q ∈ Sparse( S ) makes the constraint X Q Γ = X Q ∈ Sparse( R ⋆ S ) redundant and thus P S,R ⋆ S with Γ = I p is equiv alent to (12). This concludes the pr o of. Essentially , Theor e m 4 shows that QI is equiv alent to R ⋆ S ≥ I p + ∆ S . Since X Q ∈ Sparse( I p + ∆ S ) by (8) when Y Q ∈ Sparse( S ), the cons tr aint X Q ∈ Sparse( R ⋆ S ) b eco mes redundant if and only if QI ho lds and the convex pro gram w e obtain with SI, namely P S,R ⋆ S with Γ = I p , is equiv alent to P K due to the results of [7]. Theorems 1, 2 and 4, and Cor ollaries 1 – 3 can b e summar iz ed as follows. 1. Given any distributed sparsity-constrained control pro blem P K , o ne c a n alw ays cast and solve its conv ex restriction P S,R ⋆ S , where R ⋆ S is gene r ated by Algor ithm 1. 2. If P S,R ⋆ S is feasible, its optimal s o lution is also feas ible for P K , and is certified to b e globally optimal if Spa r se( S ) is QI with re spe c t to G . W e remark that v erifying Q I is optional and can b e done a -p o steriori to c he ck glo bal optimality o f the solution, but QI is not part of the con tr o ller des ig n proce dure in the SI a pproach. Hence, Theor em 4 expands the applica bility of c o nv ex pr ogramming to compute distributed co n trollers for ar bitrary systems and spa rsity pa tterns, while maintaining prev ious global optimality results. Example 2 Consider the unstable system and the spar sity pa ttern S of Ex ample 1. W e can v erify that S ∆ S 6≤ S , where ∆ = Sparse( G ), and hence Sparse( S ) is not QI with resp ect to G . Instea d, let us co ns ider 14 the new spa rsity pattern S 2 =         0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         . (15) W e can verify that S 2 ∆ S 2 ≤ S 2 . Hence, Sparse( S 2 ) is QI with resp ect to G . By applying Algor ithm 1 we obtain R ⋆ S 2 =         1 1 1 1 1 0 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         , I p + ∆ S 2 =         1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         , R ⋆ S =         1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1         , I p + ∆ S =         1 0 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1 0 1 1 1 1 1         . In accorda nce with Theo rem 4 we hav e that R ⋆ S 2 ≥ I p + ∆ S 2 , but R ⋆ S 6≥ I p + ∆ S (s e e the entries highlighted in red). By Cor ollary 3, we co nclude that the conv ex progr am P S 2 ,R ⋆ S 2 with Γ = I p is equiv a lent to P K with the spa rsity constr a int K ∈ Sparse( S 2 ), while P S,R ⋆ S is a conv ex res tr iction of P K for every inv ertible Γ ∈ R p × p c . Next, w e sho w that SI generaliz e s the class of restrictions of [21], based on finding QI subsets of Sparse( S ) which are near est to Sparse( S ). The res ult is a s traightforw ard corolla ry of Theor em 4. Corollary 4 L et Sp arse ( T QI ) ⊆ Sp arse ( S ) b e Q I with r esp e ct to G and let k S k 0 − k T QI k 0 b e minimal as pr op ose d in [21]. Then, ther e exists T ≤ S such that J ⋆ ≤ J QI , wher e J ⋆ is t he minimum c ost of P T ,R ⋆ T with Γ = I p , and J QI is the minimum c ost of pr oblem (12 ) with the c onstr aint Y Q ∈ Sp arse ( S ) r eplac e d by Y Q ∈ Sp arse ( T QI ) . Pro of Let T = T QI . Since Spar s e( T QI ) is QI with re s pec t to G , we hav e R ⋆ T ≥ I p + ∆ T by Theo rem 4 . Hence, for every Y Q Γ = Y Q ∈ Sparse( T ), the matrix X Q = I p + GY Q belo ngs to Sparse( I p + ∆ T ) for ev e r y Q ∈ R m × p ∞ and the constraint X Q Γ = X Q ∈ Sparse( R ⋆ T ) is redundant. It follows that the choice T = T QI achiev es J ⋆ = J QI . Therefore, there ex ists a choice of T such that the o ptimal solution of P T ,R ⋆ T with Γ = I p per forms at least a s well as that o f the problem obtained by considering a near est Q I subset a s sugge s ted in [2 1]. This completes our pr o of. Corollar y 4 prov es th at the class of conv ex restr ic tions consider ed in [21] is a sp ecial case in the framework of SI, obtained by choos ing T = T QI and co mputing R ⋆ T QI with our Algorithm 1. F ur thermore, it is p ossible to choo se T ≤ S to obtain strictly more p er fo rming co nvex restrictio ns, a s we will show n umerically in Section 5. 15 4.4 Strictly B ey ond QI So far, we hav e shown that the SI approach na turally recov ers the previous QI re sults of [7] and [21] as sp ecific cases by using Algo rithm 1. Here and in Section 5, we show through examples the stronge r results that 1. SI can recover globa lly optimal so lutions when QI do es not hold, 2. strictly b etter p erfor mance than the approach o f [21] can be obta ine d. F or point 2), we refer to the numerical results in Section 5. F or p o int 1), we consider a n example taken from [1 4]. Example 3 Consider the o ptimal control pro blem: minimize K ( z ) lim L →∞ 1 L L X t =0 E || x ( t ) || 2 2 sub ject to x ( t + 1) = Ax ( t ) + u ( t ) + w ( t ) , u ( z ) = K ( z ) x ( z ) , K ( z ) ∈ Sparse( A bin ) , where z ∈ e j R , A ∈ R n × n , A bin = Struct( A ) a nd w ( t ) denotes i.i.d. disturbances distributed acco rding to a normal distribution N (0 n × 1 , I n ). The discrete- time transfer function of this system is G ( z ) = ( z I p − A ) − 1 . This pr oblem without the spars it y constr aint on K is known as the LQ R problem. By a dding the sparsity constraint, it is an instance of P K in discrete-time. Notice that QI do es not hold whenever the graph defined by A is strong ly connected b ecaus e ∆ = Struct( G ( z )) = Struct  ( z I n − A ) − 1  is eq ual to 1 n × n in general, and so A bin ∆ A bin 6≤ A bin th us vio lating Q I. The reason to co ns ider a disc rete-time instance of P K is that one c an solve analytically the corresp onding problem where spar s it y co nstraints are removed b y computing a simple Ricc a ti equatio n [29]. It s o happ ens that the optimal solution for this problem is K ( z ) = − A , which is also feas ible and hence globally optimal for P K . Now, consider problem P T ,R with Γ ( z ) = G ( z ), T = A bin and R = R ⋆ A bin . W e can verify that a feasible s o lution for P T ,R is Y Q ( z ) = − A z ( z I n − A ), b ecause Y Q Γ = Y Q ( z I n − A ) − 1 = − A z ∈ Sparse( A bin ) . This implies X Q ( z ) = I n − A z by (8). Hence, X Q ( z ) Γ ( z ) = X Q ( z )( z I n − A ) − 1 = I n z . Since R ⋆ A bin ≥ I n by design (see Algorithm 1), we hav e X Q ( z ) Γ ( z ) ∈ Spar se( R ⋆ A bin ) a s desired. It is immediate to verify tha t the re s ulting controller is K ( z ) = Y Q ( z ) X Q ( z ) − 1 = − A . W e conclude that, despite a lack of Q I, a conv ex approximation which co n tains the glo bal optimum of P K is found by using the prop osed SI appro ach. R emark 5 The g lobal optimality result for this example was also obtained using the SLP in [1 4]. The sparsities for the sy s tem level parameters in [14] were c hosen empirically , while w e provide an e xplicit metho dology based o n the SI condition (10) and Algor ithm 1. F urthermor e, w e wis h to cla rify tha t obtaining global optimality certificates for P K for sys tems with no n-QI cons traints is still an op en problem, which is not addressed neither b y the system level a pproach [14] nor by o ur SI approach. Bo th our approach and that of [14] can cer tify optimality o f the so lution b ecause the optimal so lution of this simple insta nce is already known a nalytically . 16 4.5 SI for static con troller design W e conc lude this section by highlighting ano ther adv antage of the SI notion o ver the QI notion; the SI notion can b e us e d to compute spar se s tatic control p olicies in a con vex wa y , that is p o lic ie s in the form u ( t ) = K y ( t ) where K is a re al matrix in Spars e( S ). This topic has b een thor oughly studied in our earlier work [18], where we derived a notion of SI limited to the static controller case. H ere, we highlig h t that in contrast to the QI notion, SI is useful b oth for static and dyna mic spars e controller design. The main observ a tion is that the Y oula pa rametrizatio n canno t a chiev e a conv exification o f the static controller design problem in genera l, b ecause enforcing K = ( V r − M r Q )( U r − N r Q ) − 1 to b e a r e al matrix is a non-convex requirement on the tr ansfer matrix Q . Consequent ly , a different para metrization should b e used and the QI pro per t y , tightly linked to using a Y oula-like parametrizatio n, will not b e relev ant an ymore. The most well-known tec hniques to conv exify the H 2 and H ∞ norm-optimal state-feedback sta tic controller design problems ar e based on co mputing appropr ia te quadratic Lyapunov functions through Linear Matrix Inequalities (LMI); see [30, 3 1] for a co mprehensive rev iew. The more gener al c ase of static output-feedback is known to b e NP-hard [5 ] and an exact conv ex formulation do e s not exist. As we illustrated in [18], when the distr ibuted s tatic control pro ble m is for m ulated through LMIs, the controller is recov ered as K = Y X − 1 , where Y and X a re rea l decision v aria bles, X is symmetric p ositive semidefinite and V ( x ) = x T X − 1 x is a quadratic Lyapuno v function for the closed-lo op system. If the controller must lie in a spar s it y subspace Sparse( S ), the o nly source of non- conv exity stems from requir ing that Y X − 1 ∈ Sparse( S ). This expressio n fo r the static controller in terms of the decisio n v aria bles matches that of K = Y Q X − 1 Q , which is v alid for dynamic co ntrollers in terms o f the Y oula parameter. According to Theorem 1 a nd Corollar y 1 , convex restr ictions ca n b e obtained by choo sing binary matrices T a nd R as per (10) that satisfy the SI condition (9 ), and req uiring that Y Γ ∈ Spar se( T ) and X Γ ∈ Spar se( R ) for any inv ertible real matrix Γ ∈ R n × n . W e refer the int erested reader to [1 8] for details. Based on the discussion a b ove, SI is a fra mework-indep endent notion which deals with spar sit y patterns. Spec ific a lly , the SI notion tra nslates, separately , to genera lizations of Q I-based synthesis of spa rse dy na mic controllers and o f blo ck-diagonal qua dratic Lyapunov functions for designing s parse static co n trollers. 5 Exp erimen ts With the go al o f providing insight in to our prop ose d metho d and showing its p otential b enefits when co m- bined with standar d controller desig n tec hniques, we co ntin ue here our Exa mple 1 and provide numerical results. 5.1 Finite-dimens ional appro ximation Since the conv ex programs we ha ve ca st are infinite-dimensional, due to the decisio n v ariables b eing transfer matrices whos e order is not fixed, it is necess ary to res ort to finite-dimensional approximations. When using the Y oula parametriza tion in co nt inuous-time, one can adapt the semidefinite pr ogra mming tec hnique of [32] to the H 2 norm b y exploiting standar d results fro m [31, 3 3]; when using the SLP or IOP parametriza tions in discrete-time, one can use the co rresp onding finite impulse resp onse (FIR) approximations o f [14, 25 ]. The key c o mmon idea b ehind these approximations is to expr ess each dec is ion v ar iable U , which is a general 17 stable tr a nsfer ma tr ix in contin uous-time (res p. discrete-time), in the approximated for m U = N X i =0 U [ i ]( s + a ) − i , resp. U = N X i =0 U [ i ] z − i ! , (16) for some N ∈ N a nd a ∈ R with a > 0. The real matrices U [ i ] for all i beco me the finitely many rea l decision v ariables to optimize over. The appr oximation (16) is ba sed o n the well-kno wn idea of Ritz approx- imations [3 4] a nd we refer the r eader to [14, 25] for deta ils on SLP and IO P . Example 1 (c ontinue d) W e will address the distributed controller design pro blem for m ulated in Example 1 bo th in discrete- and con tin uous-time. W e hav e observed in E xample 2 that Sparse( S ) is not QI with r esp ect to G . As we hav e summarized in Section 4.2, [2 1 ] sug gests identifying a binary matrix T QI < S such that Sparse( T QI ) is QI with resp ect to G and k S k 0 − k T QI k 0 is minimized. In this case, we v erify by ins p ectio n that S 2 in (15) is the only QI spar sity patter n T QI such that k S k 0 − k T QI k 0 ≤ 2 . As sugg ested in [21], we can thus s ubstitute the constr aint Y Q ( X Q ) − 1 ∈ Sparse( S ) with Y Q ∈ Sparse( S 2 ) and the corresp onding conv ex pro gram is a restr ic tion of P K . Our g oal is to co mpa re tig h tness o f this co nv ex restric tion with tha t of P S,R ⋆ S obtained thro ugh SI. 5.2 Numerical Results As outlined ab ov e, w e solved finite-dimensional appro ximations o f t he con vex re s triction pr op osed in [21] and of our co nv ex restriction P S,R ⋆ S with Γ = I p obtained through SI. All the numerical progr ams were so lved with MO SEK [3 5], ca lled throug h MA TLAB via Y ALMIP [36], o n a standa rd lapto p computer. 5.2.1 IOP in discrete-ti me In our fir st exp eriment we consider ed the dis crete-time version of G . Since the approa ch of [32] requires finding a n initial s table a nd stabilizing controller in Spa r se( S ) heuristica lly , which is no trivial task in general, we used the IOP par ametrization [2 5] and the discrete-time finite-dimensio na l approximation (16) for a ll decision v ariables . Using the no tation of [26], wher e K = UY − 1 and U , Y a re input-output para meters, the closest QI s ubset approa ch of [21] requires U ∈ Sparse( S 2 ), while our SI approach transla tes to U ∈ Sparse( S ) and Y ∈ Sparse( R ⋆ S ). Within this setting, no feasible solution co uld be obtained using the clos e st QI s ubset approach; instead, up on conv ergence over N , we obtained a co st of 6 . 72 78 using the pr op osed SI appr oach. T o ev aluate the subo ptimalit y , w e additio nally solved fo r the nearest QI sup erset of S defined as the binary matrix S 3 ≥ S such that S 3 is Q I and k S 3 k 0 − k S k 0 is minimized [21]; the corr esp o nding optimal co st ser ves as a low er b o und for that of P K . The QI sup erset is unique and is computed with the algor ithm (13 )-(14) of [21]. It turns out that S 3 is the full lower-triangular matrix. By so lving for S 3 we obtained the low er bo und 6 . 7268 up on conv ergence ov er N , and hence the SI s olution has ne a r-optimal p erfor mance. 5.2.2 Y oula i n cont inuous-time In our second exp er imen t we considered the contin uo us-time version o f G and used the finite-dimensional approximation technique of [3 2 ]. A doubly-coprime factorization of G was computed as p er [7, Theorem 17] using the stable and stabilizing controller K nom suggested in [7, Page 1 995]. In (16), we chose a > 0 and increased the v a lue of N until the improv emen t on the cost was neg lig ible, thus a ppr oaching conv er gence to the optimal cos t of the infinite-dimensiona l progr am. Up on con vergence over N , th e c losest QI subset m etho d of [21] led to a cost of 7 . 336 7 while the SI metho d led to a co st o f 7 . 309 8. T o ev alua te this improv emen t in 18 per formance, we additionally solved for S 3 and obtained a low er b o und of 7 . 216 3. W e conclude that our SI solution ha s a relative improvemen t ov er that of [2 1] ba s ed on QI subsets of a t least 7 . 3367 − 7 . 3098 7 . 3367 − 7 . 2163 = 22 . 3%. 6 Conclusions W e hav e pr op osed the fra mework of Sparsity Inv ariance (SI) for convex design of optimal a nd nea r-optimal sparse controllers. One main insigh t is that the propo sed SI approach offers a dir ect generaliz ation of pr evious design metho ds ba sed on the no tion of Q uadratic Inv ar ia nce (QI). Indeed, SI can b e dir ectly applied to any systems and spa rsity cons traints. The recovered solution is globa lly optimal when QI ho lds a nd p erforms at least a s well as the nearest QI s ubset when Q I do es not ho ld. W e hav e shown the p otential b enefits of SI over previous metho ds throug h examples, and remarked that SI is natura lly applicable to spar se s tatic controller design. Since the condition (10) is necessary and sufficien t for the SI prop erty (9), our results approach the limits in p erfor mance o f co n vex res trictions o f the spa r sity constr a ined control pr oblem base d on structura l conditions for the Y oula parameter. This op ens up the ques tio n of whether different and more p erforming design metho dologie s can b e develop ed for this c hallenging problem. Another dir ection for research is to further r efine the SI approach, by developing tractable heuristics to optimally design the bina ry matrices T and R and the pa rameter Γ simultaneously based on the knowledge of the s ystem P . This could po ten tially improv e upon A lgorithm 1 . Fina lly , it would b e r e lev ant to extend the SI idea to the case o f delay constraints; in dis crete-time, this might b e p ossible by r efining the results of [37]. A App endix A.1 Pro of of Theorem 1 The pro of r e lies on tw o Lemmas, whose pr o ofs are rep orted in App endix A.2 and Appendix A.3. Lemma A1 L et R ∈ { 0 , 1 } p × p with R ≥ I p . Th en, 1. F or any invertible tr ansfer matrix X in Sp arse ( R ) , Struct  X − 1  ≤ R p − 1 . 2. Ther e exists an invertible tr ansfer matrix X ∈ Sp arse ( R ) such that Struct  X − 1  = R p − 1 . Lemma A2 L et T ∈ { 0 , 1 } m × p and R ∈ { 0 , 1 } p × p , and Struct ( W ) = R . Then, t her e exists Z ∈ Sp arse ( T ) such that Struct ( ZW ) = T R . W e are now rea dy to prov e Theorem 1. 1) ⇒ 2 ): Let X ∈ Sparse( R ) b e inv er tible. By Lemma A1 we k now that X − 1 ∈ Sparse( R p − 1 ). Now let Y ∈ Sparse( T ). Since T R p − 1 ≤ S , w e have YX − 1 ∈ Sparse( S ). 19 2) ⇒ 1 ): W e pr ov e by c o nt rap ositive. First, suppos e that T R p − 1 6≤ S . By the second statement of Lemma A1 it is poss ible to select X ∈ Sparse( R ) such that Struct( X − 1 ) = R p − 1 . By the latter a nd Lemma A2, we ca n select Y ∈ Sparse( T ) s uch that Struct  YX − 1  = T R p − 1 , or equiv alently YX − 1 6∈ Sparse( S ). Nex t, supp ose that T 6≤ S . Since R ≥ I p by hypothesis , then T R 6≤ S and T R p − 1 6≤ S . Hence, the s ame re asoning applies . A.2 Pro of of Lemma A 1 Suppo se X ∈ Sparse( R ) is inv er tible. B y Cayley-Hamilton’s t heorem P n i =0 λ i X i = 0 where { λ i } p i =0 , λ i ∈ R c for every i = 1 , . . . , p a r e the co efficients o f the characteris tic po lynomial of X and λ 0 = det X 6 = 0. W e remark that Cayley-Hamilton is v a lid over squar e matr ic es defined ov er a commutativ e ring, such as that of causal transfer functions [38]. By pr e-multiplying b y X − 1 and re a rrang ing the terms: X − 1 = − λ − 1 0 ( λ 1 I p + λ 2 X + λ 3 X 2 + · · · + λ p X p − 1 ) . (17) Since R ≥ I p we hav e that R a ≥ R b for every integer a ≥ b . Hence, λ i X i ∈ Spa rse  R p − 1  for every i and the fir st statement follows by (17). F or the seco nd statement, we iteratively cons truct X starting from X = I p . Let α ∈ R c . Define ˜ X = X + αe i e T j . Let X − 1 : ,i ∈ R p × 1 c and X − 1 j, : ∈ R 1 × p c be the i -th column a nd the j - th row of X − 1 resp ectively , and let X − 1 ij be the entry ( i, j ) of X − 1 . Using the Sherman-Mor rison identit y [3 9], if ˜ X is in vertible we obtain ˜ X − 1 i, : = X − 1 i, : − α X − 1 ii 1 + α X − 1 j i X − 1 j, : . (18) Recall that ea ch entry o f a transfer matrix is a transfer function defined o ver s = j ω . Hence, by the definition of an inv er tible tr ansfer matrix (see Section 2), (18) ho lds for a lmost every ω ∈ R . F ro m (18), for any i and α ∈ R c , if X − 1 ii 6 = 0, then ˜ X − 1 ii 6 = 0. It follows that by choosing α such that α X − 1 j i 6 = − 1 and α  X − 1 ii X − 1 j k − X − 1 j i X − 1 ik  6 = X − 1 ik for a lmost all ω ∈ R , ∀ k sub ject to X − 1 j k and X − 1 ik are not both null , (19) we obta in tha t Struct  ˜ X − 1 i, :  = Struct  X − 1 i, :  + Struct  X − 1 j, :  , (20) for a lmost all ω ∈ R . The co ndition (19) is derived by setting the right hand s ide of (18) to b e differ en t from 0 for every k such that X − 1 ik and X − 1 j k are not both n ull for every ω ∈ R . Observe that α as p er (19) alwa ys exists, because there is no k such that X − 1 ik and X − 1 j k are b oth n ull for every ω ∈ R , and hence α  X − 1 ii X − 1 j k − X − 1 j i X − 1 ik  6 = X − 1 ik alwa y s admits a solution in α ∈ R c . The structural augmentation (20) is ex plo ited in the algorithm b elow. 20 1: Set X = I p 2: rep eat ⊲ max . ( | R | − p )( p − 1) itera tions 3: for eac h ( i, j ) such that i 6 = j a nd R ij = 1 do 4: Cho ose α a ccording to (19) 5: X ← X + αe i e T j 6: end for 7: until Struct( X − 1 ) = R p − 1 8: Return X The algor ithm returns a matrix X such tha t Str uct( X − 1 ) = R p − 1 . Sp ecifically , by exploiting (20) we obtain that Struct( X − 1 ) ≥ R s at the end of the s -th itera tio n of the “rep eat-until” cycle. A.3 Pro of of Lemma A 2 Let Z b e any tra nsfer ma trix in Sparse( T ). Assume that Struct( ZW ) < T R . Then, for some ( i, j, k ) we hav e that ZW ij = 0 and T ik = R kj = 1. W e know by hypothesis that W kj 6 = 0. Since P p l =1 Z il W lj = 0, it is sufficient to upda te Z ik with Z ik + α for any α 6 = 0 in R c to guarantee that ZW ij 6 = 0. F urther more, by choosing α 6 = − ZW it W kt for all t s uc h tha t ZW it 6 = 0, we av o id that adding α to Z ik brings ZW it to 0 when ZW it 6 = 0. Hence, it is alwa y s possible to choose k and α such that ZW + αe i e T k > ZW and Z ∈ Sparse( T ). By iterating the pro cedur e for all ( i, j ) such that Struct( ZW ) ij < T R ij , we conv erge to Struct ( ZW ) = T R . References [1] F. D¨ orfler, M. R. Jov a novi ´ c, M. Chertko v, and F. Bullo, “Sparsity-promoting optimal wide-ar ea con trol of p ow er netw orks,” IEEE T r ans. on Pow. Syst. , vol. 29 , no. 5, pp. 2281– 2291, 20 1 4. [2] T. P . Pres cott and A. Papachristodoulo u, “ Lay er ed dec ompo sition for the mo del order reduction of timescale separ ated bio chemical reaction netw orks,” Journal of the or etic al biolo gy , vol. 35 6 , pp. 1 13– 122, 2 0 14. [3] Y. Zheng, S. E. Li, K . Li, F. Bo rrelli, and J. K . Hedr ick, “Distributed mo del predictive con trol for heterogeneo us vehicle pla to ons under unidirectional top olo gies,” IEEE T r ansactions on Contr ol Systems T e chnolo gy , vol. 2 5, no. 3, pp. 8 9 9–910 , 20 17. [4] H. S. Witsenhausen, “A counterexample in sto chastic optimum c o nt rol,” S IA M Journal on Contr ol , vol. 6, no. 1, pp. 13 1–147 , 19 68. [5] V. D. Blo ndel and J. N. Tsitsiklis , “A survey of computational complexity r esults in systems and control,” Automatic a , vol. 36, no. 9, pp. 1249– 1274, 2000. [6] C. H. Papadimitriou and J. Tsitsiklis, “Intractable proble ms in c ontrol theor y ,” SIAM jour. on c ontr. and opt. , vol. 2 4, no . 4, pp. 639– 654, 1 986. [7] M. Rotko witz a nd S. Lall, “A c ha racteriza tion of con vex problems in decen tralized cont rol,” IEEE T r ansactions on Automatic Contr ol , vol. 5 1, no. 2, pp. 274– 2 86, 20 06. 21 [8] L. Les s ard and S. Lall, “ Quadratic inv ar iance is necessa ry and sufficient for conv e xit y ,” in A meric an Contr ol Confer en c e (ACC), 2011 . IEEE, 20 11, pp. 5360 –5362 . [9] N. Matni, “Communication delay co-design in H 2 -distributed con trol using atomic nor m minimization,” IEEE T r ansactions on Contr ol of Network Syst ems , vol. 4, no. 2, pp. 26 7–278 , 2015 . [10] N. Matni and V. Chandr asek ara n, “Regular ization for design,” IEEE T r ansactions on Automatic Con- tr ol , vol. 6 1, no. 12, pp. 3991– 4 006, 201 6. [11] G. F azelnia, R. Mada ni, A. Kalbat, and J. Lav aei, “Conv ex relax a tion for optimal distributed co n trol problems,” IEEE T r ansactions on Automatic Contr ol , vol. 62, no. 1, pp. 206 –221, 2017 . [12] Y. W a ng, J. A. Lop ez, and M. Sznaier, “Con vex optimization approaches to information structured decentralized co n trol,” IEEE T r ansactions on Automatic Contr ol , 20 18. [13] K . Dvijotham, E . T odo rov, and M. F azel, “Co nvex s tructured controller design in finite horizo n,” IEEE T r ansactions on Contr ol of N etwork Systems , vol. 2, no . 1, pp. 1–10, 2 015. [14] Y.-S. W ang, N. Matni, and J. C. Doyle, “A system level approach to controller syn thesis,” IEEE T r ansactions on Automatic Contr ol , 2 019. [15] S. Sridhar, A. Hahn, M. Govindarasu et al. , “Cy ber -physical sys tem security for the electric power grid.” Pr o c e e dings of the IEEE , vol. 1 00, no. 1, pp. 210–2 24, 20 1 2. [16] C. Co nt e, N. R. V o ellmy , M. N. Zeilinger, M. Mora ri, and C. N. Jones, “Distributed synthesis a nd control o f constra ined linear systems,” in Americ an Contro l Confer enc e (ACC) , 2012 , pp. 6 017–6 022. [17] Y. Zheng , M. K amgarp our , A. So otla, and A. Papachristodoulo u, “Conv ex design o f str uctured con- trollers us ing blo ck-diagonal Lyapuno v functions,” arXiv pr eprint arXiv:1709 .00695 , 2017 . [18] L. F ur ieri, Y. Zheng, A. Papachristodo ulo u, and M. K amgarp our , “On separ a ble quadra tic Lyapunov functions for convex design of distributed co n trollers ,” in Eu ro p e an Contr ol Confer enc e (ECC) , 20 1 9, pp. 4 2–49. [19] F. Lin, M. F a rdad, and M. R. Jov a novi ´ c, “Design o f optimal spa rse feedback ga ins via the alternating direction metho d of mult ipliers,” IEEE T r ans actions on Automatic Contr ol , vol. 58, no. 9, pp. 2426– 2431, 201 3. [20] N. K . Dhingra, M. R. Jov anovi ´ c, a nd Z.-Q. Luo, “An ADMM algo rithm for optimal sensor and actuator selection,” in IEEE Confer enc e on De cision and Contr ol (CDC) , 201 4 , pp. 4039– 4044. [21] M. C. Ro tkowitz a nd N. C. Ma rtins, “On the nearest quadra tically inv ariant infor mation constraint,” IEEE T r ansactions on Automatic Contr ol , vol. 57, no. 5 , pp. 1314 –131 9 , 2012 . [22] A. F r ancis, A c ourse in H ∞ c ontr ol the ory . Spring er-V erla g, 1 987. [23] A. Alavian and M. Rotko witz, “Stabilizing decent ralized systems with ar bitrary information structure,” in I EEE Confer enc e on De cision and Contr ol ( CDC) . IEEE, 2 014, pp. 403 2–403 8. [24] D. Y oula, H. J abr, and J. Bo ngiorno, “Mo der n Wiener-Hopf design of optimal controllers–Part I I: The m ultiv ariable case,” IEEE T r ansactions on Automatic Contr ol , vol. 21, no. 3, pp. 31 9 –338, 1976 . 22 [25] L. F urier i, Y. Zheng, A. Papachristo doulou, a nd M. Kamga rp our, “An inputoutput parametriza tion o f stabilizing controllers: a midst Y oula and system level synthesis,” IEEE Contr ol S ystems L ett ers , 201 9. [26] Y. Zheng, L. F urier i, A. Papachristodoulo u, N. Li, and M. Ka mgarp our, “On the equiv a le nce of youla, system-level and input-output pa rameterizatio ns,” IEEE T r ansactions on Automatic Contr ol , pp. 1– 8, 2020. [27] J . C. Geromel, J. Ber nussou, and P . L. D. Peres, “ Decent ralized control through parameter space optimization,” Automatic a , vol. 30, no. 10 , pp. 1565 –1578 , 1994. [28] S ¸ . Sa b˘ au and N. C. Martins, “ Y oula-like pa rametrizations sub ject to Q I subspace co nstraints,” IEEE T r ansactions on Automatic Contr ol , vol. 5 9, no. 6, pp. 1411 – 1422 , 20 14. [29] P . La nc a ster and L. Ro dma n, Algebr aic Ric c ati e quations . Clarendon press, 1995 . [30] S. Boyd, L. E l Ghaoui, E. F ero n, and V. B alakrishna n, Line ar matrix ine qualities in system and c ontr ol the ory . Siam, 1994 , vol. 15. [31] C. Scherer and S. W eiland, “Linea r matrix inequa lities in c o nt rol,” L e ctur e Notes, Dutch In stitute for Systems and Contr ol, Delft, The Netherlands , vol. 3, no . 2 , 20 00. [32] A. Alavian and M. C. Rotkowitz, “Q -parametriz ation a nd an SDP for H ∞ -optimal decentralized con- trol,” in IF AC Worksh. on Dist ribute d Est imation and Contr. in Networke d Syst ems , 2013 , pp. 301– 308. [33] C. W. Scherer, “An efficien t solution to m ulti-ob jective c ontrol pro blems with LMI ob jectiv es,” Systems & Contr ol letters , vol. 4 0, no . 1 , pp. 43– 5 7, 2 000. [34] S. Boyd and C. Barratt, “Linear con troller design: limits o f perfor mance,” Stanford Univ ersity Stanfor d United States, T ech . Rep., 19 9 1. [35] MO SEK Aps, “The MOSEK optimization to o lbox for MA TLAB manual. Versio n 8.1.” 2017. [36] J . L¨ ofb erg, “Y ALMIP : A T o olb ox for Mo deling and Optimization in MA TLAB,” in In Pr o c. of the CAC SD Conf. , T aip ei, T a iwan, 2 004. [37] L. F urieri and M. K amgarp our , “Robust distributed co ntrol b eyond q uadratic in v aria nc e ,” in IEEE Confer enc e on De cision and Contr ol (CDC) , 2018, pp. 37 28–3 7 33. [38] P . Picar d, J. Lafay , and V. K ucera, “ F eedback rea liz ation of precomp ensator s for linear systems with delays,” in IEEE Confer enc e on De cision and Contr ol (CDC) , vol. 2, 1995 , pp. 2 035–2 040. [39] J . Sherman and W. J. Morriso n, “Adjustment of a n inv ers e matrix co rresp onding to a change in o ne element of a given matrix,” The Annals of Mathematic al Statistics , vol. 21 , no. 1, pp. 124–1 2 7, 195 0. 23