A Quasi-separation Principle and Newton-like Scheme for Coherent Quantum LQG Control

A quasi-separ ation principle and Newton-l ike scheme f or coherent quantum LQG contr ol 1 Igor G. Vladimirov a , Ian R. Petersen a a School of Engineering and Informatio n T echnology , T h e University of Ne w Sou th W ales at the Austr alian Defence F or ce Academ y , Canberra, A CT 2600, A ustralia, E-mail: igor.g.vladi mirov@gmail .com (I.G.Vladimir ov), i.r.petersen@g mail.com (I.R.P eter sen). Abstract This paper is concerned with constructing an opti mal controller in the coherent quantum Linear Quadratic Gaussi an problem. A coherent quantu m controller is itself a quantum system and is required to be physically realizable. The use of co- herent control a voids t he need for classical m easurements, which inherently entail the loss of quant um in formation. Physical realizability corresponds to the equi v- alence of the controll er to an open quantum harmonic os cillator and relates its state-space matrices t o the Hamiltonian, coupl ing and scattering operators of the oscillator . The H amiltonian parameterization of the controller is combin ed with Frechet dif ferentiation of the LQG c ost wit h re sp ect to the s tate-space matrices to obtain equati ons for the o ptimal controll er . A quasi-separation principl e for the gain matrices of the quantum controller is est ablished, and a Ne wton-l ike it erativ e scheme for numerical solution of the equations is outlined. K e ywor ds: quantum control, LQG cost, physical realizability, Frechet diffe renti ation 2000 MSC: 81Q93, 49N10, 93E20, 93B52 1. Intr oduction Sensitivity to observation is an inherent feature of qu antum mechanical sys- tems whose state is affected by interaction with a macroscopic measuring device. 1 A shortened version of this work is to app ear in the 1 8th IF A C W o rld Cong ress Proce edings [13]. This motiv ates the use of coherent quantum controllers to replace the classical observation-actuation control lo op by a measurement-free feedback, which is or- ganized as an in terconnection o f the quantum p lant wi th another quantum system. If such a con troller is i mplemented usin g quantum -optical components (for ex- ample, optical ca viti es and beam spl itters) medi ated by light fields [2], then it is dynamically equivalent to an open quantum harmonic osci llator , which consti- tutes a building block of q uantum systems described by l inear quantum st ochastic diffe renti al equations (QSDEs) [7, 8]. This lea ds to the notion of physical realizabilit y which imposes qu adratic c on- straints on the state-space m atrices of the controller [4, 6, 9], thus com plicating th e solution of quantum control p roblems whi ch are ot herwise reduced to appropri- ate unconstrained problems for an equi valent classical system. The links between classical control prob lems and their quant um analogues are kn own, for example, for Linear Quadratic Gaussian (LQG) and H ∞ -control. The Coherent Quantum LQG (CQLQG) p roblem seeks a phys ically realizable quantum controller to minimize the av erage outp ut “ener gy” of the cl osed-loop system per unit tim e. This problem has been addressed in [6], where a numeri- cal procedure was proposed for finding subopt imal controllers to ensure a given upper bound on the L QG cost. Instead, the present p aper focuses on necessary conditions f or optimality and second order c ond itions for local strict optimality of a physically realizable controller and computation of the opt imal controller . Both approaches make use of the fact that the CQLQG problem is equiv alent to a con- strained LQG problem for a class ical plant, with the LQG cost comput ed as the squared H 2 -norm of the s ystem in t erms of the con trollabilit y and observability Gramians satisfying algebraic L yapunov equations. W e utili ze a Hamilt onian parameterization t hat relates th e state-space matrices of a physically realizable controller to the fr ee Hamil tonian, coupling and scatter- ing o perators of an open quantum harmonic oscillator [1]. T o obtain equ ations for the optimal quantum controll er , we emp loy an algebraic approach, based on the Frechet differ enti ation of the LQG cost with respect to the state-space matrices from [12 ] and similar to [11]. The resulting equations for the opti mal controll er in volve the in verse of special self-adjoint operators on matrices that requi res the use of vectorization [5]. Their spectral properties play an im portant role in t he present study . Although th e o ptimal CQLQG controll er does not inherit the control/ filtering separation principle of the classical LQG c ont rol problem, a partial decoupling of equations for the gain mat rices still ho lds. This quasi-separati on property leads t o a Ne wton-li ke scheme for num erical computation of the quantum controller that 2 in volves the second order Frechet deri vativ e of the LQG cost which is related to the perturbation of solutions to algebraic L yapunov equations. The paper is organised as follows. Section 2 s pecifies the quantum plants being considered. Sections 3 and 4 describe physically realizable quantum controllers. Section 5 form ulates the CQLQG control prob lem. Sections 6 and 7 i ntroduce auxiliary classes of matrices and self-adjoint operators. Section 8 obtains equa- tions for the optimal CQLQG controll er . Section 9 di scusses the quasi-separation property . Section 10 est ablishes a second order con dition of optim ality . Sec- tion 11 outlines a Newton-like scheme for computing the opti mal controller . Ap- pendices provide a s ubsidiary material on in vertibilit y of the sp ecial self-adjoint operators, perturbati ons of in verse L yapunov op erators and Frechet dif ferentiation of the LQG cost. 2. Quantum plant W e consider a quantum plant with an n -dimensional state vector x t , a p - dimensional outp ut y t and inputs w t , η t of dimensio ns m 1 , m 2 . The stat e and the output are governed by the QSDEs: d x t = Ax t d t + B 1 d w t + B 2 d η t , (1) d y t = z t d t + D d w t , (2) z t = C x t . (3) Here, A ∈ R n × n , B k ∈ R n × m k , C ∈ R p × n , D ∈ R p × m 1 are con stant matrices, and z t is a “si gnal part” of y t . The s tate d imension n and the input d imensions m 1 , m 2 are ev en: n = 2 ν , m k = 2 µ k . The plant state vector x t is formed by self- adjoint operators (si milar to th e pos ition and moment um operators) and, in the Heisenberg p icture of quant um mechanics, e volves in time t . The ent ries of the m 1 -dimensional vector w t are self-adjoint q uantum W iener processes [7] whose infinitesimal increments compose with each other according to the Ito table d w t d w T t = F d t. (4) Here, F is a com plex positive semi -definite Her m itian matrix w hich, on the right- hand side of (4), is a shorthand notation for F ⊗ I , wi th I th e identity operator on t he un derlying boson Fock space and ⊗ the tensor product. W e assume that vectors are organized as columns unless ind icated otherwise, and the transpose ( · ) T acts on vectors and matrices with operator -valued e ntri es as if the latter were 3 scalars. Als o, ( · ) † := (( · ) # ) T denotes the t ranspose of the entry-wise adjoint ( · ) # . Associated with the Hermitian matrix F from (4) are real m atrices S := ( F + F ) / 2 = Re F and T := ( F − F ) /i = 2Im F , where ( · ) , Re( · ) and Im( · ) are t he entry-wise complex conjugate, real and imaginary parts, and i := √ − 1 is the im aginary unit. The symmetric matrix S contributes to the e volution of the covariance matrix of the plant s tate vector x t , whilst T is antisymm etric and af fects t he cross-commutations between th e entries of x t through [d w t , d w T t ] := d w t d w T t − (d w t d w T t ) T = ( F − F T )d t = iT d t . Here, the commutator [ α , β ] := αβ − β α appl ies entry-wis e, and t he relation F T = F is ensured b y F = F ∗ . In what follows, i t is assumed that S = I m 1 , and T is canonical in the sense that T := I µ 1 ⊗ J , J :=  0 1 − 1 0  , (5) where I r is the identi ty matrix of order r . That is, T is a block diagonal matrix with µ 1 copies of J over the diagon al. By permutin g t he rows and colu mns, the matrix T from (5) can be brought to an equiv alent canonical form T = J ⊗ I µ 1 =  0 µ 1 I µ 1 − I µ 1 0 µ 1  , (6) where 0 r denotes the ( r × r ) -matrix of zeros. The ca non ical antis ymmetric matri x J o f any order s atisfies J 2 = − I . Qu antum W iener processes will be assumed to hav e the canonical Ito matrix F = I + iJ / 2 . 3. Coher ent quantum contr oller A measurement-free coherent quantum controller is another qu antum sys- tem wit h a n -dimensional state vector ξ t with self-adjoint operator- valued entries whose interconnection with the plant (1)–(3) is described by QSDEs d ξ t = aξ t d t + b 1 d ω t + b 2 d y t , (7) d η t = ζ t d t + d ω t , (8) ζ t = cξ t . (9) Here, a ∈ R n × n , b 1 ∈ R n × m 2 , b 2 ∈ R n × p , c ∈ R m 2 × n , and ω t is a m 2 -dimensional vector of self-adjoint quantum W iener p rocesses whi ch commut e with the plant noise w t in (1) and (2). The combined set of equations (1)–(3) and (7)–(9) de- scribes the fully quantum closed-loop system in Fig. 1, whose output observables 4 plant contro ller ✛ ✛ ✛ ✛ w η ω y Figure 1: The qu antum closed-loop system described by (1)–(3) and (7)–(9), where the plant and controller noises w a nd ω are commu ting quantum W iener processes. form a p 0 -dimensional process Z t = C 0 x t + D 0 ζ t , (10) where C 0 ∈ R p 0 × n and D 0 ∈ R p 0 × m 2 are giv en matrices. The 2 n -dimension al combined state vector X t := [ x T t ξ T t ] T and the ou tput Z t of the closed-loop s ystem are gove rned by the QSDEs d X t = AX t d t + B d W t , Z t = C X t . (11) Here, the combined quant um W iener process W t := [ w T t ω T t ] T has a block di - agonal Ito table. Th e matrices A , B , C of the closed-loo p s ystem (11) are giv en by  A B C 0  =   A B 2 c B 1 B 2 b 2 C a b 2 D b 1 C 0 D 0 c 0 0   =   A B 2 c B b C a b D C 0 D 0 c 0   , (12) where b :=  b 1 b 2  , B :=  B 1 B 2  , C :=  0 C  , D :=  0 I D 0  . (13) The dependence of A , B , C on t he controller matrices a , b , c is equiv alently de- scribed by Γ :=  A B C 0  = Γ 0 + Γ 1 γ Γ 2 , γ :=  a b c 0  . (14) The affine m ap γ 7→ Γ is completely specified by the plant (1)–(3) t hrough the matrices Γ 0 :=   A 0 B 0 0 n 0 C 0 0 0   , Γ 1 :=   0 B 2 I n 0 0 D 0   , Γ 2 :=  0 I n 0 C 0 D  . (15) 5 Using th e terminolo gy introduced form ally in Section 7, the map γ 7→ Γ 1 γ Γ 2 in (14) is a grade one linear operator [ [ [Γ 1 , Γ 2 ] ] ] . 4. Physical r ealizability A controller (7)–(9) is called physically r ealizabl e (PR) [4, 6], i f its state-space matrices satisfy aJ 0 + J 0 a T + bJ b T = 0 , b 1 = J 0 c T J 2 . (16) Here, J i s a block-diagonal mat rix, partitioned in conformance with th e m atrix b from (13) as J := D  J 1 0 0 J 2  D T =  J 2 0 0 D J 1 D T  , (17) and J 0 , J 1 , J 2 are fixed real ant isymmetric matrices of orders n , m 1 , m 2 , which specify the commu tation relations for the con troller state variables ξ t and the plant and controller noises w and ω . For con venience, J 0 , J 1 , J 2 are assumed to have the canonical form (5) or (6). The relations (16) describe the equiv alence of the controller to an open qu antum harmonic oscillato r and the possibilit y of its quan- tum optical implementati on [2]. The first of these equations is the condition for preserv atio n of the canoni cal commutati on relatio ns for th e state variables of the quantum harmonic oscillator . The second PR condition, w hich relates the ma- trices b 1 and c b y a li near bijection, describes th e unitary transformati on of the quantum W iener process at the input of t he quantum harmoni c os cillator . The first of the PR cond itions (16), whi ch is a linear equ ation with respect to a , deter - mines a as a quadratic function of b u p t o t he subspace o f Hamiltonian matrices { a ∈ R n × n : aJ 0 + J 0 a T = 0 } = J 0 S n = S n J 0 , with S n the subspace o f real symmetric matrices of order n : a = J 0 R |{z} Hamiltonian matrix + bJ b T J 0 / 2 . | {z } particular solution (18) Here, R ∈ S n specifies the free Hamilt onian operator ξ T t Rξ t / 2 of the quant um harmonic oscillator [1, Eqs. (20)–(22) on pp. 8–9]. Since the m atrix bJ b T is anti- symmetric, bJ b T J 0 is skew-Hamiltonian. Therefore, (18) describes an orthogonal decompositio n of the m atrix a into projections onto the subspaces of Hami ltonian and skew-Hamiltonian matrices in the sense of the Frobenius inner product of real matrices h X , Y i := T r( X T Y ) , with k X k := p h X , X i t he Frobeniu s norm. 6 From the second PR cond ition in (16) and the canonical structure of J 0 and J 2 , it follows that the matrix c is related to b 1 by c = J 2 b T 1 J 0 = J 2 I T b T J 0 , I :=  I 0  , (19) where, i n view of (13), the matrix I “extracts” b 1 from b as b 1 = b I . In combin ation with the decomposition (18), thi s impli es that, f or a ph ysically realizable quantum controller , t he matrix γ in (14) is completely parameterized by the matrices R and b as γ =  J 0 R + bJ b T J 0 / 2 b J 2 I T b T J 0 0  . (20) In vi e w of the physical meaning of R , we wi ll refer to (20) as the Hamiltonian parameterization of the coherent quantum controller , with the S n × R n × ( m 2 + p ) - valued parameter  R b  ; see Fig. 2. The PR conditions (16) are in variant un- γ Γ ✍✌ ✎☞ ❧ R ✲ ✲ ✍✌ ✎☞ a    ✠    ✠ ❄ ❅ ❅ ❅ ❘ ❅ ❅ ❅ ❘ ❅ ❅ ❅ ❘ ❅ ❅ ❅ ❘ ✍✌ ✎☞ ❧ b ✛ ✛ ✍✌ ✎☞ c ✲ ✍✌ ✎☞ A ❄ ✲ ✍✌ ✎☞ C ✍✌ ✎☞ B ✍✌ ✎☞ E Figure 2: This d irected acyclic graph describes the depend ence of the LQG cost E of the clo sed- loop system on the matrices R a nd b . An oriented edge  α →  β signifies “ β depends on α ”. The dashed lines encircle th e matrix triples γ a nd Γ defined by (1 4). The e mergence of R and the depend encies indicated b y dou ble arrows represen t the PR co nditions fo r the quan tum controller, with a , b , c b eing otherwise independ ent. der the group of similarity transformations of the control ler matrices ( a, b , c ) 7→ ( σ aσ − 1 , σb, cσ − 1 ) , where σ is any real symplectic matrix of order n (th at is , σ J 0 σ T = J 0 ). This corresponds to the canonical state transformation ξ t 7→ σ ξ t ; see also [10, Eqs. (12)–(14)]. Any such transformation of a physi cally realiz- able controller leads t o its equivalent state-space representation, with t he matrix R transformed as R 7→ σ − T Rσ − 1 . 7 5. Coher ent quantum LQG contr ol pr oblem The Coherent Quantum LQG (CQLQG) control problem [6] c ons ists in mini- mizing the a verage output “ener gy” of th e closed-loop system (11): E := lim t → + ∞  1 t Z t 0 E ( Z T s Z s )d s  = T r( C P C T ) =T r ( B T Q B ) = − 2 hA , H i − → min . (21) The mi nimum is taken over t he n -dim ensional control lers (7)–(9) which make the matrix A in (12) Hurwitz and s atisfy the PR conditions (16). Here, E X := T r( ρX ) denotes t he quantum expectation over the underlying densi ty operator ρ , and P := lim t → + ∞ Re E ( X t X T t ) is the st eady-state cov ariance m atrix of the st ate vector of the closed-loop s ystem. Also, we use the shorthand notation H := QP , (22) with P and Q s atisfying the algebraic L yapunov equations A P + P A T + B B T = 0 , A T Q + Q A + C T C = 0 , (23) so that these m atrices are the con trollabilit y and obs erv abil ity Grami ans of the state-space realization triple ( A , B , C ) . The spectrum o f the di agonalizable m atrix H in (22) is formed by t he squared Hankel singul ar values of the sys tem, and we wil l refer to H as the Hankelian . Th e fact that E coincides with t he s quared H 2 -norm of a classical strictly proper linear time in variant system enables the CQLQG probl em (21) to be recast as a constrained LQG control problem for an equiv alent classical plant   A B B 2 C 0 0 D 0 C D 0   =     A B 1 B 2 B 2 C 0 0 0 D 0 0 0 I 0 C D 0 0     (24) driv en by a ( m 1 + m 2 ) -dimensional standard W iener pro cess, with the con troller being noiseless. W e wi ll employ t he smooth dependence of the cost E on the matrices R and b which govern the Hamiltoni an parameterization (20) of a p hys- ically realizable stabilizing controller . The conditions of optimality , obtained in Section 8, ut ilize the Frechet differ enti ation of the LQG cost with respect to th e state-space realization matrices [12] assembled into matrices with a specific spar- sity pattern and an auxiliary class of self-adjoint operators i ntroduced in Sections 6 and 7. 8 6. The Γ sparsity structur e The subsequent cons iderations in volve Frechet diffe rentiati on with respect to state-space realization matrices assembled into matrices of the “ Γ -shaped” spar- sity structure (14). W e denote by Γ r,m,p :=  ϕ σ τ 0  : ϕ ∈ R r × r , σ ∈ R r × m , τ ∈ R p × r  (25) the Hi lbert space of real ( r + p ) × ( r + m ) -m atrices who se bottom -right block of size ( p × m ) is zero. Th e space Γ r,m,p , whi ch is a subspace of R ( r + p ) × ( r + m ) , inherits the Frobenius inner p roduct of matrices. Let Π r,m,p denote th e orthogon al projection onto Γ r,m,p whose action on a ( r + p ) × ( r + m ) -matrix consists i n padding its bottom-right ( p × m ) -block ψ with zeros: Π r,m,p  ϕ σ τ ψ  =  ϕ σ τ 0  . (26) The subscripts in Γ r,m,p and Π r,m,p will often be omitted for brevity . The Frechet deriv ativ e ∂ X f of a smooth function Γ ∋  ϕ σ τ 0  =: X 7→ f ( X ) ∈ R be- longs t o t he sam e Hil bert sp ace (25) and inherits the s parsity st ructure: ∂ X f =  ∂ ϕ f ∂ σ f ∂ τ f 0  . 7. Special self-adjoint operators For the purposes of Section 8, we associate a linear operator [ [ [ α, β ] ] ] : R p × q → R s × t with a pair of matrices α ∈ R s × p and β ∈ R q × t , by [ [ [ α, β ] ] ]( X ) := αX β . (27) The map ( α , β ) 7→ [ [ [ α , β ] ] ] from t he direct product o f the m atrix spaces to the space of li near operators on matrices is bilinear . If s = p and t = q , t hen the spectrum of the operator [ [ [ α, β ] ] ] on R p × q consists of t he pairwise prod ucts λ j µ k of the eigen v alues λ 1 , . . . , λ p and µ 1 , . . . , µ q of the matrices α and β , so that their spectral radii are related by r ([ [ [ α, β ] ] ]) = r ( α ) r ( β ) . (28) 9 Furthermore, for any positive in teger r and matrices α 1 , . . . , α r ∈ R s × p and β 1 , . . . , β r ∈ R q × t , we define a linear operator [ [ [ α 1 , β 1 | . . . | α r , β r ] ] ] := r X k =1 [ [ [ α k , β k ] ] ] , (29) where the matrix pairs are s eparated by “ | ”s. Of importance will be self-adjoint linear operators on the Hilbert sp ace R p × q of the form (29) where α 1 , . . . , α r ∈ R p × p and β 1 , . . . , β r ∈ R q × q are su ch that for any k = 1 , . . . , r , th e matrices α k and β k are either both symmetric or both antisymmetric. Such an operator (29) will be referred to as a self-adjo int operator of grade r . The s elf-adjointness is understood in t he s ense o f the Frobenius inner product on R p × q and fol lows from the p roperty that, in each of the cases ( α T , β T ) = ( ± α , ± β ) , the adjoi nt [ [ [ α, β ] ] ] † = [ [ [ α T , β T ] ] ] coi ncides with [ [ [ α, β ] ] ] . In these cases, as for any s elf-adjoint operator , the eigen v alues of [ [ [ α, β ] ] ] are all real. Lemma 1. If α ∈ R p × p and β ∈ R q × q ar e both a ntisymmetric, then the spectru m of [ [ [ α, β ] ] ] is symmetric about the origi n. If α a nd β are bot h s ymmetric a nd positi ve (semi-) definite, then [ [ [ α, β ] ] ] is positi ve (semi-) definite, re sp ectively . Pr oof. If α and β are bot h antisymmet ric, then their eig en va lues λ 1 , . . . , λ p and µ 1 , . . . , µ q are all pure imagi nary and symm etric about the origin [3]. Hence, the eigen values λ j µ k of [ [ [ α, β ] ] ] also form a set which is symm etric about the origin. By a similar reasoning, if α and β are real positive (semi-) d efinite symmetri c matrices, then their eigen va lues are all real and (nonnegative) positive, and hence, so are the eigen values of [ [ [ α, β ] ] ] which implies its pos itive (semi -) definiteness. Alternative ly , t he s econd ass ertion of t he lemma also follows from t he relati on [ [ [ α, β ] ] ] = [ [ [ √ α, √ β ] ] ] 2 which holds for any positive semi-definite symmetric m a- trices α ∈ R p × p and β ∈ R q × q , so that h X , αX β i = k √ αX √ β k 2 > 0 for any X ∈ R p × q . Whilst the operator (27) with nonsingular α and β is straight forwardly in v ert- ible: [ [ [ α , β ] ] ] − 1 = [ [ [ α − 1 , β − 1 ] ] ] , the in verse of M := [ [ [ α 1 , β 1 | . . . | α r , β r ] ] ] from (29) for r > 1 (except for the case P j,k [ [ [ α j , β k ] ] ] = [ [ [ P j α j , P k β k ] ] ] , which re- duces to a grade one operator , or special L yapunov operators [ [ [ α, I ] ] ] + [ [ [ I , α ] ] ] with α = α T which are treated b y diagonalizing the matrix α ), can only be compu ted using the vectorization of matrices [5] as M − 1 ( Y ) = vec − 1 (Ξ − 1 v ec( Y )) , pro- vided t hat the matrix Ξ := P r k =1 β T k ⊗ α k is no nsingular . Here, v ec : R p × q → R pq is a li near bijection which maps a m atrix X to the vector obtained by writi ng 10 the columns X • 1 , . . . , X • q of the matrix one underneath the o ther . In v ertibil ity conditions for grade two operators is discussed in Appendix A. 8. Equations for th e optimal controller Necessary conditions for optimali ty in the class of n -dimensional physically realizable stabili zing controllers are o btained by equat- ing th e Frechet deriv ativ es of th e LQG cost E wit h respect to R and b to zero. In view of Fig. 2, the chain rule allows th e differentiation to be carried out in three steps. First, the matrices A , B , C of t he closed-loop system are considered to be independent variables. Below is an adaptation of [12, Lemma 7 of Appendix B] whose proof is giv en to make the e xposit ion self-contained. Lemma 2. Supp ose the mat rix A in (12) is Hurwitz. Then th e F r echet derivative of the LQG cost E fr om (21) with r espect to the matrix Γ from (14) is ∂ Γ E = 2  H Q B C P 0  . (30) Her e, H i s the Hankelian defined b y (22) in terms of t he Gramians P , Q fr om (23). Pr oof. As dis cussed i n Section 6, the Frechet deriv ativ e ∂ Γ E inherits the block structure of the matrix Γ : ∂ Γ E =  ∂ A E ∂ B E ∂ C E 0  . (31) W e wil l now comput e the blocks of this m atrix. T o calculate ∂ A E , let B and C be fixed. Then the first variation of E with respect to A i s δ E = h C T C , δ P i = −hA T Q + Q A , δ P i = −h Q, A δ P +( δ P ) A T i = h Q, ( δ A ) P + P δ A T i = 2 h H , δ Ai , which implies that ∂ A E = 2 H . (32) T o compute ∂ B E , suppose A and C are fixed. Then the obs erva bil ity Gramian Q , which is a function of A and C , is also constant , and t he first variation of E with respect to B is δ E = h Q, δ ( B B T ) i = h Q, ( δ B ) B T + B δ B T i = 2 h Q B , δ B i , and hence, ∂ B E = 2 Q B . (33) The deriv ativ e ∂ C E is calcul ated by a similar reasoning . Assuming A and B (and so also the control lability Gramian P ) to be fixed, t he first variation of E with 11 respect to C is δ E = h P , δ ( C T C ) i = h P , ( δ C ) T C + C T δ C i = 2 h C P , δ C i , which implies that ∂ C E = 2 C P . (34) Now , substitution of (32)–(34) into (31) yields (30). W e wi ll now take into account th e dependence of the closed-loop sy stem ma- trices A , B , C i n (12) on the controller matrices a , b , c , with the latter still con- sidered to be independent variables. In what follows, the Gramians P and Q in (23) and the Hankelian H , defined by (22), inherit t he four-block structure o f t he matrix A from (12). Their blocks ha ve s ize ( n × n ) and are numbered as f ol lows: H := ← n → ← n →  H 11 H 12 H 21 H 22  l n l n = ← n → ← n →  H • 1 H • 2  l 2 n = ← 2 n →  H 1 • H 2 •  l n l n . (35) The block ( · ) 11 is related to the state variables of the pl ant, while ( · ) 22 pertains to those of the controller . The blocks of the matrix H in (35) are expressed in terms of the block rows of Q and bl ock columns of P as H j k = Q j • P • k . Lemma 3. Supp ose the mat rix A in (12) is Hurwitz. Then th e F r echet derivative ∂ γ E =  ∂ a E ∂ b E ∂ c E 0  of E fr om (21) with r espect to the matrix γ fr om (14) is ∂ γ E = 2  H 22 H 21 C T + Q 2 • B D T B T 2 H 12 + D T 0 C P • 2 0  , (36) wher e the matrices Γ 1 , Γ 2 ar e defined by (15); H , P , Q a r e given by (22)–(23), and the notation (35) is used. Pr oof. Since E is a compos ite function of a , b , c which enter (21) through t he closed-loop system matrices A , B , C , t he chain rule gives ∂ γ E = ( ∂ γ Γ) † ( ∂ Γ E ) = Π (Γ T 1 ∂ Γ E Γ T 2 ) . (37) Here, ( · ) † is the adjoint in the sense of the Frobenius inn er product of matri- ces, and Π is the ortho gonal projection on to t he sub space Γ defined by (25)– (26). Indeed, the first variation of the affine map γ 7→ Γ , defined by (14)– (15), is given by δ Γ = Γ 1 ( δ γ )Γ 2 , which i mplies that ∂ γ Γ = [ [ [Γ 1 , Γ 2 ] ] ] . Hence, δ E = h ∂ Γ E , δ Γ i = h ∂ Γ E , Γ 1 δ γ Γ 2 i = h Γ T 1 ∂ Γ E Γ T 2 , δ γ i = h Π (Γ T 1 ∂ Γ E Γ T 2 ) , δ γ i , 12 which establishes ( 37). Substitution of the m atrices Γ 1 and Γ 2 from (15) and ∂ Γ E from (30) into the right-hand side of (37) yields ∂ γ E = 2 Π    0 I n 0 B T 2 0 D T 0   H Q B C P 0    0 C T I n 0 0 D T     = 2  H 22 H 21 C T + Q 2 • B D T B T 2 H 12 + D T 0 C P • 2 0  , where Lemma 2 and the notation (35) are also used, which proves (36). Finally , we will ut ilize the Hamil tonian parameterization (20), which makes E a function of the matrices R and b ; see Fig. 2. Theor em 1. A physi cally r ealizable stabilizing cont r oller , wit h Hamiltonian pa- rameterization (20), is a critical point of the LQG cost E from (21) if and only if ther e ex ist s a r eal antisymmetric matrix Φ of o r der n such that H 22 = − Φ J 0 , (38) M ( b ) + H 21 C T + Q 21 B D T + J 0 ( H T 12 B 2 + P 21 C T 0 D 0 ) J 2 I T = 0 . (39) Her e, M := [ [ [Φ , J | Q 22 , DD T | J 0 P 22 J 0 , I J 2 D T 0 D 0 J 2 I T ] ] ] (40) is a self-adjoint operator of gr ade thr ee in the sense of (29). Pr oof. In vie w of (20 ), the symmetric matrix R enters the controller only through a . Hence, ∂ R E = ( − J 0 ∂ a E + ( − J 0 ∂ a E ) T ) / 2 = H T 22 J 0 − J 0 H 22 , (41) where the relation ∂ a E = 2 H 22 from Lemma 3 is used. Unlike R , the matri x b both enters a and completely parameterizes c , and hence, d E / d b =(( ∂ a E ) J 0 + J 0 ( ∂ a E ) T ) bJ / 2 + ∂ b E + J 0 ( ∂ c E ) T J 2 I T =( H 22 J 0 + J 0 H T 22 ) bJ + 2( H 21 C T + Q 2 • B D T ) + 2 J 0 ( B T 2 H 12 + D T 0 C P • 2 ) T J 2 I T , (42) 13 where (36) of Lemm a 3 is used again. By introducing a real antisym metric matrix Φ := ( H 22 J 0 + J 0 H T 22 ) / 2 , (43) and recalling (12), (13) and (35), it follows from (42) that (d E / d b ) / 2 =Φ bJ + H 21 C T + Q 21 B D T + Q 22 b DD T + J 0 ( H T 12 B 2 + P 21 C T 0 D 0 ) J 2 I T + J 0 P 22 J 0 b I J 2 D T 0 D 0 J 2 I T = H 21 C T + Q 21 B D T + J 0 ( H T 12 B 2 + P 21 C T 0 D 0 ) J 2 I T + M ( b ) , where (19) and (40) are also used. Therefore, d E / d b = 0 is equivalent to (39). The definition (43), which is considered as an equation with respect to H 22 , de- termines uniquely the skew-Hamiltonian part − Φ J 0 of H 22 , so that H 22 can be represented as H 22 = (Ψ − Φ) J 0 , (44) where Ψ := ( J 0 H T 22 − H 22 J 0 ) / 2 (45) is a real symmetric matrix of order n . Direct comparison of (45) with (41) yields ∂ R E = − 2 J 0 Ψ J 0 . (46) Hence, ∂ R E = 0 hol ds if and only if Ψ = 0 , in w hich case, (44) takes the form of (38). Therefore, t he property th at the controller is a critical point of E (that is, ∂ R E = 0 and d E / d b = 0 ) i s indeed equiv alent to the fulfillment of (38) and (39) for a real antisymmetric matrix Φ of order n . For a g iv en m atrix b in the Hamilto nian parameterization (20) o f the controller , (45) defines a map R ( b ) ∋ R 7→ Ψ ∈ S n on the set R ( b ) := { R ∈ S n : A is Hurwitz } . (47) In v iew of (46), the Frechet deriv ative of thi s map wit h respect t o R is expressed in term s of the sec ond order Frechet deri vativ e o f the LQG cos t of the c lo sed-loop system as ∂ R Ψ = − 1 2 [ [ [ J 0 , J 0 ] ] ] ∂ 2 R E , (48) where we have al so used the property that [ [ [ J 0 , J 0 ] ] ] i s in v olutory since [ [ [ J 0 , J 0 ] ] ] 2 = [ [ [ J 2 0 , J 2 0 ] ] ] = [ [ [ − I , − I ] ] ] is t he identity operator . 14 9. A Quasi-separation principle The operator M , which is defined by (40) and acts on the control ler gain matrix b from (13 ), can be partitioned as M ( b ) =  M 1 ( b 1 ) M 2 ( b 2 )  (49) into two operators acting separately on the submatrices b 1 and b 2 . Here, M 1 :=[ [ [Φ , J 2 | Q 22 , I | J 0 P 22 J 0 , J 2 D T 0 D 0 J 2 ] ] ] , (50) M 2 :=[ [ [Φ , D J 1 D T | Q 22 , D D T ] ] ] (51) are self-adjoint operators of grades three and t wo. This allows the equation (39) for d E / d b = 0 to be sp lit into M 1 ( b 1 ) + Q 21 B 2 + J 0 ( H T 12 B 2 + P 21 C T 0 D 0 ) J 2 = 0 , (52) M 2 ( b 2 ) + H 21 C T + Q 21 B 1 D T = 0 , (53) which are equi valent to d E / d b 1 = 0 and d E / d b 2 = 0 . Note that (52) corresponds to the equation for the state-feedback matrix b c = − ( D T 0 D 0 ) − 1 ( B T 2 b Q 1 + D T 0 C 0 ) (54) of the st andard LQG cont roller for th e subsidi ary class ical plant (24), whil e (53) corresponds to the equation for th e Kalman filter observ atio n g ain matrix of the controller b b 2 = ( b P 1 C T + B 1 D T )( D D T ) − 1 . (55) Here, it is assumed that the matrix D 0 is of full column rank, and D i s of full ro w rank. The m atrices b c and b b 2 from (54) and (55) det ermine the dynamics matrix o f the standard LQG controller as b a := A − b b 2 C + B 2 b c and are expressed in terms of the stabilizing s olutions b Q 1 , b P 1 of t he independent con trol and filtering algebraic Riccati equations (AREs): A T b Q 1 + b Q 1 A + C T 0 C 0 = ( b Q 1 B 2 + C T 0 D 0 )( D T 0 D 0 ) − 1 ( b Q 1 B 2 + C T 0 D 0 ) T , A b P 1 + b P 1 A T + B 1 B T 1 = ( b P 1 C T + B 1 D T )( D D T ) − 1 ( b P 1 C T + B 1 D T ) T . 15 The fact, that (52) and (53) are independent linear equations wi th respect to b 1 and b 2 , as well as th e o riginal p artition (49), can be interpreted as an analogue of the classical LQG control/filtering separation principle for the CQLQG probl em. In t urn, each of the operators M k from (50) and (51) can be spl it in to th e s um of self-adjoint operators M ⋄ k and M + k of grades one and less one: M 1 := M ⋄ 1 z }| { [ [ [Φ , J 2 ] ] ] + M + 1 z }| { [ [ [ Q 22 , I | J 0 P 22 J 0 , J 2 D T 0 D 0 J 2 ] ] ] , (56) M 2 := [ [ [Φ , D J 1 D T ] ] ] | {z } M ⋄ 2 + [ [ [ Q 22 , D D T ] ] ] | {z } M + 2 . (57) By applying Lemma 1, it follo ws that the spectrum of M ⋄ k is symmet ric a bou t the origin, whi le M + k < 0 . M oreover , if Q 22 ≻ 0 , or P 22 ≻ 0 and D 0 in (10) is of full column rank, then M + 1 ≻ 0 . Indeed, th e fulfillment of at least one o f these conditions implies p ositive definiteness of at least one of the posi tiv e semi-definite operators on the right-hand side of the representation M + 1 = [ [ [ Q 22 , I ] ] ] + [ [ [ J 0 P 22 J T 0 , J 2 D T 0 D 0 J T 2 ] ] ] (58) which follows from J 0 and J 2 being antis ymmetric matrices. Similarly , the con- ditions that Q 22 ≻ 0 and D is o f full row rank ensure t hat M + 2 ≻ 0 . In particular , by adapting [12, Lemma 5 of Section VIII], it foll ows that if, in addition to the rank conditi ons on D 0 and D , the controller state-space realization is minimal, then Q 22 ≻ 0 and P 22 ≻ 0 and hence, M + 1 ≻ 0 and M + 2 ≻ 0 . T herefore, in the cases discussed above, the inv ertibil ity of the operators M 1 and M 2 in (56)–(57) can only be destroyed by the presence of the indefinite operators M ⋄ 1 and M ⋄ 2 if the matrix Φ is lar ge enough compared to Q 22 . This can be formulated in terms of the matrix ∆ := Q − 1 22 Φ (59) whose spectrum is pure imaginary and symmetric about zero. Lemma 4. Suppose the matrix D in (2) is of f ull r ow rank and Q 22 ≻ 0 . Also, suppose the spectral radius of the matrix ∆ fr om (59) s atisfies r (∆) < 1 . Then the operators M 1 and M 2 in (50) and (51) ar e positi ve definite. Pr oof. Since [ [ [ J 0 P 22 J 0 , J 2 D T 0 D 0 J 2 ] ] ] < 0 , and [ [ [ Q 22 , I ] ] ] ≻ 0 (in vie w of the as- sumption Q 22 ≻ 0 ), then (56) and (58) imply that M 1 < M ⋄ 1 + [ [ [ Q 22 , I ] ] ] < (1 − r (∆))[ [ [ Q 22 , I ] ] ] . (60) 16 Here, we us e the relation r ([ [ [ Q 22 , I ] ] ] − 1 M ⋄ 1 ) = r (∆) r ( J 2 ) = r (∆) which follows from (28) and the property that the eigen values of the canonical ant isymmetri c matrix J 2 are ± i . Therefore, i f r (∆) < 1 , th en (60) im plies t hat M 1 ≻ 0 . By a si milar reasoning, un der th e addition al assump tion that D is of full row rank (that is, DD T ≻ 0 ), it follo ws from (57 ) and (59) that M 2 < (1 − r (∆)) M + 2 ≻ 0 . Indeed, r (( M + 2 ) − 1 M ⋄ 2 ) = r (∆) r ( D J 1 D T ( D D T ) − 1 ) 6 r (∆) since − I 4 iJ 1 4 I and the Herm itian matrix ( D D T ) − 1 / 2 D ( iJ 1 ) D T ( D D T ) − 1 / 2 has all its spectrum in [ − 1 , 1] , so that r ( D J 1 D T ( D D T ) − 1 ) 6 1 . Assuming in vertibility of the operators M 1 and M 2 (for example, th e fulfill- ment o f conditi ons of Lemma 4 that ensure a stronger property – positive definite- ness of these operators), t he equations (52) and (53 ) c an be w ritten more explicitly for b 1 and b 2 : b 1 = − M − 1 1 ( Q 21 B 2 + J 0 ( H T 12 B 2 + P 21 C T 0 D 0 ) J 2 ) , (61) b 2 = − M − 1 2 ( H 21 C T + Q 21 B 1 D T ) . (62) These two equations are, in principle, amenable to further reduction (to be dis- cussed elsewhere ) and will be utilized as assignm ent operators in the iterative procedure of Section 11 for finding the optimal controller . 10. Second order condition for o ptimality A second order necessary condition for optimality of the cont roller with re- spect to the matrix R of the Hamilt onian parameterization (20) is the positive semi-definiteness ∂ 2 R E < 0 of t he approp riate second Frechet deriva tive of the LQG cost (21). Moreover , the posi tiv e definiteness ∂ 2 R E ≻ 0 is sufficient for the local strict optimali ty . T o compute the self-adjoint operator ∂ 2 R E , which acts on the subspace S n of re al symmet ric matrices of order n , we define a li near op erator J : S n → R 2 n × 2 n as an appropriate restriction of t he grade one linear o perator relating A with R : J := [ [ [  0 n J 0  ,  0 n I n  ] ] ]     S n . (63) Its adjoint is J † = −S [ [ [  0 n J 0  ,  0 n I n  ] ] ] , since J 0 is antisym metric, with S : R n × n → S n the symmetrizer defined by (B.2). 17 Lemma 5. S uppose th e matrix A in (12) is Hurwitz. Then the second F r echet derivative of E fr om (21) with r espect to the matrix R fr om (20) is ∂ 2 R E = 4 J † ( QL A S P + P L A T S Q ) J . (64) Her e, L A and S ar e the in verse Lyapunov o perator and symmetriz er fr om (B.1), (B.2), a nd Q := [ [ [ Q, I ] ] ] and P := [ [ [ I , P ] ] ] ar e grade on e self-adjoint operators (see Section 7) of th e left a nd right mu ltiplicati on by the ob servability and con- tr ollabil ity Gramians Q and P of the closed-loop system fr om (23). Pr oof. The matrix R o nly enters the cost E through the matri x A of the clo sed- loop sy stem, and A depends af finely on R , with ∂ R A = J the con stant oper- ator from (63). Hence, (64 ) follows from ∂ 2 R E = J † ∂ 2 A E J and Lemma 9 of Appendix C. From (64), it fol lows that the “matrix” representation of the self-adjoint oper- ator ∂ 2 R E on the space S n is described by v ec h( ∂ 2 R E ( M )) = 4Υ T (Ω + Ω T )Υv ec h ( M ) , where v ec h( M ) denotes the half-vectorization o f a matrix M ∈ S n , that is, t he column-wise vectorization of its triangular part below (and includin g) th e main diagonal. Here, the square matrix Ω := − ( I 2 n ⊗ Q )( I 2 n ⊗ A + A ⊗ I 2 n ) − 1 Σ( P ⊗ I 2 n ) of order 4 n 2 represents the operator QL A S P o n R 2 n × 2 n , with Σ corresponding to the symmetrizer S : R 2 n × 2 n → S 2 n . Also, Υ :=  0 n I n  ⊗  0 n J 0  Λ is a (4 n 2 × n ( n + 1) / 2) -matrix which represents the operator J , defined by (63), with Λ ∈ R n 2 × n ( n +1) / 2 the “du plication” matrix [5, 11] w hich e xp resses the full vectorization of a matrix M ∈ S n in terms of its half-vectorization by ve c ( M ) = Λv ec h( M ) . 11. A Newton-like scheme The equations (61)–(62) can be c om bined wi th iterations for solving the equa- tion Ψ = 0 for the mat rix Ψ from (45), which i s equiv alent to the s tationarity of 18 the LQG cos t E with respect to the matrix R of t he Hamiltoni an parameterization. The latter part of the scheme, aimed at finding a roo t R ∈ R ( b ) of the equation Ψ = 0 from the set (47), can be organized in the form of Newton-Raphson itera- tions R 7→ R − ( ∂ R Ψ) − 1 (Ψ) = R − ( ∂ 2 R E ) − 1 ( ∂ R E ) . (65) Here, th e symm etric matrices ∂ R E and Ψ are related by (46), and, in vie w of (48), the in verse of t he operator ∂ R Ψ is giv en by ( ∂ R Ψ) − 1 = − 2( ∂ 2 R E ) − 1 [ [ [ J 0 , J 0 ] ] ] , (66) where we ha ve again used the in volutional property of the operator [ [ [ J 0 , J 0 ] ] ] , and the second order Frechet deriv ativ e ∂ 2 R E is provided by Lemma 5 . If the lo cal strict optimality condition ∂ 2 R E ≻ 0 is satisfied, this ensures well-posedness of the in verse in (66). Thus the equatio ns (61)–(62), considered as assign ment op- erators for b 1 and b 2 , and (65) for R , con stitute a Newton-like it erativ e scheme for num erical computati on of the st ate-space realization matrices of the optimal CQLQG controll er . These three assign ment operators are alternated wit h updating the Gramians of the closed-loop sy stem via the appropriate L yapunov equations in (23). The o rder o f this alternation will influence the overall con ver gence rate of the scheme and is an important computational issue to be explored. Another iss ue to be taken into account is that the asymptoti c stabilit y of the closed-loop sys- tem matrix A can be viol ated by t he u pdate of the matrices b 1 , b 2 , R after whi ch the next iteration becomes impo ssible. Therefore, being a local optim ization algo- rithm, the p roposed s cheme requires a “st ability recovery” block. A salient feature of such an algorithm (which is currently under development) is t hat it in v ol ves the in version of special sel f-adjoint operators on matrices which, in general, can only be carried out via the vectorization of matrices mentioned in Sec ti ons 7 and 10. 12. Conclusion W e hav e o btained equations for t he optim al controller in the Coherent Quan- tum LQG problem by direct Frechet d iffe rentiati on of the L QG cost with respect to the pair of m atrices which gov ern the Hami ltonian parameterization of physically realizable quantu m controllers. W e h a ve in vestigated spectral properties of spe- cial self-adjoint operators whose i n verse plays an imp ortant role in t he equations and can onl y be carried out b y using matrix vectorization. W e hav e establis hed a partial decoupli ng of these equations wi th respect to the gain matrices of the op- timal controller , which can be interpreted as a quantum analogue of the s tandard 19 LQG control/filtering separation principle. Usi ng this quasi-separation property , we hav e outli ned a Newton-like iterative scheme for numerical computation of th e quantum controller . The scheme in volves a yet-to-be-explored freedom of choos- ing the order i n wh ich t o perform i terations with respect t o t he Hamiltonian and gain matrices of the cont roller to optimi ze the con ver gence rate. The existence and uniq ueness of sol utions to the equation s for the state-space realization matri- ces of the opti mal CQLQG controller als o remains an open problem and so d oes their further reducibil ity . Thi s circle of questi ons is a subj ect of ongoi ng research and will be tackled in subsequent publications. Acknowledgeme nt The work is supported by the Australian Researc h Council. Refer ences [1] S.C.Edwards, and V .P .Bela vkin , Opti mal quantum filt ering and quantum feedback control, arXi v:qu ant-ph/0506018v2, August 1, 2005. [2] C.W .Gardiner , and P .Zoller , Quan tum Noise . Springer , Ber li n, 2004. [3] R.A.Ho rn, and C.R.Johnso n, Matri x An alysis , Cambri dge University Press, Ne w Y ork, 2007. [4] M .R.James, H.I.Nurdin, and I.R.Petersen, H ∞ control of linear q uantum stochastic systems, IEEE T ransacti ons on A utomatic Contr ol , v ol. 53, no. 8, 2008, pp. 1787–1803. [5] J .R.Magnus, Linear Structur es , Oxford Uni versity Press, Ne w Y ork, 1 988. [6] H .I.Nurdin, M.R.James, and I.R.Petersen, Coherent quantum LQG control, Automatica , vol. 45, 2009, pp. 1837–1846. [7] K .R.Parthasara th y , An Intr oducti on to Quant um Stochastic Calculus . Birkh ¨ auser , Basel, 1992. [8] I.R.Petersen, Quantum linear s ystems theory , Proc. 19th Int. Symp. Math. Theor . Netw orks Syst., Budapest, Hungary , July 5–9, 2 010, pp. 2173–2184. 20 [9] A .J.Shaiju, and I.R.Petersen, On the physi cal realizability of general linear quantum stochastic di f ferential equations with complex coefficients, Proc. Joint 48 th IEEE Conf. Decision Control & 28t h Chinese Control Conf., Shanghai, P .R. China, December 16–18, 2009, pp. 1422–142 7. [10] R.Simon, Peres-Horodecki separabili ty criterion for cont inuous variable sys- tems, Phys. Rev . Lett. , vol. 84, no. 12, 2000, pp. 2726–2729. [11] R.E.Skelton, T .Iwasaki, and K.M.Grig oriadis, A Unified Al gebraic Appr oach to Linear Contr ol Design , T aylor & Francis, London, 1998. [12] I.G.Vladimirov , and I.R.Petersen, Hardy-Schatten norms of systems , output ener gy cum ulants and l inear quadro-quartic Gaussian cont rol, Proc. 19t h Int. Symp. M ath. Th eor . Networks Sys t., Budapest, Hungary , J uly 5 –9, 2010, pp . 2383–2390. [13] I.G.Vladimirov , and I.R.Petersen, A qu asi-separation principle and Newton- like scheme for c oherent quantum LQG cont rol. 18th IF AC W orld Congr ess , Milan, Italy , 28 Augus t–2 September , 2011, accepted for publi cation. Ap pendix A. In vertibility of grade two operators Lemma 6. Let r = 2 in (29), and let both matrices α 1 and β 1 be nonsingul ar . Then the operator M := [ [ [ α 1 , β 1 | α 2 , β 2 ] ] ] is in vertibl e if an d only if the eigen values λ 1 , . . . , λ p of α − 1 1 α 2 and the eigen values µ 1 , . . . , µ q of β 2 β − 1 1 satisfy λ j µ k 6 = − 1 for all j = 1 , . . . , p, k = 1 , . . . , q . (A.1) Pr oof. If r = 2 , the operator (29) can be represented as M := [ [ [ α 1 , β 1 | α 2 , β 2 ] ] ] = M 1 M 2 , where M 1 := [ [ [ α 1 , β 1 ] ] ] and M 2 := [ [ [ I , I | α − 1 1 α 2 , β 2 β − 1 1 ] ] ] . The operator M 1 is in vertible in view of the nonsi ngularity of th e matrices α 1 and β 1 , with M − 1 1 = [ [ [ α − 1 1 , β − 1 1 ] ] ] . Hence, the in vertibil ity of M is equivalent to that of M 2 . In turn, the o perator M 2 is in vertible if and only if its spectrum { 1 + λ j µ k : 1 6 j 6 p, 1 6 k 6 q } does no t contain 0 , which is equiv alent t o (A.1). By Lemma 6, the nonsing ularity of t he matrix P 2 k =1 β T k ⊗ α k of order pq reduces to a joi nt prop erty of indi vidual spectra of two matrices of orders p and q . This reduction does not hold for r > 2 . 21 Ap pendix B. P erturbation of in verse L yapunov operators W e associate an inve rse Lyapunov operator L A with a Hurwit z matrix A ∈ R n × n , so t hat L A maps a matrix M ∈ R n × n to the unique solution N of the algebraic L yapunov equation AN + N A T + M = 0 : L A ( M ) := Z + ∞ 0 e At M e A T t d t. (B.1) Its adj oint is L † A = L A T . Since L A commutes with the transpo se, that is, L A ( M T ) = ( L A ( M )) T , then it also commutes with a symmetrizer S defined by S ( M ) := ( M + M T ) / 2 . (B.2) The operator S : R n × n → S n is the orthogonal projection onto the subspace of real symmetric matrices of order n . Lemma 7. The F r echet derivatives o f th e contr ollabil ity and o bservability Grami- ans P and Q of an as ymptotically s table system ( A, B , C ) with r espect to the matrix Γ :=  A B C 0  ar e e xpr essed i n terms of (B.1) and (B.2) as ∂ Γ P = 2 L A S [ [ [  I 0  ,  P B T  ] ] ] , (B.3) ∂ Γ Q = 2 L A T S [ [ [  Q C T  ,  I 0  ] ] ] . (B.4) Pr oof. The Frechet dif ferentiability of P and Q is ensured by the assum ption th at A is Hurwitz. The first variation o f the algebraic L yapun ov equation AP + P A T + B B T = 0 yields 0 = ( δ A ) P + Aδ P + ( δ P ) A T + P δ A T + ( δ B ) B T + B δ B T = Aδ P + ( δ P ) A T + 2 S   δ A δ B   P B T  . This is an algebraic L yapunov equation with respect t o δ P with t he same matrix A , which proves (B.3) in view of the identity  A B  =  I 0  Γ . The relation (B.4) is obtained b y a sim ilar reasoning from th e first variation of the L yapunov equation for t he observability Gramian Q , or by using the du ality between P and Q . 22 Ap pendix C. Secon d Fre chet deriv ative of the LQG cost Lemma 8. The second F r echet derivative of the squared H 2 -norm E := k ( A, B , C ) k 2 2 of an asymptotically stable s ystem with r espect to the matrix Γ :=  A B C 0  is computed as ∂ 2 Γ E =4[ [ [  I 0  ,  P B  ] ] ] L A T S [ [ [  Q C T  ,  I 0  ] ] ] + 4[ [ [  Q C  ,  I 0  ] ] ] L A S [ [ [  I 0  ,  P B T  ] ] ] + 2[ [ [  Q 0 0 I  ,  0 0 0 I  |  0 0 0 I  ,  P 0 0 I  ] ] ] . (C.1) Her e, L A and S ar e the in verse Lyapunov o perator and symmetriz er fr om (B.1), (B.2), and P , Q ar e the cont r ollabili ty and observabil ity Gramians of the system. Pr oof. Lemma 2 impli es that the first variation of the Frechet deriv ative ∂ Γ E is computed as δ ∂ Γ E / 2 = δ  QP QB C P 0  =  I 0  δ Q  P B  +  Q C  δ P  I 0  +  0 Qδ B ( δ C ) P 0  . Hence, (C.1) is obtained b y us ing the Frechet deriv atives of the Gram ians from Lemma 7 of Appendix B and the identity  0 Qδ B ( δ C ) P 0  =  Q 0 0 I  δ Γ  0 0 0 I  +  0 0 0 I  δ Γ  P 0 0 I  . Lemma 9. The second F r echet derivative of the squared H 2 -norm E := k ( A, B , C ) k 2 2 of an asymptoticall y s table system with r esp ect to A is ∂ 2 A E = 4 R , R := QL A S P + P L A T S Q . (C.2) Her e, Q := [ [ [ Q, I ] ] ] and P := [ [ [ I , P ] ] ] are grade one self-adjo int operators (see Section 7) of the left and right multiplicat ion by the observabilit y and contr olla- bility Gramians o f the system. 23 Pr oof. In view of Lemma 7, the first variation of ∂ A E = 2 QP with respect to A is δ ∂ A E = 2( Qδ P + ( δ Q ) P ) = 4( Q L A S (( δ A ) P ) + L A T S ( Q ( δ A )) P ) which establishes (C.2). Al ternativ ely , (C.2) can be obtain ed from (C.1) of Lemma 8. Note that at least some eigen values of the sel f-adjoint operator R in (C.2) are positive, si nce R ( A ) = − QP is the negative of t he Hankelian, and h A, R ( A ) i = −h A, QP i = k ( A, B , C ) k 2 2 / 2 > 0 . 24