Robust State Space Filtering under Incremental Model Perturbations Subject to a Relative Entropy Tolerance

1 Rob ust State Space Filtering under Incremental Model Perturbations sub ject to a Relat i v e Entropy T olerance Bernard C. Le vy and Ramine Nikoukhah Abstract This paper considers robust filtering for a n ominal Gaussian state-space m odel, wh en a relative entropy tolerance is applied to each time incremen t of a dy namical mod el. The pro blem is for mulated as a dynamic minimax game wher e the m aximizer adop ts a myopic strategy . This g ame is sh own to admit a saddle po int who se structu re is char acterized by apply ing and extending results presented earlier in [1] for static least-squares estimation . The r esulting minimax filter takes the fo rm of a risk-sensitive filter with a time v arying risk s ensitivity par ameter, wh ich depe nds on the tolerance bou nd ap plied to the model d ynamics and observations at the c orrespon ding time in dex. Th e least-fav orable model is con structed an d used to evaluate the perf ormance of alternative filters. Simulatio ns com paring the propo sed risk-sensitive filter to a standard Kalman filter show a significant perform ance advantage wh en applied to the least-favorable model, and only a small perfo rmance loss for the no minal mod el. Index T erms commitmen t, dynamic minimax g ame, least-fa vorable model, myo pic strategy , relati ve entropy , risk- sensiti ve filtering, ro bust filtering. I . I N T R O D U C T I O N Soon after the introduction of W iener and Kalman filters, it was recognized that these filters were vulnerable t o modelling errors, in the form of either parasitic si gnals or perturbations of th e B. Le vy is with the Department of Electrical and Computer Engineering, 1 Shields A venu e, U ni versity of California, Davis, CA 9561 6. E mail: bclevy@uc davis.edu, phone: (530) 752-8025, fax: (530) 752-8428. R. Nikouk hah is wit h the Institut de Rec herche en Informatique et Automatique (INRIA), Domaine de V oluceau, Rocquencou rt, 78153 Le Chesnay Cedex France. email: ramine.niko ukhah@inria.fr . February 2, 2018 DRAFT 2 system dy namics. V arious approaches were proposed over the last 35 years to construct filt ers with a guaranteed lev el o f immunity to modelling uncertainti es. Drawing from the frame work dev eloped by Hub er for robust statistics [2], Kassam, Poor and their collaborators propo sed an approach [3]–[5] where the optimum filter is selected by solvin g a minimax problem. In thi s approach, the set of possible system mo dels is d escribed by a nei ghborhood centered about the nominal model, and t wo players affront each ot her . One player (say , nature) selects the least- fa vora ble m odel in the allowa ble neighborhood and the other player d esigns the opt imum filter for the least-fa vorable model. While minim ax filtering is conceptually simple, its implementa- tion can be very difficult, since it depend s on the specification of the allow able neighborhood and of the loss function to be min imized. After s ome early success in the design of W i ener filters for nei ghborhoods s pecified by ǫ -contamination mod els or power spectral bands, progress stalled gradually and researchers started looking in different directions to develop robust filters. The 1 980s saw the dev elopment of an enti rely different class of robust filters based on t he minimizati on of ri sk-sensitive and H ∞ performance crit eria [6]–[10]. This approach seeks to a void large errors, ev en if these errors are u nlikely based on the nominal mod el. For example, risk-sensitive filters replace t he stand ard quadratic loss function of least-squares filtering by an exponential qu adratic function, which of course penalizes s e verely large errors. Howe ver , errors in the model dynamics are not introdu ced explicitly in H ∞ and ris k-sensitive filterin g, and the growing aware ness of the i mportance of such errors prom pted a number of researchers in t he early 2000s [11 ]–[13] to revi ve the minimax filtering viewpoint, but in a context where m odelling errors are d escribed i n terms of norms for state-space dynamics pertu rbations. Th e present paper , which is a contin uation o f [1], can be viewed as part of a larger ef fort initiated by Hans en and Sar gent [14]–[16] and other researchers [17], [18] which i s aimed at reinterpreting risk-sensitive filtering from a mi nimax viewpoint. In this context, modell ing uncertainties are described by specifying a tol erance for the relativ e entropy between t he actual system and the nominal mo del. T o a fixed t olerance level describing the model ler’ s confidence in the no minal m odel corresponds a ball of possible models for which it is then pos sible to apply t he minim ax filtering approach proposed by Kassam and Poor . The m inimax formulation of robust filtering based on a relativ e entropy const raint has sev eral attractiv e features. First, relative ent ropy i s a natural measure of model mismatch which is commonly used by st atisticans for fitting s tatistical models by u sing techniq ues such as the February 2, 2018 DRAFT 3 expectation maxim ization iteratio n [19]. More fundament ally , it was shown by Chentsov [20] and by Amari [21] that manifolds of stati stical models can be endowed wit h a non-Riemannian diffe rential geometric structure in volving two dual connections ass ociated to the relative ent ropy and the rev erse relative entropy . In additi on to this strong theoretical just ification, it turns out th at minimax W iener and Kalman filtering problems wit h a relative entrop y constraint admit solu tions [1], [14], [15], [18] i n the form of risk-sensi tiv e filt ers, thu s providing a new int erpretation for such filters. The main di f ference between earlier works and the present paper i s that, ins tead of pl acing a sing le relative entropy constraint on the entire s ystem mo del, we apply a separate constraint to each time increment of t he model . This approach, which is clos er to t he one advoca ted in [12], [13] is based on the following consideration. When a si ngle relative entropy constraint is placed on the compl ete sys tem m odel, the maximizing p layer has the opportun ity to allocate almost all of its mismatch budget to a sin gle element of the mo del mos t sus ceptible to uncertainties. But this strategy may lead to overly pessim istic conclus ions, since in practice modellers allocate the same level of ef fort to modelling each time component of the sy stem. Thus i t probabl y makes bett er sens e t o specify a fixed uncertainty tolerance for each mo del increment, instead of a single bound for the overall model. The analysis presented reli es in part on applying and extending st atic least-squares estimation results derived i n [1] for nominal Gaussian models. These results are revie wed in Section II . In Section III, the rob ust state-space filtering problem with a n incremental relati ve entropy constraint is form ulated as a d ynamic gam e where the maximizer adopts a myopic strategy whose goal is to maximize the mean-square estimati on error at the current ti me only . The existence of a s addle point is establish ed in Section IV, wh ere t he least-fa vorable mod el specifying the s addle poi nt is characterized by extending a Lem ma of [1] t o t he dynamic case. A careful examination of the least-fa vorable m odel structure allows th e transformation of the dynamic est imation game i nto an equiv alent stati c on e, to which the result of [1] become applicable. The robust filter that we obtain is a risk -sensitive filter , but with a time-varying risk-sensitivity parameter , representing the in verse of the Lagrange mul tiplier associated to t he model component constraint for the corresponding ti me period . The l east-fa v orable state-space model for which the robust filter is optimal is constructed in Section V. Thi s model extends to the fini te-horizon tim e-v arying case a model derived asym ptotically in [14] for the case of constant syst ems. The l east-fa vorable s tate- space m odel all ows performance ev aluation studies comparing the performance of the mi nimax February 2, 2018 DRAFT 4 filter with that of other filt ers, such as the standard Kalman filter . Simulations are presented in Section VI which illustrate the dependence of the filt er performance on the relative entropy tolerance appl ied to each model ti me component . The robust filter is compared t o the ordi nary Kalman filter by e xamining their respective performances for both the nominal and least-fa vorable systems. Finally , some conclusi ons are presented in Section VII. I I . R O B U S T S T A T I C E S T I M A T I O N W e start by revie wing a robust s tatic estimati on resul t deriv ed in [1], si nce its extension to the dynami c case is the basis for the robust filtering scheme we propose. Let z =   x y   (2.1) be a random vector of R n + p , where x ∈ R n is a vector to be estimated, and y ∈ R p is an observed vector . The nominal and actual probabi lity densities of z are deno ted respective ly as f ( z ) and ˜ f ( z ) . The deviation of ˜ f from f is m easured by the relative entropy (Kullback-Leibler div ergence) D ( ˜ f , f ) = Z R n + p ˜ f ( z ) ln  ˜ f ( z ) f ( z )  dz . (2.2) The relativ e entropy is not a distance, since it is not sym metric and does not s atisfy t he triangle inequality . Howev er it has the property that D ( ˜ f , f ) ≥ 0 wit h equalit y if and only if ˜ f = f . Furthermore, since the function θ ( ℓ ) = ℓ ln( ℓ ) is con ve x for 0 ≤ ℓ < ∞ , D ( ˜ f , f ) is a con vex function of ˜ f . For a fixed t olerance c > 0 , if F denotes the class of prob ability densities over R n + p , this ensu res that the ”ball” B = { ˜ f ∈ F : D ( ˜ f , f ) ≤ c } (2.3) of densities ˜ f located wi thin a diver gence tolerance c of the nomi nal densit y f is a closed con ve x set. B represents th e set of al l possible true densi ties of random vector z consist ent wit h the allowed mism odelling t olerance. Throughout this paper we shall adopt a minimax viewpoint of robustness similar to [2], [14], where whenever we seek to d esign an esti mator m inimizing an appropriately selected loss function, a hostile player , say ”nature, ” conspi res t o select t he worst possible model in the allowed set, here B , for th e p erformance ind ex to be min imized. This approach is rather February 2, 2018 DRAFT 5 conservati ve, and the performance of estimators in the presence of modelling uncertainties could be ev aluated differently , for example by ave raging the performance index over the entire b all B of po ssible models. Howe ver t his av eraging operation is computationall y demanding, as it requires a Mo nte Carlo simulatio n, and typi cally does not yield analyt ically tractable results. It is als o worth pointing out that the degree of conservati veness result ing from the s election of a minimax estim ator can be controlled by appropriate s election of the tolerance parameter c , which ensures that an adequate balance between performance and robustness is reached. In this paper , we shall us e the mean-square error (scaled by 1 / 2 ) J ( ˜ f , g ) = 1 2 ˜ E [ || x − g ( y ) | | 2 ] = 1 2 Z R n + p || x − g ( y ) || 2 ˜ f ( z ) dz (2.4) to e valuate the p erformance o f an est imator ˆ x = g ( y ) of x based on obs erv ation y . In (2.4), if v denotes a vector of R n with entries v i , || v || = ( v T v ) 1 / 2 =  n X i v 2 i  1 / 2 denotes the usual Eucli dean vector norm. Let G denote the class of estimators such th at ˜ E [ ˆ x 2 ] is finite for all ˜ f ∈ B . Then t he optimal robust estimator solves t he minimax problem min g ∈G max ˜ f ∈B J ( ˜ f , g ) . (2.5) Since the functional J ( ˜ f , g ) i s q uadratic i n g , and t hus con ve x, and linear in ˜ f , and thus concave , a saddle-point ( ˜ f 0 , g 0 ) of minim ax problem (2.5) exists, so that J ( ˜ f , g 0 ) ≤ J ( ˜ f 0 , g 0 ) ≤ J ( ˜ f 0 , g ) . (2.6) Howe ver , characterizing precisely this s addle-point is difficult, except when the nominal density is Gaussian, i.e. f ( z ) ∼ N ( m z , K z ) , (2.7) where in con formity with partit ion (2.1) of z , the mean vector m z and covariance matrix K z admit the partitio ns m z =   m x m y   , K z =   K x K xy K y x K y   . February 2, 2018 DRAFT 6 Then it was shown in Theorem 1 of [1] (see also [14, Sec. 7.3] for an equiv alent result deriv ed from a stochastic game theory perspectiv e) that t he s addle point of mini max poblem (2.5) admit s the following structure. Theor em 1: If f admits the Gaussian dist ribution (2.7), the least-fa vorable density ˜ f 0 is also Gaussian with dist ribution ˜ f 0 ∼ N ( m z , ˜ K z ) , (2.8) where the covariance matrix ˜ K z =   ˜ K x K xy K y x K y   (2.9) is o btained b y perturbi ng onl y the covar iance matrix of x , l ea ving the cross- and co-variance matrices K xy and K y unchanged. Accordingl y , the robust estimator ˆ x = g 0 ( y ) = m x + K xy K − 1 y ( y − m y ) (2.10) coincides with the usual least-squ ares estim ator for nom inal density f . The perturbed cova riance matrix ˜ K x can be ev aluated as follows. Let P = K x − K xy K − 1 y K y x ˜ P = ˜ K x − K xy K − 1 y K y x (2.11) denote the nom inal and least-fa vora ble error covar iance matrices of x giv en y . Then ˜ P − 1 = P − 1 − λ − 1 I n , (2.12) where λ denotes the Lagrange multipl ier correspon ding to constraint D ( ˜ f , f ) ≤ c . Note that to ensure that ˜ P is a po sitive definite m atrix, we must h a ve λ > r ( P ) , w here r ( P ) denot es the spectral radius (the lar gest eigen v alue) of P . T o explain prec isely how λ is selected to ensure that th e Karush-Kuhn-T ucker ( KKT) condition λ ( c − D ( ˜ f 0 , f )) = 0 (2.13) holds, ob serve first that for two Gauss ian densities f ∼ N ( m z , K z ) and ˜ f ∼ N ( ˜ m z , ˜ K z ) , th e relativ e entropy can be expressed as [22] D ( ˜ f , f ) = 1 2 h || ∆ m z || 2 K − 1 z + tr  K − 1 z ˜ K z − I n + p  − ln det  K − 1 z ˜ K z  i , (2.14) February 2, 2018 DRAFT 7 where ∆ m z = ˜ m z − m z and || v || K − 1 △ = ( v T K − 1 v ) 1 / 2 . Then for the nominal and least-fa vorable densities specified by (2.7 ) and (2.8)–(2.9), we hav e ∆ m z = 0 and K z =   I n G 0 0 I p     P 0 0 K y     I n 0 G T 0 I p   ˜ K z =   I n G 0 0 I p     ˜ P 0 0 K y     I n 0 G T 0 I p   , where G 0 △ = K xy K − 1 y denotes the gain m atrix o f the optim al est imator (2.10). Then after si mple algebraic manipul ations, we find D ( ˜ f 0 , f ) = 1 2 h tr ( ˜ P P − 1 − I n ) − ln det( ˜ P P − 1 ) i . (2.15) Substitutin g (2.12) gives γ ( λ ) △ = D ( ˜ f 0 , f ) = 1 2 h tr (( I n − λ − 1 P ) − 1 − I n ) + ln det( I n − λ − 1 P ) i (2.16) where γ ( λ ) is d iff erentiable over ( r ( P ) , ∞ ) . By using the matrix differentiation ident ities [23, Chap. 8] d dλ ln det M ( λ ) = tr [ M − 1 dM dλ ] d dλ M − 1 ( λ ) = − M − 1 dM dλ M − 1 , for a square i n vertible matrix function M ( λ ) , we find dγ dλ = − tr  ( I − λ − 1 P ) − 1 λ − 3 P 2 ( I − λ − 1 P ) − 1  < 0 (2.17) so that γ ( λ ) is monoto ne decreasing over ( r ( P ) , ∞ ) . Since lim λ → r ( P ) γ ( λ ) = + ∞ , lim λ →∞ γ ( λ ) = 0 this ensures that for an arbitrary tolerance c > 0 , there exists a uni que λ > r ( P ) such t hat γ ( λ ) = c . For the case where the nomi nal dens ity f is non-Gaussian, some results characterizing the solution of the minimax probl em (2.5) were described recently in [24]. In addition , it is worth noting that i t is assu med in Th eorem 1 t hat the whol e dens ity f ( z ) = f ( x, y ) is subject to February 2, 2018 DRAFT 8 uncertainties. But this assumptio n does not fit all sit uations. Consid er for example a mutiple- input multi ple-output (MIM O) least-squares equ alization problem for a flat channel described by the nom inal linear model y = C x + v (2.18) where x denotes t he transmitted data, C is the channel matrix and v ∼ N (0 , R ) represents the channel noise, whi ch is assumed independent of x . Since t he transmitted data x is under the control of the d esigner , it s prob ability distri b ution f ( x ) is known exactly and i t i s no t realis tic to assume that it is affected by modelling uncertainties. Thus if f ( y | x ) ∼ N ( C x, R ) denot es the nominal conditional distribution specified by (2.18), the actual density of z can be represented as ˜ f ( z ) = ˜ f ( y | x ) f ( x ) , where ˜ f ( y | x ) represents the true channel model, and where the data density f ( x ) i s not perturbed. This c onstraint changes the structure of the mini max problem (2.5), and a solution to this modified problem is presented in [24] and [25 ]. I I I . R O B U S T F I LT E R I N G V I E W E D A S A D Y NA M I C G A M E W e consider a robust state-space filtering problem for processes described by a nomin al Gauss- Markov state-space m odel of the form x t +1 = A t x t + B t v t (3.1) y t = C t x t + D t v t , (3.2) where v t ∈ R m is a WGN wi th unit variance, i .e., E [ v t v T s ] = I m δ ( t − s ) , (3.3) where δ ( r ) =    1 r = 0 0 r 6 = 0 denotes the Kronecker delta function. The noi se v t is assum ed to be independent of the initial state, whose nom inal di stribution is given by f 0 ( x 0 ) ∼ N ( m 0 , P 0 ) . (3.4) February 2, 2018 DRAFT 9 Let z t △ =   x t +1 y t   , The model (3.1)–(3.3) can be viewed as specifying the n ominal conditional density φ t ( z t | x t ) ∼ N    A t C t   x t ,   B t D t   h B T t D T t i  . (3.5) of z t giv en x t . W e ass ume th at the nois e v t af fects all com ponents of the d ynamics (3.1) and observations (3.2), so t hat the cova riance matrix K z t | x t =   B t D t   h B T t D T t i (3.6) is positive definite. T o interpret this assum ption, observe t hat i n general, state-space m odels of the form (3.1)–(3.2) are formed by a mixture of noisy and determinis tic li near relatio ns (see for example the decompo sition employed in [26]). This means that the resulting cond itional densities are concentrated o n lower -dimensi onal subspaces o f R n + p . As soon as these sub spaces are sligh tly perturbed, it is possible to discriminate perfectly b etween the nomi nal and perturbed models, i.e., the relativ e entropy of t he two m odels is i nfinite (the correspondi ng probabilit y measures are not absolutely cont inuous wi th respect t o each other). Accordingl y , when the relativ e entropy is used to measure the prox imity of st atistical m odels, all determin istic relations between dy namic variables or observations are int erpreted as i mmune from un certainty , and only relations where noise is already present can be perturbed. Since this l imitation is rather unsatisfactory , it is con venient to assume, like earlier robust filtering st udies [12], [17], [18], t hat the noise v t af fects all compo nents of t he dy namics and observations, possibl y with a very small var iance for relations which are viewed as almo st certain. In this case, since the matrix Γ t △ =   B t D t   has full row rank, we can ass ume wit hout l oss of generality that Γ t is square and in vertible, so that m = n + p . Otherwise, if m > n + p , we can find an m × m ortho normal m atrix U t which compresses the colum ns of Γ t , so that Γ t U t = h ¯ Γ t 0 i February 2, 2018 DRAFT 10 where ¯ Γ t is in vertible. Then if we denot e U T t v t =   ¯ v t v c t   , we hav e Γ t v t = ¯ Γ t ¯ v t where ¯ v t is a zero-mean WGN of dimensi on n + p with un it cov ariance matrix. Over a finit e interval 0 ≤ t ≤ T , the joint no minal probability densit y of X T +1 =           x 0 . . . x t . . . x T +1           and Y T =           y 0 . . . y t . . . y T           can be expressed as f ( X T +1 , Y T ) = f 0 ( x 0 ) T Y t =0 φ t ( z t | x t ) , (3.7) where the init ial and the combined state t ransition and observation d ensities are given by (3.4 ) and (3.5). Assum e that the t rue probabil ity density of X T +1 and Y T admits a simi lar Markov structure of the fo rm ˜ f ( X T +1 , Y T ) = ˜ f 0 ( x 0 ) T Y t =0 ˜ φ t ( z t | x t ) . (3.8) By taking the expectation of ln  ˜ f ( X T +1 , Y T ) f ( X T +1 , Y T )  = ln  ˜ f 0 ( x 0 ) f 0 ( x 0 )  + T X t =0  ˜ φ t ( z t | x t ) φ t ( z t | x t )  with r espect to ˜ f ( X T +1 , Y T ) , we find that the relati ve entropy between ˜ f ( X T +1 , Y T ) and f ( X T +1 , Y T ) satisfies the chain rule D ( ˜ f , f ) = D ( ˜ f 0 , f 0 ) + T X t =0 D ( ˜ φ t , φ t ) , (3.9) with D ( ˜ φ t .φ t ) = ˜ E  ln  ˜ φ t ( z t | x t ) φ t ( z t | x t )   = Z Z ˜ φ t ( z t | x t ) ˜ f t ( x t ) ln  ˜ φ t ( z t | x t ) φ t ( z t | x t )  dz t dx t , (3.10) February 2, 2018 DRAFT 11 where ˜ f t ( x t ) denot es the true m ar ginal dens ity of x t . Up to this po int, most results on robust Kalman filtering with a relativ e ent ropy constraint have been obtained by cons idering a fixed interval and applying a sin gle const raint t o the relative entropy D ( ˜ f , f ) of the true and nominal probability densiti es of the state and o bserva tion sequences over the wh ole interval. This was also the po int of view adopt ed in Section 4 o f [1] which examined the robust causal W iener filtering problem over a finit e interval. T o treat the robust filtering problem over an infinite horizon , one approach cons ists of d ividing the dive rgence over a finite interval by t he l ength T of the interval, and letting T tend to i nfinity , assuming this sequence has a li mit. This is the case for stationary Gaussian processes , s ince in this case the limit is the Itakura-Saito spectral dist ortion measure [1, Sec. 3]. Alt ernati vely , it is als o possible to apply a discount factor [15] t o future additive t erms appearing in t he chain rule decomposit ion (3.9). Howe ver , on e potent ial weakness of applyi ng a single dive rgence cons traint to the filtering problem over a finite o r i nfinite in terv al is that it allows th e maxim izer (nat ure) to i dentify t he mo ment w here the dynamic model (3.1)-(3.3) is most susceptible to dist ortions and to allocate most of the d istortion budget specified by the tolerance parameter c to this single element of t he model. If the p urpose of robust filtering is t o protect the esti mator from modell ing inaccuracies, and if th e modeller exerc ises the sam e level of ef fort to characterize each ti me component of the model (3.1)–(3.3), it may be more appropriate to specify separate m odelling tolerances for each time step of the transiti on densit y (3.5). Such a viewpoint has in fact been adopted widely [11]–[13] in the robust state-space filtering li terature, except that in t hese earlier studies the tolerance is usually expressed in t erms of matrix bounds in v olvin g the matrices A t , B t , C t and D t parametrizing the state dynam ics and o bserv ations. The main diffe rence with these earlier studies is that we use here the relativ e entropy D ( ˜ φ t , φ t ) between the true and no minal transition and observation densit ies ˜ φ t ( z t | x t ) and φ t ( z t | x t ) at time t to measure m odelling errors. The expression (3.10) for the relativ e entropy raises im mediately the issue of how t o choose the probabil ity density ˜ f t ( x t ) used to ev alu ate the diver g ence. W e assume that, like t he estimati ng player , at time t the maximizer has access t o the o bserva tions { y ( s ) , 0 ≤ s ≤ t − 1 } coll ected prior to this point. In addition, it is reasonable to hold the maximizer to the same Markov structure (specified by (3.8 )) as the estimating player . T herefore, the maxim izer is required to commit to all the least-fa vorable model com ponents ˜ φ s ( z s | x s ) wi th 0 ≤ s ≤ t − 1 generated at earlier stages of its min imax gam e wi th th e estimating player . Using the termi nology coined in February 2, 2018 DRAFT 12 [14], [15], the maximi zer operates ”under commit ment. ” Thus if Y t − 1 denotes the vector formed by t he observations { y s , 0 ≤ s ≤ t − 1 } , we us e the conditi onal d ensity ˜ f t ( x t | Y t − 1 ) based on the least fa vorable model and the given observations prior to time t , t o ev aluate the diver gence (3.10) between th e true and n ominal t ransition and observation densities. The mod el m ismatch tolerance can therefore be expressed as ˜ E h ln  ˜ φ t ( z t | x t ) φ t ( z t | x t )     Y t − 1 i ≤ c t , (3.11) where c t denotes the tol erance parameter for the time t component of t he model, with ˜ E h ln  ˜ φ t ( z t | x t ) φ t ( z t | x t )     Y t − 1 i = Z Z ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) ln  ˜ φ t ( z t | x t ) φ t ( z t | x t )  dz t dx t . (3.12) Let B t denote the con vex ball of functions ˜ φ t ( z t | x t ) satis fying i nequality (3.11). If G t de- notes the cl ass of estim ators wi th finite second-order moment s with respect to all densities ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) su ch that ˜ φ t ( z t | x t ) ∈ B t , the dynami c min imax game we consider can be expressed as min g t ∈G t max ˜ φ t ∈B t J t ( ˜ φ t , g t ) (3.13) where J t ( ˜ φ t , g t ) = 1 2 ˜ E h || x t +1 − g t ( y t ) || 2 | Y t − 1 i = 1 2 Z Z || x t +1 − g t ( y t ) || 2 ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dz t dx t . (3.14) denotes the m ean-square error of esti mate ˆ x t +1 = g t ( y t ) of x t +1 e valuated with respect to the true prob ability density of z t . Not e that since ˆ x t +1 is a function of Y t , it depends not on ly on y t , but also on earlier observations, but this dependency is sup pressed to simpl ify no tations. Note that by taki ng iterated expectations, we hav e ˜ E [ || x t +1 − ˆ x t +1 || 2 ] = ˜ E [ ˜ E [ || x t +1 − ˆ x t +1 || 2 Y t − 1 ]] = Z J t ( ˜ φ t , g t ) ˜ f t − 1 ( Y t − 1 ) d Y t − 1 , (3.15) where ˜ f t − 1 ( Y t − 1 ) represents t he l east fa vorable density o f observation vector Y t − 1 based on the least-fa vorable m odel increments sel ected by the maximi zer up to ti me t − 1 . Since this densit y February 2, 2018 DRAFT 13 is non-negativ e and in dependent of both ˜ φ t and g t , the game (3.1 3) is equiv alent to min g t ∈G t max ˜ φ t ∈B t ˜ E [ || x t +1 − ˆ x t +1 || 2 ] . (3.16) This i ndicates that the robust least-squ ares filtering prob lem we consider is local in the sense that the estimato r and the maxi mizer focus respectiv ely on mi nimizing and maximizi ng th e mean- square est imation error at the current tim e. In other words, i n add ition to bein g commi tted to past least-fa vorable model increments ˜ φ s with 0 ≤ s ≤ t − 1 i t has already selected, t he m aximizer is confined to a myopic strate gy , where ˜ φ t is selected exclusively to maxim ize the mean-square estimation error at tim e t . By doi ng so, the maximizer foregoes the possibil ity o f trading off a lesser increase i n the mean-square error at the current time against larger increases in th e future. If we compare the dynami c estimatio n game (3.13) and its static counterpart (2.5), we see that the two problems are simil ar , but the dynamic game (3.13) includes a conditi oning operation with respect to the pri or state x t , combined with an av eraging o peration with respect to ˜ f t ( x t | Y t − 1 ) . Thus Lemma 1 and Theorem 1 of [1] need to be extended sl ightly to accommod ate these diffe rences. Before proceeding with thi s task it is worth poin ting o ut th at we do no t require that ˜ φ t ( z t | x t ) should be a cond itional probabi lity densi ty for each x t . It is on ly requi red that the product ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) should be a probability density for   z t x t   =      x t +1 y t x t      , so that I t ( ˜ φ t ) △ = Z Z ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dz t dx t = 1 . (3.17) T o p ut i t anot her way , th e maxim izer’ s commi tment to earlier components of the least-fa vorable model is o nly of an a-prio ri nature, sin ce the a-posteriori m ar ginal dens ity o f x t specified by th e joint density ˜ φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) is not required to coincide with the a priori dens ity ˜ f t ( x t | Y t − 1 ) . Howe ver , by integrating out z t , th e resulting a-posteriori densit y wi ll be of t he form ψ t ( x t ) ˜ f t ( x t | Y t − 1 ) where ψ t ( x t ) does not depend on Y t − 1 . Relation to prior work: At this point it is poss ible to compare precisely the robust filtering problem discus sed here with earlier formulations of robust filtering wit h a relative entropy February 2, 2018 DRAFT 14 tolerance considered in [1], [14], [15], [18]. In [14], [15], Hansen and Sar gent i ntroduce th e likelihood ratio function M t = ˜ f ( X t +1 , Y t ) f ( X t +1 , Y t ) (3.18) between the distorted joint densi ty of states { x s , 0 ≤ s ≤ t + 1 } and observations { y s , 0 ≤ s ≤ t } and their nomi nal joint density . Setting M − 1 = ˜ f 0 ( x 0 ) f 0 ( x 0 ) , if Z t denotes the σ -field generated by { z s , 0 ≤ s ≤ t } and x 0 , we hav e E [ M t |Z t − 1 ] = M t − 1 , for t ≥ 0 , where E [ · ] denot es the expectation with respect to t he nominal model dist ribution, so M t is a martingale. Then t he relativ e entropy betw een the p erturbed and nomi nal m odels over interval 0 ≤ t ≤ T can be expressed as D ( ˜ f , f ) = E [ M T ln M T ] . (3.19) Let J = T X t =0 ˜ E [ || x t − ˆ x t || 2 ] = T X t =0 E [ M t || x t − ˆ x t || 2 ] (3.20) denote the sum of th e m ean square estimation errors over int erv al [0 , T ] , wh ere the est imate ˆ x t = g t ( Y t ) depends causall y on the p ast and current observations. If g = ( g t , t ≥ 0) , the robust Kalman filtering prob lem of [14 ], [15] and t he robust causal W iener filtering problem of [1, Sec. 4] can be written as min g max ˜ f J (3.21) where ˜ f satisfies the constraint D ( ˜ f , f ) = E [ M T ln M T ] ≤ c . (3.22) Since the causal W iener filtering problem o f [1, Sec. 4] is treated as a constrained stati c estimation prob lem, it do es no t requi re any additional structure. On th e oth er hand for t he robust Kalman filtering problem of [14], [15], the mi nimax problem (3.21) is formulated dynamicall y by introducing th e m artingale increments m t =    M t M t − 1 if M t − 1 > 0 1 if M t − 1 = 0 , (3.23) February 2, 2018 DRAFT 15 which can be expressed here as m t = ˜ φ ( z t | x t ) φ ( z t | x t ) . Then the chain rule (3.10) takes the form E [ M T ln M T ] = T X t =0 E [ M t − 1 E [ m t ln m t |Z t − 1 ]] + E [ M − 1 ln M − 1 ] , (3.24) and the dynami c m inimax problem considered i n [14], [15] can b e expressed as min g t ,t ≥ 1 max m t ,t ≥ 1 J (3.25) where g t and m t ≥ 0 are adapted respectiv ely to Y t (the si gma field spanned by Y t ) and Z t . Comparing (3.16) and (3.25), we see that (3.16) represents just a local or incremental version of minimax problem (3.25). In this respect it is worth poin ting out that by integrating the incremental relativ e entropy constraint (3.1 1) with respect to the least fa vora ble density ˜ f t − 1 ( Y t − 1 ) of Y t − 1 and s umming over all 0 ≤ t ≤ T , we find that cumulatively , the incremental con straints (3.11 ) imply D ( ˜ f , f ) ≤ T X t =0 c t + D ( ˜ f 0 , f 0 ) , (3.26) which h as the form (3.22). In other words, the incremental constraint (3.11 ) j ust represents a way of dividing the relative entropy t olerance budget c in separate porti ons c t allocated to t he distortion of th e M arko v model transi tion at each time step. The robust filtering prob lem we consider can also be interpreted in terms of the robust filt ering approach described in [12 ]. T o do so , assume that the di storted transition ˜ φ t ( z t | x t ) ∼ N    ˜ A t ˜ C t   x t , ˜ K z t | x t  is Gaussian and admi ts a parametrization s imilar to (3 .5). By usin g expression (2.1 4) for the relativ e entropy of Gaussian densiti es, and denoting ∆ A t △ = ˜ A t − A t , ∆ C t △ = ˜ C t − C t : , we find that cond itioned on the k nowledge o f x t D ( ˜ φ t ( . | x t ) , φ t ( . | x t )) = 1 2 h ||   ∆ A t ∆ C t   x t || 2 K − 1 t + tr  K − 1 t ˜ K t − I  − ln det( K − 1 t ˜ K t ) i , (3.27) February 2, 2018 DRAFT 16 where for sim plicity we fa ve used the compact not ation K t = K z t | x t and ˜ K t = ˜ K z t | x t . Then, if ˜ f t ( x t | Y t − 1 ) ∼ N ( ˆ x t , V t ) , is Gaussian and W t △ = V t + ˆ x t ˆ x T t , the expression (3.10) for the relative entropy between the distorted and nomi nal transitions at tim e t yields D ( ˜ φ t , φ t ) = 1 2 h || K − 1 / 2 t   ∆ A t ∆ C t   W 1 / 2 t || 2 F + tr  K − 1 t ˜ K t − I  − ln det( K − 1 t ˜ K t ) i , (3.28) where K 1 / 2 t and W 1 / 2 t are arbitrary matrix sq uare roots of K t and W t , respective ly , and || . || F denotes the Frobenius norm of a matrix [27, p. 2 91]. The first term of expression (3.28) is a weighted matri x norm of the perturbat ions ∆ A t and ∆ C t of th e state transitio n d ynamics which is simil ar in n ature to the mi smodelling measures consi dered in [12], [13]. On th e oth er hand, the second term models the dist ortion o f th e process and measurement nois e cov ariance K t = K z t | x t , and is differe nt from the d istortion metrics considered in [12], [13]. Thus the robust filtering problem we consider can on one hand be viewed as i ncremental version of the results of [14], [15] for robust filtering w ith a relativ e ent ropy tol erance, but it can also be viewed as a variant of th e robust K alman filters dis cussed in [12 ], [13] wi th a differ ent l ocal mismo delling m easure. Formu lation without commitment: The commi tment assum ption can be removed from t he incremental formulation of robust filtering described abo ve by adopting the conceptual frame work of Hansen and Sargent in [14, Chap. 18] and [16], [28] for robust filtering wi thout commitment. The key idea is, at time t , to apply distortio ns to both the transit ion dynamics φ t ( z t | x t ) and the least-fa vorable a-priori distribution ˜ f t ( x t | Y t − 1 ) . In [16] this i s accomplished by in troducing two distortion operators with constant risk -sensitivity parameters θ 1 and θ 2 , acting respectively on the t ransition dynami cs and on the a-priori density at ti me t based on the prior dist ortions and observations. Here, if ˇ f t ( x t | Y t − 1 ) denotes a distorted version of ˜ f t ( x t | Y t − 1 ) , in addi tion to the constraint (3.11) for transi tion dynamics distorti on, we could impose a con straint of the form D ( ˇ f t , ˜ f t ) ≤ d t (3.29) on the allowed disto rtion of th e l east fa vorable conditi onal disto rtion for x t based on the past observations Y t − 1 Then in the m inimax prob lem (3.13) the maximizati on can be performed jointly over pairs ( ˜ φ t , ˇ f t ) s atisfying cons traints (3.11) and (3.29). Since the two constraints are February 2, 2018 DRAFT 17 con vex, the st ructure of th e resul ting minim ax problem is similar to (3.13). The m ain difficulty is algorithmic. The inner m aximization i ntroduces t wo Lagrange mul tipliers whi ch need to be selected such that Karush-Kuhn-T ucker conditions are satisfied. The computation of the Lagrange multipli ers seems rather d if ficult, in contrast to t he case of a sing le Lagrange m ultiplier arising from the formulation of robust filtering with commi tment. So we focus here on t he case with commitment , leaving op en the possibil ity t hat an implement able algorithm migh t be dev eloped later for robust filtering wi thout commitm ent under incremental constraint s (3.11) and (3.29). Finally , note that a thi rd opti on would be to com bine t he dis tortions for the transitio n density φ t ( z t | x t ) and for conditio nal d ensity ˜ f t ( x t | Y t − 1 ) and to apply a single relative entropy constraint to the pro duct distorted densi ty ˜ φ t ( z t | x t ) ˇ f t ( x t | Y t − 1 ) . Since t his p roduct corresponds t o the least fa vora ble jo int demsity of z t and x t , Theorem 1 is applicable to this prob lem, so it is n ot necessary to analyze this version of the robust filtering problem. I V . R O B U S T M I N I M A X F I L T E R The solutio n of t he dynamic game (3.13) relies o n extending L emma 1 of [1] to the dynam ic case. W e start by observing that the obj ecti ve function J t ( ˜ φ t , g t ) specified by (3.14) is quadratic in g t , and thus con vex, and li near i n ˜ φ t and thus concave . The set B t is con vex and compact. Similarly G t is con vex. It can als o be m ade compact by requi ring that t he second mom ent of estimators g t ∈ G t should have a fixed but large upper bound. Then by V on Neumann’ s min imax theorem [29, p. 319], there exists a saddle p oint ( ˜ φ 0 t , g 0 t ) such that J t ( ˜ φ t , g 0 t ) ≤ J t ( ˜ φ 0 t , g 0 t ) ≤ J t ( ˜ φ 0 t , g t ) . (4. 1) The real challenge is, howe ver , not to establish the existence of a s addle point, but to characterize it compl etely . Th e second i nequality in (4.1) implies that estimator g 0 t is the conditi onal m ean of x t +1 giv en Y t based on the least-fa vora ble density ˜ f t +1 ( x t +1 | Y t ) = R ˜ φ 0 t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dx t R R ˜ φ 0 t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dx t +1 dx t (4.2) obtained by marginalization and application of Bayes’ rul e to the least-fa vorable joint densit y ˜ φ 0 t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) of ( z t , x t ) = ( x t +1 , y t , x t ) given Y t − 1 . The robust estimator is then giv en by ˆ x t +1 = g 0 t ( y t ) = ˜ E [ x t +1 | Y t ] = Z x t +1 ˜ f t +1 ( x t +1 | Y t ) dx t +1 . (4.3) February 2, 2018 DRAFT 18 T ogether , the cond itional density ev aluation (4.2) and expectation (4.3) imp lement the second inequality of saddle point identi ty (4.1). Let us turn now to the first inequality . For a fixed estimator g 0 t , it requires finding the jo int transition and o bserv ation dens ity ˜ φ 0 t ( z t | x t ) maximizing J t ( ˜ φ t , g 0 t ) under the diver gence const raint (3.11). Th e solution of th is problem takes the following form. Lemma 1: For a fixed estim ator g t ∈ G t , the function ˜ φ 0 t maximizing J t ( ˜ φ t , g t ) under constraints (3.11) and (3.17) is give n by ˜ φ 0 t ( z t | x t ) = 1 M t ( λ t ) exp  1 2 λ t || x t +1 − g t ( y t ) || 2  φ t ( z t | x t ) . (4.4) In this expression, the n ormalizing cons tant M t ( λ t ) is selected such th at (3.17) hol ds. Furth er - more, given a tolerance c t > 0 , there exits a unique Lagrange mul tiplier λ t > 0 such that D t ( ˜ φ 0 t , φ t ) = c t . (4.5) Pr oof: For a g iv en g t , th e function J t ( ˜ φ t , g t ) is li near in ˜ φ t and th us concav e over the closed con vex set B t , so it admi ts a uniqu e maxim um in B t . Because of the l inearity of J t with respect to ˜ φ t , th is maximum is in fact located on the bpu ndary of B t . T o find the m aximum, consider the Lagrangian L t ( ˜ φ t , λ t , µ t ) = J t ( ˜ φ t , g ) + λ t ( c t − D t ( ˜ φ t , φ t )) + µ t (1 − I t ( ˜ φ t )) , (4.6) where the Lagrange mul tipliers λ t ≥ 0 and µ t are associated to inequalit y constraint (3.11) and equalit y const raint (3.17), respectively . W e do not require explicitly that ˜ φ t ( z t | x t ) should be nonnegati ve, since the form (4.4) of the maximizin g solution indicates that this constraint is satisfied automatically . Then the Gateaux deriv ative [30, p. 17] of L t with respect to ˜ φ t in the direction of an arbit rary function u is giv en by ∇ ˜ φ t ,u L t ( ˜ φ t , λ t , µ t ) = lim h → 0 1 h [ L t ( ˜ φ t + hu, λ t , µ t ) − L t ( ˜ φ t , λ t , µ t )] = Z Z h 1 2 || x t +1 − g t ( y t ) || 2 − ( λ t + µ t ) − λ t ln  ˜ φ t φ t  i u ( z t , x t ) ˜ f t ( x t | Y t − 1 ) dz t dx t . (4.7) The Lagrangian is m aximized by setting ∇ ˜ φ t ,u L t ( ˜ φ t , λ t , µ t ) = 0 for all functions u . Ass uming λ t > 0 , thi s giv es ln  ˜ φ t φ t  = 1 2 λ t || x t +1 − g t ( y t ) || 2 − ln M t , (4.8) February 2, 2018 DRAFT 19 where ln M t △ = 1 + µ t λ t . Exponentiating (4.8) g iv es (4.4), where t o ensure that normalizatio n (3.1 4) ho lds, we must select M t ( λ t ) = Z Z exp  1 2 λ t || x t +1 − g t ( y t ) || 2  φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dz t dx t . (4.9) At th is point, all what is left is finding a Lagrange multi plier λ t such th at the so lution ˜ φ 0 t giv en by (4.4) satisfies the KKT condit ion λ t ( c t − D t ( ˜ φ 0 t , φ t ) = 0 . Since we already know th at t he maximi zing ˜ φ 0 t is on t he boundary of B t , the Lagrange mult iplier λ t > 0 , so the KKT condi tion reduces to (4.5). By substit uting (4.4) inside expression (3.11) for D ( ˜ φ 0 t , φ t ) , we find D ( ˜ φ 0 t , φ t ) = 1 λ t J t ( ˜ φ 0 t , g t ) − ln( M t ( λ t )) . (4.10) Diffe rentiating ln M t ( λ t ) gives d dλ t ln M t ( λ t ) = − 1 λ 2 t J t ( ˜ φ 0 t , g t ) , (4.11) so that γ ( λ t ) △ = D ( ˜ φ 0 t , phi t ) = − λ t d dλ t ln M t ( λ t ) − ln( M t ( λ t ) . (4.12) The deriv ativ e of γ ( λ t ) is giv en by dγ t dλ t = − λ t d 2 dλ 2 t ln M t − 2 d dλ t ln M t = − 1 λ t h d dλ t  λ 2 t d dλ t ln M t i = 1 λ t d dλ t J t ( ˜ φ 0 t , g t ) = − 1 4 λ 3 t ˜ E  || x t +1 − g t ( y t ) || 2 − ˜ E [ || x t +1 − g t ( y t ) || 2 | Y t − 1 ]  2 | Y t − 1  < 0 , (4.13) so t hat γ ( λ t ) is a m onotone decreasing functi on of λ t . As λ t → ∞ we hav e o bviously ˜ φ 0 t → φ t , so t hat γ ( ∞ ) = 0 . Thus provided c t is l ocated in the range of γ t , which is t he case if c t is suffi ciently sm all, th ere exists a unique c t such that γ t ( λ t ) = c t .  Note t hat Lemma 1 makes no assumpt ion about the form of the nominal transition densi ty φ t ( z t | x t ) and a priori d ensity ˜ f t ( x t | Y t − 1 ) . W i thout additional assump tions, it is difficult to characterize precisely the range of function γ t . When both of these densi ties are Gauss ian, February 2, 2018 DRAFT 20 it wi ll b e sh own below th at the range of γ t is R + , so that any posi tiv e diver g ence tolerance c t can be achiev ed. Howe ver , in practice t he tolerance c t needs to be rath er small in order to ensure that the robust estim ator is not overly conserva tive. At this poi nt is is also worth observing that Lemma 1 i s just a variation of Theorem 2.1 in [31, p. 38] which sought to cons truct t he minimum discrimi nation density (i.e., t he density mi nimizing t he div ergence) wit h respect to a nominal density under various moment constraints . Here we seek to maximi ze th e mom ent ˜ E [ || x t +1 − g t ( y t ) || 2 | Y t − 1 ] under a div ergence con straint. From an opti mization point of view , the two problems are obviously similar , and in fact the functional form (4.4) of the sol ution is t he same for bot h problems. Up to this p oint we have made no assumpti on on either the nom inal transition and observations density φ t ( z t | x t ) and estim ator g t , and in the characterization of t he s addle point solu tion ( ˜ φ 0 t , g 0 t ) provided by identities (4.3) and ( 4.4), the rob ust estimator g 0 t depends on least-fa vorable transition function ˜ φ 0 t , and the least fa vorable transition density ˜ φ 0 t depends on robust estimat or g 0 t . This type of deadlock is typ ical of s addle point analyses, and to break it, we introduce now the assum ption that φ t ( z t | x t ) admits th e Gaussian form (3.5) where as indicated earlier , the covariance matrix K z t | x t is positive definite, and we assu me also that at time t the a-priori conditional density ˜ f t ( x t | Y t − 1 ) ∼ N ( ˆ x t , V t ) . (4.14) Then, observe that the distortion term exp( || x t +1 − g t ( y t ) || 2 / (2 λ t )) appearing in expression (4.4) for the least -fa vorable transi tion function ˜ φ 0 t ( z t | x t ) depends only on z t , but no t x t . Accordingl y , if we int roduce the marginal dens ities ¯ f t ( z t | Y t − 1 ) = Z φ t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dx t (4.15) ˜ f 0 t ( z t | Y t − 1 ) = Z ˜ φ 0 t ( z t | x t ) ˜ f t ( x t | Y t − 1 ) dx t , (4.16) the density ¯ f ( z t | Y t − 1 ) can be viewed as the pseudo-nominal density of z t = ( x t +1 , y t ) condit ioned on Y t − 1 computed from the conditional l east fav orable d ensity ˜ f t ( x t | Y t − 1 ) and nominal transit ion density φ ( z t | x t ) , and from (4.4) we obt ain ˜ f 0 t ( z t | Y t − 1 ) = 1 M ( λ t ) exp  1 2 λ t || x t +1 − g t ( y t ) || 2  ¯ f t ( z t | Y t − 1 ) . (4.17) Since densities φ t ( z t | x t ) and ˜ f t ( x t | Y t − 1 ) are both Gaussian, the int egration (4. 15) yields a February 2, 2018 DRAFT 21 Gaussian pseudo-nomi nal densi ty ¯ f t ( z t | Y t − 1 ) ∼ N    A t C t   ˆ x t , K z t ) (4.18) where the condi tional covariance matrix K z t is give n by K z t =   A t C t   V t h A T t C T t i +   B t D t   h B T t D T t i . (4.19) By integrating out x t in (4.9), we find M t ( λ t ) = Z exp  1 2 λ t || x t +1 − g 0 ( y t ) || 2  ¯ f t ( z t | Y t − 1 ) dz t , which ensures t hat ˜ f 0 t ( z t | Y t − 1 ) is a probability densi ty . Furthermore, by direct subs titution, we hav e D ( ˜ f 0 t , ¯ f t ) = D ( ˜ φ 0 t , φ t ) = c t . (4.20) The least-fav orable density ˜ f t +1 ( x t +1 | Y t ) specified by (4.2) can also be expressed in t erms of ˜ f 0 t ( z t | Y t − 1 ) as ˜ f t +1 ( x t +1 | Y t ) = ˜ f 0 t ( z t | Y t − 1 ) R ˜ f 0 t ( z t | Y t − 1 ) dx t +1 . (4.21) Equiv alent Static Problem: Let ¯ B t = { ˜ f t : D ( ˜ f t , ¯ f t ) ≤ c t } (4.22) denote the ball of dist orted densities ˜ f t ( z t ) wi thin a div ergence tol erance c t of ps eudo-nominal density ¯ f t ( z t | Y t − 1 ) . T o this ball we can of course att ach a static min imax est imation problem min g t ∈G t max ˜ f t ∈ ¯ B t J t ( ˜ f t , g t ) . (4.23) The solut ion of this problem satisfes the saddle poi nt in equality J ( ˜ f t , g 0 t ) ≤ J ( ˜ f 0 t , g 0 t ) ≤ J t ( ˜ f 0 t , g t ) . (4.24) At this poi nt, observe that if ( ˜ φ 0 t , g 0 t ) solves the dynamic mi nimax game (3.12), and if ˜ f 0 t is given by (4.17) with g t = g 0 t , where λ t is selected such that constraint (4.20) i s sat isfied, then ( ˜ f 0 t , g 0 t ) is a saddle poin t of the static problem (4. 23). In other words, the marginalization operation (4.16) has t he ef fect of mapping the solution of dyn amic game (3.12) int o a solu tion of the static est imation problem problem (4.23). Note ind eed t hat the solu tion of the maxim ization February 2, 2018 DRAFT 22 problem formed by the first inequali ty of (4.24) is giv en by (4.17) w ith g t = g 0 t . Sim ilarly , since g 0 t is the mean of t he conditio nal densit y ˜ f t +1 ( x t +1 | Y t ) s pecified by (4.21), it obeys the second inequality of (4.24). Since the pseudo-nominal densit y ¯ f t ( z t | Y t − 1 ) specifyin g the center of ball ¯ B t is Gauss ian, Theorem 1 is applicable with f → ¯ f t , ˜ f 0 → ˜ f 0 t and g 0 → g 0 t . Hence the least-fav o rable density takes the form ˜ f 0 t ( z t | Y t − 1 ) ∼ N    A t C t   ˆ x t , ˜ K z t  (4.25) where the covariance matrix ˜ K z t =   ˜ K x t +1 K x t +1 y t K y t x t +1 K y t   is obtained by perturbing only the (1,1) block K x t +1 = A t V t A T t + B t B T t of the cova riance matrix K z t giv en by (4.19). The robust estimator takes the form ˆ x t +1 = g 0 t ( y t ) = A t ˆ x t + G t ( y t − C t ˆ x t ) (4.26) with the m atrix gain G t = K x t +1 y t K − 1 y t = ( A t V t C T t + B t D T t )( C t V t C T t + D t D T t ) − 1 . (4.27) The least-fa vorable cov ariance m atrix ˜ K x t +1 can be ev aluated as follows. Let P t +1 = K x t +1 − K x t +1 y t K − 1 y t K y t x t +1 = ( A t − G t C t ) V t ( A t − G t C t ) T + ( B t − G t D t )( B t − G t D t ) T (4.28) and V t +1 = ˜ K x t +1 − K x t +1 y t K − 1 y t K y t x t +1 (4.29) denote the nom inal and least-fa vora ble conditional cov ariance matrices of x t +1 giv en Y t . Then V − 1 t +1 = P − 1 t +1 − λ − 1 t I n , (4.30) where the Lagrange multipli er λ t > r ( P t +1 ) is selected such that γ t ( λ t ) = 1 2 h tr (( I n − λ − 1 t P t +1 ) − 1 − I n ) + ln det( I n − λ − 1 t P t +1 ) i = c t . (4.31) February 2, 2018 DRAFT 23 where as indicated in (2.17), γ t ( λ t ) is monot one d ecreasing over ( r ( P t +1 ) , ∞ ) and h as for range R + . Th us for any dive rgence to lerance c t > 0 , th ere exists a matching Lagrange multip lier λ t > r ( P t +1 ) . Summary: The least-fa vorable conditional distribution of x t +1 giv en Y t is give n by ˜ f t +1 ( x t +1 | Y t ) ∼ N ( ˆ x t +1 , V t +1 ) , (4.32) where the estimate ˆ x t +1 is obtain ed by propagating the filter (4.26)–(4.27) and the conditi onal cov ariance matrix V t +1 is obtained from (4.28) and (4.30), with th e Lagrange multip lier λ t specified by (4.31). By writ ing θ t = λ − 1 t , we recognize i mmediately that the robust filter is a form of risk-sensitive filt er of the type discuss ed in [7] [32, Chap. 10]. Howe ver , there is a n e w twist in t he sense that, wh ereas stand ard risk-sensit iv e filt ering uses a fixed risk sensitivity parameter θ , here θ is time-dependent. Specifically , in classical risk-sensit iv e filters, θ is an exponentiation parameter appearing in the exponential of quadratic cost to be m inimized. Similarly , in earlier works [1], [14], [15], [17], [18] relating ri sk sens itiv e filtering with mi nimax filtering with a relative entropy const raint, a singl e global relative entropy constraint i s imposed, result ing in a s ingle Lagrange m ultiplier/ris k sensitivity parameter . Here each compon ent φ t ( z t | x t ) of t he model has an associated relative entrop y constraint (3.10), where the tolerance c t var ies in in verse proportion with the modeller’ s confidence in the m odel component . In thi s respect, even if t he state-space m odel (3.1)–(3.2) is tim e-in variant ( A , B , C and D are const ant) and the tol erance c t = c is const ant, the risk sensi tivity parameter λ − 1 t will b e generally t ime-var ying. O n the other hand, i f we insist on ho lding θ = λ − 1 constant, it m eans that the tolerance c t = γ t ( λ ) i s ti me var ying si nce th e cov ariance matrix P t +1 giv en by (4.28) depends on time. V . L E A S T - F A V O R A B L E M O D E L In last section, we derived the rob ust filter , which is of course the most important component of the soluti on of the mi nimax filtering problem. Howe ver for simu lation and performance e valuation purpos es, it is als o useful t o construct the least fa vorable mo del corresponding to the optimum filter . Before proceeding, note that if e t = x t − ˆ x t denotes th e stat e estimatio n error , by subtracting (4.26) from the state dynamics (3.1) and taking i nto account expression (3.2) for the observations, t he est imation error dynam ics are given by e t +1 = ( A t − G t C t ) e t + ( B t − G t D t ) v t , (5.1) February 2, 2018 DRAFT 24 where in the no minal model, the driving noise v t is independent of error e t = x t − ˆ x t , since ˆ x t depends exclusiv ely o n observations { y ( s ) , 0 ≤ s ≤ t − 1 } . T o find the least-fa vorable model, we use t he characterization (4.4) where g t = g 0 t is giv en by the robust filter (4.26). This gives ˜ φ 0 t ( z t | x t ) = 1 M t ( λ t ) exp  || e s +1 || 2 2 λ t  φ t ( z t | x t ) . (5.2) At th is poi nt, recall that ˜ φ 0 t ( z t | x t ) is an unormalized density . Specifically , integrating it over z t does not yield one, but as we shall see below , a positive function of e t = x t − ˆ x t . This feature indicates that the maxim izing pl ayer has t he oppo rtunity t o change retroacti vely the least-fa vorable densi ty of x t (and therefore o f earlier states) by selecting the mod el component ˜ φ 0 t ( z t | x t ) . Properly accounting for th is retroactive change forms an im portant aspect of the deriv at ion of the least-fa vorable m odel. Instead of attem pting to characterize directly the least- fa vora ble densit y of z t giv en x t , it is easier to find the least-fav orable densit y of the driving noise v t = Γ − 1 t  z t −   A t C t   x t  . (5.3) Giv en x t , the transformat ion (5.3) establ ishes a one-to-one correspondence between z t and v t , so there i s no loss o f informati on in characterizing the least-fav o rable model i n terms of v t . Let ψ t ( v t ) and ˜ ψ t ( v t | e t ) denote respective ly t he nominal and least-fa vorable densities of n oise v t , where as wil l be shown below , ˜ ψ t ( v t | e t ) actually depend s on e t . The nominal distribution is giv en by ψ t ( v t ) = 1 (2 π ) n + p exp( −|| v t || 2 / 2) . (5.4) Assume t hat we seek to construct the least-fa vorable m odel of z t over a fixed interval 0 ≤ t ≤ T , and that the least-fa vora ble n oise distribution ˜ ψ s ( v s | e s ) has been identified for t + 1 ≤ s ≤ T . Accordingly , we hav e T Y s = t +1 exp  || e s +1 || 2 2 λ s  ψ s ( v s ) ∼ exp  || e t +1 || 2 Ω − 1 t +1 / 2  T Y s = t +1 ˜ ψ s ( v s | e s ) , (5.5) where ∼ ind icates equal ity up to a m ultiplicative constant. The term exp  || e t +1 || 2 Ω − 1 t +1 / 2) appear - ing i n the above expression accounts for the cum ulative effect of retroactiv e probabi lity density changes performed by th e maximizin g player . Here Ω t denotes a p ositive definite matrix of February 2, 2018 DRAFT 25 dimension n wh ich i s ev aluated recursi vely . Then the least-fa vorable model ˜ ψ t ( v t | e t ) is o btained by backward inductio n. Decrementing th e index t b y 1 i n (5.5) gives t he identity exp   || e t +1 || 2 Ω − 1 t +1 + λ − 1 t || e t +1 || 2  / 2  ψ t ( v t ) ∼ ˜ ψ t ( v t | e t ) exp  || e t || 2 Ω − 1 t / 2  . (5.6) Let W t +1 △ = (Ω − 1 t +1 + λ − 1 t I n ) − 1 . (5.7) Then by subs tituting the error dynamics (5.1), t he left hand si de of identi ty (5.6) becomes exp   || ( A t − G t C t ) e t + ( B t − G t D t ) v t ) || 2 W − 1 t +1 − || v t || 2  / 2  , (5.8) and the right-hand side of (5.6) is obtained by decomposing th e quadratic exponent of (5. 8) as a sum of squ ares in v t and e t . By doing so, we find that the least-fa vorable noise d ensity is g iv en by ˜ ψ t ( v t | e t ) ∼ N ( H t e t , ˜ K v t ) , (5.9) where ˜ K v t =  I n + p − ( B t − G t D t ) T W − 1 t +1 ( B t − G t D t )  − 1 (5.10) and H t = ˜ K v t ( B t − G t D t ) T W − 1 t +1 ( A t − G t C t ) . (5.11) Thus the l east-fa vorable density of the noise v t in v olves a perturbation of both the mean and the variance of the nominal nois e distribution. The mean pertu rbation is propo rtional to t he filtering error e t , which creates a coupling beetween th e robust filt er and the least fa v orable model specified by dynamics and obs erv ations (3.1 )–(3.2) and least-fa vorable noise st atistics (5.9)–(5.11). Finally , by matching quadratic com ponents in e t on both s ides of (5.6), we find Ω − 1 t = ( A t − G t C t ) T W − 1 t +1 ( A t − G t C t ) − H T t ˜ K v t H t (5.12) where ˜ K v t and H t are giv en by (5.10) and (5.11). By usin g the matrix i n version l emma [33, p. 48] ( α + β γ δ ) − 1 = α − 1 − α − 1 β ( γ − 1 + δ α − 1 β ) − 1 δ α − 1 February 2, 2018 DRAFT 26 with α = W t , β = ( B t − G t D t ) , γ = − I n + p and δ = ( B t − G t D t ) T on t he right-hand s ide of (5.12), we ob tain Ω − 1 t = ( A t − G t C t ) T [ W t +1 − ( B t − G t D t )( B t − G t D t ) T ] − 1 ( A t − G t C t ) . (5.13) The recursion (5.13), together with (5.7) sp ecifies a backward backward recursion which i s used to account for retroactiv e changes of previous least-fav orable model d ensities performed b y the maximizing player . The backards recursion is initialized with Ω − 1 T +1 = 0 , or equiv alently , W T +1 = λ T I n + p . (5.14) In this respect, it i s interesti ng to note that recursion (5.13) can be re written in t he forward direction as W t +1 = ( A t − G t C t )Ω t ( A t − G t C t ) T + ( B t − G t D t )( B t − G t D t ) T (5.15) and (5.7) is of course equivalent to Ω t +1 = ( W − 1 t +1 − λ − 1 t I n ) − 1 . (5.16) Thus Ω t and W t obey exactly the same forward recursions as as V t and P t , but they are com puted in t he backward direction. Indeed, observe that the matri ces Ω − 1 t and W − 1 t are t ypically very small, so it is m uch easier to m aintain positive-defi niteness by using (5. 7) which accumulates small positive terms, instead of using (5.16) which subtracts a small positive-definite matrix from another one. The least-fa vorable noise model (5.9)–(5.11) can be viewed as a t ime-var ying version of the least-fa vorable m odel derive d asym ptotically for the case o f a constant model b y Hansen and Sar gent in [14, Sec. 17.7]. Specifically , the dynamics of the l east-fa v orable model described in [14] are expressed in terms of the sol ution of a determinist ic infini te-horizon li near -quadratic regulator problem. The counterpart o f th is regulator is formed here by backward recursion (5. 13), (5.7). The model (5.9)–(5.11) i ndicates that the driving noise v t admits the representation v t = H t e t + L t ǫ t (5.17) where L t is an arbitrary m atrix square root of ˜ K v t , i.e., L t L T t = [ I n + p − ( B t − G t D t ) T W − 1 t +1 ( B t − G t D t )] − 1 , (5.18) February 2, 2018 DRAFT 27 and ǫ t is a zero-mean WGN of var iance I n + p . Accordingl y , as was previously observed in [14 ], if ξ t △ =   x t e t   , ( 5.19) the least-fa vorable mo del admits a st ate-space representation ξ t +1 = ˜ A t ξ t + ˜ B t ǫ t y t = ˜ C t ξ t + ˜ D t ǫ t (5.20) with twice th e di mension of the n ominal state-space model, where ˜ A t =   A t B t H t 0 A t − G t C t + ( B t − G t D t ) H t   , ˜ B t =   B t ( B t − G t D t )   L t ˜ C t = h C t D t H t i , ˜ D t = D t L t . ( 5.21) Note that the model (5.20 )–(5.21) is constructed by performing first a forward sweep of the risk-sensitive filter (4.27)–(4.28) over in terv al [0 , T ] to generate the gains G t , followed by a backward sweep used to ev aluate t he matrix sequence W t . Thus, the least-fav orable m odel is constructed in a nonsequenti al manner , since i ncreasing the s imulation interval beyond [0 , T ] requires performing a n e w backward sweep of recursion (5.13), (5.7). The model (5.20) can be used t o assess th e performance of any estimatio n fi lter designed under the assum ption that th e nominal model (3.1)–(3.2) is valid. Let G ′ t be an arbitrary time-dependent gain sequence, and l et ˆ x ′ t be the st ate estimate generated by t he recursion ˆ x ′ t +1 = A t ˆ x ′ t + G ′ t ( y t − C t ˆ x ′ t ) . (5. 22) Let e ′ t = x t − ˆ x ′ t denote the corresponding filt ering error . When the actual data is generated by the least-fa vora ble model (5.20)–(5.21), by subtracting recursion (5.22) from the first com ponent of the state d ynamics (5.20), we obtai n   e ′ t +1 e t +1   =  ˜ A t −   G ′ t 0   ˜ C t    e ′ t e t   +  ˜ B t −   G ′ t 0   ˜ D t  ǫ t . (5.23) The recursion (5.23) can be used to e valuate t he performance of filter (5.22) when the data is generated by the least-fa vorable model (5.20)–(5.21). Specifically , consid er the covariance matrix Π t = ˜ E h   e ′ t e t   h ( e ′ t ) T e T t i i . February 2, 2018 DRAFT 28 By using the dynami cs (5.23) derived u nder the ass umption t hat the data is generated by the least-fa vorable mo del, we obtain the L yapunov equation Π t +1 =  ˜ A t −   G ′ t 0   ˜ C t  Π t  ˜ A t −   G ′ t 0   ˜ C t  T +  ˜ B t −   G ′ t 0   ˜ D t  ˜ B t −   G ′ t 0   ˜ D t  T , (5.24) which can be us ed to ev aluate the p erformance of filter (5.23 ) when it is applied to the least- fa vora ble model. For t he special case where G ′ t is the Kalman gain sequence, thi s yields t he performance of the standard Kalman filter . V I . S I M U L A T I O N S T o i llustrate the behavior of the robust filtering algorithm s pecified b y (4.26)–(4.31), we consider a cons tant st ate-space model employed earlier in [11], [12]: A =   0 . 9802 0 . 0 196 0 0 . 9802   , B B T = Q =   1 . 9608 0 . 0 195 0 . 0195 1 . 9 605   C = h 1 − 1 i , D D T = 1 . The nom inal p rocess nois e B v t and measurement noise D v t are assu med t o be uncorrelated, so that B D T = 0 , and t he i nitial value of the least-fa vorable error cov ariance mat rix is selected as V 0 = I 2 . W e apply the robust filtering algo rithm over an i nterva l of length T = 2 00 for progressively tighter v alues 10 − 2 , 10 − 3 and 10 − 4 of the relativ e entropy tolerance c . The corresponding time- var ying risk-sensitivity parameters θ t = λ − 1 t obtained from (4.31) are p lotted in Fig. 1. The least-fa vorable variances (the (1,1) and (2,2) entries o f V t ) of the two states are plott ed as functions of ti me in Fig. 2 and Fig . 3. As can be seen from the plot s, alth ough the relative entropy tolerance bo unds that we consi der are small, increasing the t olerance c by a factor 10 leads to an increase of about 7dB in the stat e error variances. Next, for a tolerance c = 10 − 4 , we compare the performance of the risk-sensit iv e and K alman filters for t he nom inal model, and for the l east-fa v orable model constructed as indicated in Section V. The variance s o f the two-states for th e nom inal model are shown i n Fig. 4 and February 2, 2018 DRAFT 29 0 20 40 60 80 100 120 140 160 180 200 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1 t l o g 1 0 θ t c=10 −2 c=10 −3 c=10 −4 Fig. 1. Plot of t ime varying parameter θ t = λ − 1 t (logarithmic scale) for c = 10 − 2 , 10 − 3 and 10 − 4 . Fig. 5, respectively . Clearly , the los s of performance of the risk-sens itiv e filter com pared t o the Kalman filter is less than 1dB. On the oth er hand, as indicated in Fig. 6 and Fig. 7, when the risk-sens itiv e and Kal man filters are app lied to t he least-fav orable mod el, the Kalman filter performance is about 8dB worse than the robust filter . Note that to allow the b ackward recursion (5.13), (5.7) to reach steady state, the backward model is computed for a larger interv al, and only the fi rst 200 samples of the simulation interval are retained, si nce l ater sam ples are affec ted by transients of the least-fa vora ble model. V I I . C O N C L U S I O N In this paper , we have considered a robust state-space filtering prob lem w ith an in cremental relativ e ent ropy constraint. The problem was formulated as a dyn amic mi nimax game, and by extending results presented i n [1], it was s hown that t he min imax filt er is a risk -sensitive filter wit h a ti me varying risk -sensitive parameter . The associated least-fav orable model was constructed by performin g a backward recursion which keeps track of retroactiv e probabili ty February 2, 2018 DRAFT 30 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 35 t V 1 t ( d B ) c=10 −2 c=10 −3 c=10 −4 Fig. 2. Error v ariance of x 1 t (dB scale) for c = 10 − 2 , 10 − 3 , and 10 − 4 . changes made by th e maximi zing player . The results obtained are sim ilar in nature t o those deriv ed by Hansen and Sar gent [14 ], [15] for the mi nimax probl em (3.25) when a single relative entropy constraint is a pplied to the overall state-space model and the maximi zing agent is required to operate under com mitment.. A number of issues remain to be resolved. For t he case of a constant state-space model, it w ould be of interest to establish the con ver gence under appropriate cond itions of the robust filtering recursions and of the backwards least-fa vorable model recursion s. One also has t o wonder if the results derived here for G auss-Markov models could be extended to classes of systems, such as parti ally observed Markov chains, for which robust filtering with an overall relative entropy constraint was considered previously in [34 ]. R E F E R E N C E S [1] B. C. Levy and R. Nikoukhah, “Robust least-squares estimation w ith a relativ e entropy constraint, ” IEEE T rans. Informat. Theory , vol. 50, pp. 89–104, Jan. 2004. February 2, 2018 DRAFT 31 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 35 t V 2 t ( d B ) c=10 −2 c=10 −3 c=10 −4 Fig. 3. Error v ariance of x 2 t (dB scale) for c = 10 − 2 , 10 − 3 , and 10 − 4 . [2] P . J. Huber , Robust Statistics . New Y ork: J. Wile y & Sons, 1981. [3] S. A. Kassam and T . L. Lim, “Robust Wiener filt ers, ” J. Fr anklin Instit ute , vol. 304, pp. 171–185, 1977. [4] H. V . Poor , “On robust W iener fi ltering, ” IEEE Tr ans. Automat. Contr ol , pp. 531–536 , June 1980 . [5] S. A. Kassam and H. V . P oor , “Robust techniques for signal processing: a survey , ” Pr oc. IEEE , vo l. 73, pp. 433–481, Mar . 1985. [6] J. L. Speyer , J. Deyst, and D. H. Jacobson, “Optimization of stochastic linear systems with additiv e measurement and process noise using expon ential performance criteria, ” IEEE T rans. Automat. Control , vol. 19, pp. 358–36 6, 1974. [7] P . W hittle, R isk-sensitive Optimal Contr ol . Chichester, England: J. Wiley , 1980. [8] K. N. Nagpal and P . P . Khargonekar , “Filtering and smoothing in an h ∞ setting, ” IE EE Tr ans. Automat. Contro l , vo l. 36, pp. 152–1 66, Feb. 1991. [9] D. Mustafa and K. Glover , Minimum E ntr opy H ∞ Contr ol . N o. 146 in L ecture Notes in Control and Information Sciences, Berlin: Springer V erlag, 1990. [10] B . Hassibi, A. H. Sayed, and T . Kailath, Indefinite-Quadratic Estimation and Contr ol– A Unified Appro ach to H 2 and H ∞ Theories . Philadelphia: Soc. Indust. Appl. Math., 199 9. [11] I . R. Petersen and A. V . Savkin, Robust Kalman Filtering for Signals and Systems wit h L ar ge Uncertainties . Boston, MA: Birkh ¨ auser , 1999. [12] A . H. Sayed, “ A framew ork for state-space estimation with uncertain models, ” IEEE T ran s. Automat. Contr ol , vol. 46, pp. 998–1 013, July 2001. February 2, 2018 DRAFT 32 0 20 40 60 80 100 120 140 160 180 200 0 2 4 6 8 10 12 14 16 18 20 t V 1 t ( d B ) K a l m a n R o b u s t Fig. 4. Er ror v ariance of x 1 t (dB scale) when t he risk-sensiti ve fi lter with c = 10 − 4 and the Kalman fil ter are applied to the nominal model. [13] L . El Ghaoui and G. Calafiore, “Robust filteri ng for discrete-time systems with bounded noise and parametric uncertainty , ” IEEE T rans. Automat. Contr ol , v ol. 46, pp. 1084–1089, July 2001. [14] L . P . Hansen and T . J. Sargent, Robustness . Princeton, NJ: Pr inceton Univ ersity Press, 2008 . [15] L . P . Hansen and T . J. Sargent, “Robust estimation and control under commitment, ” J . Economic Theory , vol. 124, pp. 258– 301, 2005 . [16] L . P . Hansen and T . J. Sargent, “Recursi ve robu st estimation and control without commitment, ” J. Economic Theory , vol. 136, pp. 1–27, 2007. [17] R . K. Boel, M. R. James, and I. R. Petersen, “Robustess and risk-sensitive filtering, ” IEEE T rans. Automat. Contr ol , vol. 47, pp. 451–4 61, Mar . 2002. [18] M.-G . Y oon, V . A. Ugrinovskii, and I. R. P etersen, “Robust fi nite horizon minimax filteri ng for discrete-time stochastic uncertain systems, ” System & Contr ol L et. , vol. 52, pp. 99–112, 2004. [19] G . J. McLachlan and Krishnan, T he EM Algorithm and Extensions . Ne w Y ork: Wiley , 1997. [20] N . N. Chentsov , Statistical Decision Rules and Optimal Infer ence , vol. 53 of T ranslations of Mathematical Monog raphs . Providence , RI: American Math. S ociety , 1980. [21] S .-I. A mari and H. Nagaoka, Method s of Information Geometry . Providence, RI: American Mathematical S ociety , 2000. [22] M. Basseville, “Information: Entropies, div ergence s et moyen nes, ” T ech. Rep. 1020, Institut de Recherche en Informatique et Syst ` emes Al ´ eatoires, Rennes, France, May 1996 . February 2, 2018 DRAFT 33 0 20 40 60 80 100 120 140 160 180 200 0 2 4 6 8 10 12 14 16 18 20 t V 2 t ( d B ) K a l m a n R o b u s t Fig. 5. Er ror v ariance of x 2 t (dB scale) when t he risk-sensiti ve fi lter with c = 10 − 4 and the Kalman fil ter are applied to the nominal model. [23] J. R. Magnus and H. Neudecker , Matrix Differ ential Calculus with Applications i n Statistics and E conometrics . Chichester, England: J. Wiley & Sons, 1988. [24] Y . Socratous, F . Rezaei, and C. D. Charalambous, “Nonlinear estimation for a class of systems, ” IEEE T rans. Informat. Theory , vol. 55, pp. 1930–1938, Apr . 2009. [25] Y . Guo and B. C. Levy , “Robust MSE equalizer design for MIMO communication systems in the presence of model uncertainties, ” IEEE T rans. Sig . Pr oc. , vol. 54, pp. 1840–1852 , May 2006. [26] B . C. Levy , A. B en veniste, and R. Nikoukhah, “High-leve l primitives for recursiv e maximum l ikelihoo d estimation, ” IEEE T ran s. Automat. Contro l , vol. 41, pp. 1125– 1145, Aug. 1996. [27] R . A. Horn and C. R. Johnson, Matrix Analysis . New Y ork: Cambridge Univ ersity Press, 1985. [28] L . P . Hansen and T . J. Sargent, “Fragile beliefs and the price of uncertainty , ” Q uantitative economics , vol. 1, pp. 129–16 2, 2010. [29] J. P . Aubin and I. Ekland, Applied Nonlinear Analysis . New Y ork: J. Wiley , 1984. [30] D . P . Bert sekas, A. Nedic, and A. E. Ozdaglar , Con vex Analysis and Optimization . Belmont, Mass: Athena Scientific, 2003. [31] S . Kullback, Information Theory and Statistics . New Y ork: J. Wiley & S ons, 1959. Reprinted by Dove r Publ., Mineola, NY , 1968. [32] J. L. Speyer and W . H. Chung, Stochastic Pr ocesses, Estimation, and Contr ol . Philadelphia, P A: Soc. Indust. Applied February 2, 2018 DRAFT 34 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 t V 1 t ( d B ) R o b u s t K a l m a n Fig. 6. Er ror v ariance of x 1 t (dB scale) when t he risk-sensiti ve fi lter with c = 10 − 4 and the Kalman fil ter are applied to the least-fav orable model. Math., 2008. [33] A . J. Laub, Matrix Anaysis for Scientists and Engineers . P hiladelphia, P A: Soc. I ndust. Applied Math., 2005. [34] L . Xie, V . A. Ugrinov skii, and I. R. Petersen, “Finite horizon robust state estimation for uncertain finite-alphabet hidden Marko v models with conditional relative entropy constraints, ” SIAM J . Contr ol Optim. , vol. 47, no. 1, pp. 476–508, 2008. February 2, 2018 DRAFT 35 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 t V 2 t ( d B ) R o b u s t K a l m a n Fig. 7. Er ror v ariance of x 2 t (dB scale) when t he risk-sensiti ve fi lter with c = 10 − 4 and the Kalman fil ter are applied to the least-fav orable model. February 2, 2018 DRAFT