On Real-Time Communication Systems with Noisy Feedback

On R eal- T ime Communication Sy stems with N oisy F eedbac k A ditya Maha jan and Demosthenis T eneketzis Department of EECS , Univ . of Mic higan, Ann Arbor , MI – 48109. { adity am,teneket } @eecs.umich.edu Abstract — W e consider a real- -time communication sy stem with noisy f eedback consisting of a Mark o v source, a forw ard and a bac kward discrete memoryless channels, and a receiv er with finite memory . The objectiv e is to design an optimal com- munication strategy (that is, encoding, decoding, and memory update strategies) to minimize the total expected distortion ov er a finite horizon. W e present a sequential decomposition f or the problem, which results in a set of nested optimality equations to determine optimal communication strategies. This pro vides a systematic methodology to determine globall y optimal joint source- -channel encoding and decoding strategies for real- -time communication systems with noisy feedbac k. I. Pr oblem Formulation Consider a real- -time communication system with noisy f eedback as sho wn in Figure 1. This sys tem consists of a source, a real- -time encoder, a noisy forward c hannel, a noisy backw ard channel, and a real- -time decoder with finite mem- ory . The communication sys tem operates in discrete time f or a time hor izon T . Source Encoder Forward Channel Backward Channel Decoder X t Z t Y t ˆ X t ˜ Y t N t ˜ N t Fig 1. A real- -time communication system with noisy feedback At each stag e t , the source produces an output X t taking values in a finite alphabet X . W e assume that the output sequence { X t , t = 1 , . . . , T } f or ms a first- -order Mark ov chain with initial distribution P X 1 and matrix of transition probabilities P X t +1 | X t . The communication system consis ts of tw o channels: a f or - ward channel and a backw ard channel. W e assume that both channels are independent DMC (discrete memor yless chan- nels). The f or ward channel is a |Z | - -input |Y | - -output DMC , while the backw ard channel is a |Y | - -input | ˜ Y | - -output DMC . These channels can be described as Y t = h ( Z t , N t ) , t = 1 , . . . , T , (1a) ˜ Y t − 1 = ˜ h ( Y t − 1 , ˜ N t − 1 ) , t = 2 , . . . , T , (1b) where h ( · ) and ˜ h ( · ) denote the f or ward and backw ard chan- nels at time t , respectiv ely ; Z t and Y t − 1 are the inputs to the f or ward and the backw ard channels at time t , respectivel y; Y t and ˜ Y t − 1 are the outputs of the f or ward and the back - ward channels at time t , respectivel y; and N t and ˜ N t − 1 are the channel noise in the f or ward and the backw ard channels at time t , respectiv el y . The sequential order in which these sys tem variables are generated is shown in Figure 2. The variables Z t , Y t , ˜ Y t , N t , and ˜ N t take values in finite alpha- bets Z , Y , ˜ Y , N , and ˜ N , respectivel y . W e assume that { N t , t = 1 , . . . , T } and { ˜ N t , t = 1 , . . . , T } are sequences of i.i.d. random variables with PMF (probability mass function) P N and P ˜ N , respectivel y . These sequences are independent of each other and are also independent of the source output { X t , t = 1 , . . . , T } . At each stage t , the encoder obser v es the output X t of the source and the output ˜ Y t − 1 of the backw ard channel. It gen- erates an encoded symbol Z t using all its past obser vations using an encoding rule c t , i.e., Z 1 = c 1 ( X 1 ) , (2a) Z t = c t ( X t , Z t − 1 , ˜ Y t − 1 ) , t = 2 , . . . , T , (2b) where X t is a short hand notation f or the sequence X 1 , . . . , X t and Z t − 1 and ˜ Y t − 1 are similarl y defined. This encoded symbol is transmitted ov er the f or ward chan- nel (1a) producing a channel output Y t . At the next time instant, Y t gets transmitted ov er the backw ard channel (1b). The receiver consists of a decoder and a memor y . The content of the memor y is denoted by M t and takes values in a finite alphabet M . At each stage t , the receiv er generates an estimate ˆ X t of the source taking values in a finite alphabet ˆ X using a decoding rule g t , i.e., ˆ X 1 = g 1 ( Y 1 ) , (3a) ˆ X t = g t ( Y t , M t − 1 ) , t = 2 , . . . , T , (3b) and updates the content of its memory using a memory up- date r ule l t , i.e., M 1 = l 1 ( Y 1 ) , (4a) M t = l t ( Y t , M t − 1 ) , t = 2 , . . . , T . (4b) The perf or mance of the sy stem is q uantified by a unif ormly bounded distor tion function ρ : X × ˆ X → [0 , ρ max ] , where ρ max < ∞ . The distortion at time t is giv en by ρ ( X t , ˆ X t ) . The collection C : = ( c 1 , . . . , c T ) of encoding r ules for the entire horizon is called an encoding strat egy . Similarl y , the collection G : = ( g 1 , . . . , g T ) of decoding rules is called a decoding str ategy and the collection L : = ( l 1 , . . . , l T ) of memory update rules is called a memor y update strat egy . Further, the choice ( C, G, L ) of communication r ules f or the entire hor izon is called a communication strat egy or a design . The per f or mance of a communication strategy is quantified b y the expected total distor tion under that strategy and is giv en by J T ( C, G, L ) : = E ( T X t =1 ρ ( X t , ˆ X t )      C, G, L ) . (5) W e are interested in the f ollowing optimization problem: Problem 1: Assume that the encoder and the receiv er kno w the source statis tics P X 1 and P X t +1 | X t , t = 1 , . . . , T , the f or w ard and backw ard channel functions h , ˜ h , the for - ward and the backw ard channel noise statis tics P N and P ˜ N , the distor tion functions ρ and the time hor izon T . Choose a communication strategy ( C ∗ , G ∗ , L ∗ ) that is optimal with respect to per f or mance cr iter ion of (5), i.e., J T ( C ∗ , G ∗ , L ∗ ) = J ∗ T : = min C ∈ C T G ∈ G T L ∈ L T J T ( C, G, L ) , (6) where C T : = C 1 ×· · ·× C T , C t is the f amily of functions from X t × ˜ Y t − 1 × ˜ Z t − 1 to Z , G T : = G × . . . × G ( T - -times), G is the f amily of functions from Y × M to ˆ X , L T : = L × . . . L ( T - -times), and L is the famil y of functions from Y × M to M . The design of an optimal communication strategy for a real- -time communication sy stem with noisy f eedback has not been considered in the literature so far . The w ork on real- - time communication assumes either a noiseless f or ward chan- nel [1]–[4], or no feedbac k [5]–[8], or it assumes noiseless f eedback [9]. The w ork on noisy f eedback [10]–[13] does not assume a real- -time constraint on inf or mation transmission. The ke y contribution of this paper is the presentation of a sys tematic methodology f or the design of globally optimal strategies f or real- -time communication with noisy f eedback. W e treat the design of an optimal communication strategy as a decentralized multi- -ag ent sequential optimization problem. W e sho w that an optimal communication strategy can be ob- tained b y proceeding backw ards in time and sol ving a set of nested optimality equations. The rest of this paper is org anized as f ollow s. W e present some preliminary results in Section II. Then we present qual- itativ e proper ties of optimal encoding and decoding strate- gies in Section III and descr ibe an algor ithm f or determining globally optimal communication strategies in Section IV. W e conclude in Section V. II. Some Preliminaries A. Problem Classification Problem 1 is a sequential stochas tic optimization problem as defined in [14]. T o understand the sequential nature of the problem, we need to refine the notion of time. W e call each step of the sys tem a stag e . For each stag e, w e consider three time instances: 1 t + , t + 1 / 2 , and ( t + 1) − . For the ease of notation, we will denote these time instances by 1 t , 2 t , and 3 t , respectiv ely . Assume that the system has three “ag ents”, the encoder (agent 1), the decoder (agent 2), and the memory update (agent 3), which act sequentially at 1 t , 2 t , and 3 t , respectiv ely . The order in which the random v ar iables are generated in the system is illustrated in Figure 2. Since the ordering of the decision mak ers can be done independentl y of the realization of the system variables, Proper ty C of [15] is trivially satisfied and hence Problem 1 is a causal sequential stoc hastic optimization problem as defined in [14]. Problem 1 is a multi- -agent problem where all agents ha ve the same objectiv e given b y (6). Such problems are called team problems [16], and are fur ther classified as static teams or dynamic teams on the basis of their inf or mation struc- ture. In static teams, an agent ’ s information is a function of pr imitiv e random variables only , while in dynamic teams, in general, an agent ’ s inf or mation depends on the functional f or m of the decision r ules of other agents. In Problem 1 the receiv er’ s inf or mation depends on the functional form of the encoding r ule. Thus Problem 1 is a dynamic team. Dynamic teams are, in general, functional optimization problems hav - ing a comple x interdependence among the decision rules [17]. This interdependence leads to non- -con ve x (in policy space) optimization problems that are hard to sol ve. For the ease of notation, at time ins tances 1 t , 2 t , and 3 t , w e will denote the cur rent decision r ule by 1 φ t , 2 φ t , and 3 φ t and the past decision r ules by 1 φ t − 1 , 2 φ t − 1 , and 3 φ t − 1 , i.e., 1 φ t : = c t , 1 φ t − 1 : = ( c t − 1 , g t − 1 , l t − 1 ) , (7a) 2 φ t : = g t , 2 φ t − 1 : = ( c t , g t − 1 , l t − 1 ) , (7b) 3 φ t : = l t . 3 φ t − 1 : = ( c t , g t , l t − 1 ) . (7c) B. The Notion of Information W e believ e that the traditional information theoretic no- tions entropy and mutual inf or mation are asymptotic concepts which are not directl y applicable to real- -time communication problems. So, we first describe a decision theoretic notion of inf ormation. Let (Ω , F , P ) be the probability field with respect to which all primitive random variables are defined. Suppose i O t is the obser vation of agent i at time i t , and i φ t − 1 is the past decision rules of all agents. Since the prob- lem is sequential, f or any choice of i φ t − 1 , i O t is measurable with respect to F . Fur thermore, f or any choice of i φ t − 1 , let σ ( i O t ; i φ t − 1 ) denote the smallest subfield of F with respect to which i O t is measurable. Then, the information field of agent k at time i t is σ ( i O ; i φ t − 1 ) . Using this notion of in- f or mation, we define variables that represent the inf or mation field at the encoder’ s and receiver ’ s sites just before each agent acts on the system. Definition 1: Let 1 E t , 2 E t , and 3 E t denote the observ ation and 1 E t , 2 E t , and 3 E t denote the information field at the encoder’ s site at time 1 t , 2 t , and 3 t , respectivel y , i.e., 1 E t : = ( X t , Z t − 1 , ˜ Y t − 1 ) , 1 E t : = σ ( 1 E t ; 1 φ t − 1 ) , (8a) The actual values of these time instances is not impor tant; w e just need 1 three values in increasing order . Stage t t + t + 1 / 2 ( t + 1) − Actual Time 1 t 2 t 3 t Time Notation X t Z t N t Y t ˆ X t ˜ N t ˜ Y t M t System Variables c t 1 φ t g t 2 φ t l t 3 φ t Decision Rules 1 B t 2 B t 3 B t Belief of the encoder 1 A t 2 A t 3 A t Belief of the decoder 1 π t 2 π t 3 π t Information State Fig 2. Sequential order ing of different variables in the system 2 E t : = ( X t , Z t , ˜ Y t − 1 ) , 2 E t : = σ ( 2 E t ; 2 φ t − 1 ) , (8b) 3 E t : = ( X t , Z t , ˜ Y t ) , 3 E t : = σ ( 3 E t ; 3 φ t − 1 ) . (8c) Further, let 1 R t , 2 R t , and 3 R t denote the obser vation and 1 R t , 2 R t , and 3 R t denote the inf or mation field at the re- ceiv er’ s site at time 1 t , 2 t , and 3 t , respectivel y , i.e., 1 R t : = ( M t − 1 ) , 1 R t : = σ ( 1 R t ; 1 φ t − 1 ) , (9a) 2 R t : = ( Y t , M t − 1 ) , 2 R t : = σ ( 2 R t ; 2 φ t − 1 ) , (9b) 3 R t : = ( Y t , M t − 1 ) , 3 R t : = σ ( 3 R t ; 3 φ t − 1 ) . (9c) Problem 1 is a decentralized problem because, at an y time t , the inf or mation fields at the encoder’ s site and the re- ceiv er’ s site are non- -comparable, that is, 1 E t 6⊆ 1 R t and 1 E t 6⊇ 1 R t ; and similar relations hold betw een 2 E t and 2 R t , and between 3 E t and 3 R t . Thus, at no time dur ing the e vo- lution of the sys tem does the encoder “know” ex actly what is “kno wn ” to the receiver and vice- -versa. Hence the inf or - mation in the syst em is decentralized . Notice that the in- f or mation fields at the encoder and the receiv er are coupled through decision r ules. 1 E 1 and 1 R 1 are kno wn before the sys tem star ts operating. The choice of 1 φ 1 determines 2 E 1 and 1 R 1 , the choice of 2 φ 1 determines 3 E 1 and 3 R 1 , and so on. Thus, 1 E t and 1 R t are determined completel y b y 1 E 1 , 1 φ t − 1 and 1 R 1 , 1 φ t − 1 , respectiv ely . Thus, the inf or mation 1 E t and 1 R t is coupled through the past decision r ules 1 φ t − 1 . Hence, Problem 1 has a non- -classical inf or mation structure (see [18, 19]). C. Ag ent’ s Beliefs and their Evolution Due to decentralization of inf or mation, it is impor tant to characterize what one agent thinks about the other agent ’ s observation, i.e., what the encoder “thinks ” that the receiv er “sees ” and what the receiv er “thinks ” that the encoder “sees”. This is captured b y the encoder’ s belief about the observa- tions of the receiver , and the receiver ’ s belief about the ob- servations of the encoder at time instances 1 t , 2 t , and 3 t . These beliefs are giv en below . Definition 2: Let 1 B t , 2 B t , and 3 B t denote the encoder’ s belief about the receiv er’ s obser v ation at 1 t , 2 t , and 3 t , re- spectiv ely , i.e., for i = 1 , 2 , 3 , i B t ( i r ) : = Pr  i R t = i r   i E t  . (10) Definition 3: Let 1 A t , 2 A t , and 3 A t denote the receiver ’s belief about the encoder’ s observation at 1 t , 2 t , and 3 t , re- spectiv ely , i.e., for i = 1 , 2 , 3 , i A t ( i e ) : = Pr  i E t = i e   i R t  . (11a) Further, let ˆ A t denote the receiv er’ s belief about the source output at time instance 2 t , i.e., ˆ A t ( x t ) : = Pr  X t = x t   2 R t  . (11b) The sequential ordering of these belief s is shown in F ig- ure 2. For an y par ticular realization 1 e t of 1 E t , and an y arbitrary (but fix ed) choice of 1 φ t − 1 , the realization 1 b t of 1 B t is a PMF on M . If E t is a random vector , then 1 B t is a random v ector belonging to P ( M ) , the space of PMF s on M . Similar interpretations hold f or 2 B t , 3 B t , 1 A t , 2 A t , and 3 A t . The time ev olution of these beliefs of the encoder and the receiv er are coupled through their decision r ules. Specifi- cally , Lemma 1: For each stage t , there e xist deterministic func- tions 1 F , 2 F , and 3 F such that 1 B t = 1 F ( 3 B t − 1 , l t − 1 ) , (12a) 2 B t = 2 F ( 1 B t , Z t ) , (12b) 3 B t = 3 F ( 2 B t , ˜ Y t ) . (12c) The functions 1 F and 2 F are linear in their first argument. Lemma 2: For each stage t , there e xist deterministic func- tions 2 K , 3 K and ˆ K suc h that 2 A t = 2 K ( 1 A t , Y t , c t ) , (13a) 3 A t = 3 K ( 2 A t ) , (13b) ˆ A t = ˆ K ( 2 A t ) . (13c) The functions 3 K and ˆ K are linear in their first argument. Further, there e xist deter ministic functions 1 K t f or each t such that 1 A t = 1 K t ( 1 A 1 , M t − 1 , c t − 1 , l t − 1 ) . (13d) Due to lack of space the complete proofs of these lemmas are omitted. Detailed proofs can be f ound in [20]. An observ ation that simplifies the global optimization prob- lem is the fact that the beliefs 1 B t and 1 A t are independent of the decoding strategy G . This is because decoding is a filtering problem that does not affect the future ev olution of the system. Bef ore looking at the global optimization problem, we first identify qualitativ e properties of optimal encoders and de- coders. III. Str uctural Resul ts In this section, we pro vide qualitativ e proper ties of opti- mal encoders (respectivel y , decoders) that are tr ue f or all arbitrary but fix ed decoding and memory update strategies (respectiv ely , encoding and memor y update strategies). A. Structur al Results of Optimal Real- -Time Encoders Theor em 1: Consider Problem 1 for any arbitrary (but fix ed) decoding and memor y update strategies, G = ( g 1 , . . . , g T ) and L = ( l 1 , . . . , l T ) , respectiv ely . Then there is no loss in optimality in restricting attention to encoding rules of the form Z t = c t ( X t , 1 B t ) , t = 2 , . . . , T . (14) Proof. W e look at the problem from the encoder’ s point of view . Note that { X t , t = 1 , . . . , T } is a Marko v process independent of the noise in the forw ard and the backw ard channel. This fact together with the results of Lemma 1 implies that Pr  X t +1 , 1 B t +1   X t , 1 B t , Z t , c t , g t , l t  = Pr ( X t +1 | X t ) Pr  1 B t +1   1 B t , Z t , l t  = Pr  X t +1 , 1 B t +1   X t , 1 B t , Z t , l t  (15) Thus { ( X t , 1 B t ) , t = 1 , . . . , T } is a controlled Mark o v process with control action Z t . Further , the e xpected con- ditional instantaneous distor tion can be written as E n ρ ( X t , ˆ X t )    3 E t o = = X y t ∈Y m t − 1 ∈M ρ  X t , g t ( y t , m t − 1 )  Pr  y t , m t − 1   3 E t  = X y t ∈Y m t − 1 ∈M ρ  X t , g t ( y t , m t − 1 )  2 F ( 1 B t , Z t ) =: 1 ρ ( X t , 1 B t , Z t , g t ) . (16) Thus, the total e xpected distortion can be wr itten as E ( T X t =1 ρ ( X t , ˆ X t )      C, G, L ) = E ( T X t =1 E n ρ ( X t , ˆ X t )    3 E t o      C, G, L ) = E ( T X t =1 1 ρ ( X t , 1 B t , Z t , g t )      C, G, L ) . (17) Hence from the encoder’ s point of vie w , w e ha ve a per - f ectly obser v ed controlled Marko v process { ( X t , 1 B t ) , t = 1 , . . . , T } with control action Z t and an instantaneous distor - tion 1 ρ ( X t , 1 B t , Z t , g t ) (recall that G is fix ed). From Marko v decision theor y [21] we kno w that there is no loss of optimal- ity in restricting attention to encoding r ules of the f or m (14).  Theorem 1 immediately implies the f ollowing: Corollary 1: The optimal per f or mance J ∗ T giv en by (6) can be deter mined by J T ( C ∗ , G ∗ , L ∗ ) = J ∗ T : = min C ∈ C T S G ∈ G T L ∈ L T J T ( C, G, L ) , (18) where C T S : = C S × · · · × C S ( T - -times), C S 2 is the space of functions from X × P ( M ) to Z , G T and L T are defined as before, and P ( M ) denotes the space of all probability measures on M . B. Structur e of Optimal Real- -Time Decoders Theor em 2: Consider Problem 1 for any arbitrary (but fix ed) encoding and memor y update strategies, C = ( c 1 , . . . , c T ) and L = ( l 1 , . . . , l T ) , respectivel y . Then there is no loss in optimality in restricting attention to decoding rules of the form ˆ X t = ˆ g ( ˆ A t ) : = arg min ˆ x ∈ ˆ X X x ∈X ρ ( x, ˆ x ) ˆ A t ( x ) . (19) Proof. W e look at the problem from the decoder’ s point of view . Since decoding is a filter ing problem, minimiz- ing the total distor tion J T ( C, G, L ) is equiv alent to min- imizing the conditional expected instantaneous distortion E n ρ ( X t , ˆ X t )    2 R t o f or each time t . This conditional ex- pected instantaneous distor tion can be written as Note that the S in C S is a shor t form of separated, and should not be 2 confused with C t defined in Problem 1. E n ρ ( X t , ˆ X t )    2 R t o = X x t ∈X ρ ( x t , ˆ X t ) Pr  x t   2 R t  = X x t ∈X ρ ( x t , ˆ X t ) ˆ A t ( x t ) and is minimized b y the decoding r ule giv en in (19).  IV. Determining Glob all y Optimal Communica tion Stra tegy A globally optimal design for Problem 1 alwa ys e xists because there are finitel y man y designs and we can alwa ys choose the one with best per f or mance. How ev er , a br ute f orce ev aluation of each design to find the optimal one is computationally impractical. So we want to determine a sys- tematic algorithm to search f or an optimal design. One such sys tematic approach, called sequential decomposition , is to sequentiall y deter mine optimal decision r ules f or all stages b y proceeding backw ards in time. This procedure simpli- fies e xponentially the comple xity of searching f or an optimal solution. The resultant “simplified” problem is still exponen- tial in comple xity , which reflects the comple xity of finding optimal strategies in decentralized sys tems. The ke y step in obtaining a sequential decomposition is to identify an information state sufficient for per f or mance evaluation (also called a sufficient statistic for contr ol ). For Problem 1 one such inf or mation state is given by the f ollow - ing unconditional probability la ws. Definition 4: Define 1 π t , 2 π t , and 3 π t as follo w s: 1 π t = Pr  X t , M t − 1 , 1 B t  , (20a) 2 π t = Pr  X t , Y t , M t − 1 , 2 B t  , (20b) 3 π t = Pr  X t , Y t , ˜ Y t , M t − 1 , 3 B t  . (20c) Let 1 Π denote the space of probability measures on X × M × P ( M ) , 2 Π denote the space of probability measures on X × Y × M × P ( Y × M ) and 3 Π denote the space of probability measures on X × Y × ˜ Y × M × P ( Y × M ) . Then, 1 π t takes values in 1 Π , 2 π t takes values in 2 Π and 3 π t takes values in 3 Π . Lemma 3: 1 π t , 2 π t , and 3 π t are inf or mation states f or the encoder , decoder , and memor y update, respectivel y , i.e., 1. there are linear transf or mations 1 Q , 2 Q , and 3 Q such that 2 π t = 1 Q ( c t ) 1 π t , (21a) 3 π t = 2 Q 2 π t , (21b) 1 π t +1 = 3 Q ( l t ) 3 π t . (21c) 2. the expected instantaneous cost can be expressed as E n ρ ( X t , ˆ X t )    c t , g t , l t − 1 o = 2 ρ ( 2 π t , g t ) (22) where 2 ρ ( · ) is a deter ministic function. Due to lack of space, the proof of this Lemma is omitted. Detailed proof can be found in [20]. Using this result, the performance cr iter ion of (5) can be rewritten as J T ( C, G, L ) = E ( T X t =1 ρ ( X t , ˆ X t )      C, G, L ) ( a ) = T X t =1 E n ρ ( X t , ˆ X t )    c t , g t , l t − 1 o ( b ) = T X t =1 2 ρ ( 2 π t , g t ) (23) where ( a ) f ollow s from the sequential ordering of sys tem variables and ( b ) f ollow s from Lemma 3. A. An Equivalent Optimization Problem Consider a centralized deter ministic optimization problem with state space alter nating between 1 Π , 2 Π , and 3 Π and action space alternating between C S , G , and L . The sy stem dynamics are giv en by (21) and at each stag e t the decision rules c t , g t and l t are determined according to meta- -rules 1 ∆ t , 2 ∆ t , and 3 ∆ t , where 1 ∆ t is a function from 1 Π to C S , 2 ∆ t is a function from 2 Π to G and 3 ∆ t is a function from 3 Π to L . Thus the system equations (21) can be wr itten as c t = 1 ∆ t ( 1 π t ) , 2 π t = 1 Q ( c t ) 1 π t , (24a) g t = 2 ∆ t ( 2 π t ) , 3 π t = 2 Q 2 π t , (24b) l t = 3 ∆ t ( 3 π t ) , 1 π t +1 = 3 Q ( l t ) 3 π t . (24c) At each stag e an instantaneous cost 2 ρ ( 2 π t , g t ) is incur red. The choice ( 1 ∆ 1 , 2 ∆ 1 , 3 ∆ 1 , . . . , , 1 ∆ T , 2 ∆ T , 3 ∆ T ) is called a meta- -design and denoted b y ∆ T . The per f or mance of a meta- -design is given b y the total cost incur red by that meta- - design, i.e., J T (∆ T ) = T X t =1 2 ρ ( 2 π t , g t ) . (25) No w consider the follo wing optimization problem: Problem 2: Consider the dynamic sy stem (24) with kno wn transf or mations 1 Q , 2 Q , and 3 Q . The initial state 1 π 1 is giv en. Determine a meta- -design ∆ T to minimize the total cost giv en by (25). Observe that f or any initial state 1 π 1 , a choice of meta- - design ∆ T determines a design ( C , G, L ) through (24). Re- lation (22) implies that the expected distortion under design ( C, G, L ) , given b y (5), is equal to the cost under meta- -de- sign ∆ T giv en b y (25). Thus, if the transf or mation 1 Q , 2 Q , and 3 Q in Problem 2 are chosen as in Lemma 3, an optimal meta- -design f or Problem 2 determines an optimal design f or Problem 1. Problem 2 is a classical deter ministic control problem and optimal meta- -designs can be deter mined as f ol- lo ws: Theor em 3: An optimal meta- -design ∆ ∗ ,T f or Problem 2, and consequentl y an optimal design ( C ∗ , G ∗ , L ∗ ) f or Prob- lem 1 can be deter mined as follo w s. For any 1 π ∈ 1 Π , 2 π ∈ 2 Π , and 3 π ∈ 3 Π , define the f ollowing functions: 1 V T +1 ( 1 π ) = 0 , (26a) and for t = 1 , . . . , T 1 V t ( 1 π ) = inf c ∈ C S 2 V t  1 Q ( c ) 1 π  , (26b) 2 V t ( 2 π ) = min g ∈ G 2 ρ ( 2 π , g ) + 3 V t ( 2 Q 2 π ) , (26c) 3 V t ( 3 π ) = min l ∈ L 1 V t +1  3 Q ( l ) 3 π  . (26d) The ar g min (or ar g inf) at eac h s tep determines the optimal meta- -design ∆ ∗ ,T . After an optimal meta- -design has been determined, an optimal design ( C ∗ , G ∗ , L ∗ ) can be deter - mined through (24). Further more, the optimal performance is given b y J ∗ T = 1 V 1 ( 1 π 1 ) . (27) Proof. This is a standard result, see [21, Chapter 2].  The abo v e nested optimality eq uations deter mine a glob- ally optimal design and the globally optimal per f or mance. Observe that the functional f or m of the optimality equations does not change with time. So, the results presented here can be easily e xtended to infinite hor izon problem with e xpected discounted distor tion or av erag e distortion per unit time cri- teria. Such an extension to infinite hor izon will result in a fix ed point equation to deter mine a time- -in variant (station- ary) meta- -design; the design at eac h stag e will be time vary- ing . Due to decentralization of information, optimal designs f or infinite hor izon are not stationary . This phenomenon also occurs in real- -time communication with no f eedback. V. Discussion and Conclusion The solution frame work presented in this paper and some of our pre vious papers [7, 8, 20] provides an alter nativ e ap- proach to real- -time communication problems. Inf ormation theoretic performance bounds or coding theoretic lo w- -com- ple xity coding schemes are not known f or noisy real- -time communication sys tems. In the absence of such results, the designer of a real- -time communication sys tem has to choose a good heur istic communication strategy and hope that it meets the per f or mance requirements. If it does not, the de- signer needs to try different communication strategies until one that meets the per f or mance requirements is f ound. In this paper we hav e presented an alter nativ e, sys tematic methodology to design an optimal communication strategy f or real- -time communication sys tems with noisy feedbac k. Instead of tr ying out heur istic strategies one b y one, optimal communication strategies can be deter mined b y solving the nested optimality eq uations of Theorem 3. Note that these are not typical dynamic prog ramming equations as each step is a functional optimization problem. Hence, although the sys tematic methodology presented here exponentiall y simpli- fies the comple xity of finding an optimal design as compared to a brute f orce approach, sol ving the resultant nested opti- mality equations is a f or midable computational task. It may be possible to e xtend the computational techniques f or sol v- ing dynamic programming equations to efficiently sol v e equa- tions of the f or m (26). The solution of (26) also deter mines the optimal per f or mance of the sys tem and can be used to chec k the deg ree of sub-optimality of heur istic designs. A ckno wledgements This research w as supported in par t by NSF Grant ccr - 0325571 and N ASA Grant nnx06ad47g . The authors are grateful to A. Anastasopoulos and S. Pradhan f or insightful discussions. They are also grateful to the anon ymous re view - ers whose comments helped in improving the presentation of this paper . References [1] H. S. Witsenhausen, “On the structure of real-time source coders,” Bell Syst em T echnical Journal , v ol. 58, no. 6, pp. 1437-1451, July- Augus t 1979. [2] N. T . Gaarder and D. Slepian, “On optimal finite-state digital transmis- sion sys tems,” IEEE T r ans. Inform. Theor y , vol. 28, no. 2, pp. 167–186, 1982. [3] T . Linder and G. Lugosi, “A zero-dela y sequential scheme for lossy coding of individual sequences,” IEEE T r ans. Inf or m. Theor y , vol. 47, no. 6, pp. 2533–2538, 2001. [4] A. Gyor gy, T . Linder, and G. Lugosi, “Efficient adaptive algor ithms and minimax bounds for zero-delay lossy source coding,” IEEE T rans. Signal Processing , vol. 52, no. 8, pp. 2337–2347, 2004. [5] D. T eneketzis, “On the structure of optimal real-time encoders and decoders in noisy communication,” IEEE T rans. Inform. Theor y , pp. 4017-4035, Sep. 2006. [6] S. Matloub and T . W eissman, “On competitv e zero-dela y joint source- channel coding,” in proceedings of the CISS Conf erence , 2004, Prince- ton, NJ. [7] A. Mahajan and D. T eneketzis, “On jointly optimal encoding, decod- ing and memory update f or noisy real-time communication sys tems,” submitted to IEEE T rans. Inf or m. Theor y , Jan. 2006. [8] ——, “Fix ed delay optimal joint source-channel coding f or finite- memory systems,” in pr oceedings of the IEEE Int er national Symposium on Information Theor y , 2006, Seattle, W A, pp. 2319-2323. [9] J. C. W alrand and P . V araiya, “Optimal causal coding—decoding prob- lems,” IEEE T rans. Inform. Theory , vol. 29, no. 6, pp. 814-820, No v . 1983. [10] R. L. Kashy ap, “Feedback coding sc hemes for an additiv e noise channel with a noisy feedbac k link,” IEEE T rans. Inform. Theor y , vol. IT -14, no. 3, pp. 1355-1387, 1968. [11] S. S. Lav enberg, “Feedbac k communication using or thogonal signals,” IEEE T r ans. Inform. Theor y , vol. IT -15, pp. 478–483, 1969. [12] S. Draper and A. Sahai, “Noisy f eedback improv es communication reliability ,” in Proceedings of the IEEE International Symposium on Information Theory , 2006, Seattle, W A, pp. 69–73. [13] Y .-H. Kim, A. Lapidoth, and T . W eissman, “The Gaussian channel with noisy f eedback,” in Proceedings of the IEEE International Symposium on Information Theor y , 2007, Nice, France. [14] H. S. W itsenhausen, “A standard form f or seq uential stochas tic control,” Mathematical Syst ems Theory , vol. 7, no. 1, pp. 5-11, 1973. [15] ——, “On inf or mation structures, feedbac k and causality ,” SIAM Jour - nal of Contr ol , v ol. 9, no. 2, pp. 149–160, Ma y 1971. [16] J. Marschak and R. Radner, Economic Theor y of T eams , Y ale U niversity Press, New Hav en, 1972. [17] Y .-C. Ho, “T eam decision theor y and information structures,” Proc. IEEE , vol. 68, no. 6, pp. 644–654, 1980. [18] H. S. Witsenhausen, “Separation of estimation and control for discrete time systems,” Proc. IEEE , v ol. 59, no. 11, pp. 1557–1566, No v . 1971. [19] ——, “The intr insic model for stochastic control: Some open prob- lems,” in Lecture N otes in Economics and Mathematical Sy stems , 1975, Springer V er lag, pp. 322–335. [20] A. Mahajan and D. T eneketzis, “On the design of globall y optimal com- munication strategies f or real-time communcation systems with noisy f eedback,” submitted to IEEE J. Select. Ar eas Commun. , Mar. 2007. [21] P . R. Kumar and P . V araiya, Stoc hastic Sys tems: Estimation Identifica- tion and Adaptiv e Control , Prentice Hall, 1986.