A New Upper Bound on the Capacity of a Class of Primitive Relay Channels

A Ne w Upper Bound on the Capa city of a Class of Primiti v e Relay Channels Ravi T andon Sennur Uluk us Department of Electrical an d Compu ter Engineering Univ ersity of Maryland , College Park, MD 2 0742 ravit@umd.edu uluku s@umd.ed u Abstract — W e obt ain a new upper b ound on the capacity of a class of d iscrete memor yless relay channels. For this class of relay channels, the r elay observes an i .i.d. sequence T , which is independ ent of the channel input X . The channel is described by a set of probability transition functions p ( y | x, t ) f or all ( x, t, y ) ∈ X × T × Y . Furthermore, a noiseless link of finite capacity R 0 exists from the relay to the receiver . Al though the capacity fo r these channels is not known in general, the capacity of a subclass of th ese channels, namely when T = g ( X , Y ) , fo r some d eterministic function g , was obtained in [1] and it was shown to be equ al to the cut-set b ound. Another instance where the capacity was obtained was in [2], where the chann el output Y can be writ ten as Y = X ⊕ Z , where ⊕ denotes modulo- m addition, Z is indepen dent of X , |X | = |Y | = m , and T is some stochastic function of Z . The compress-and-forward (CAF) achiev ability scheme [3] was shown to b e capacity achieving in both cases. Using our up per bound we recov er the capacity results of [1] and [2]. W e also obtain the capacity of a class of ch annels which d oes n ot f all into either of the classes studied in [1] and [2]. F or thi s class of channels, CAF scheme is sh own to be opt imal but capacity is strictly less than the cut-set bound fo r certain values of R 0 . W e further illu strate the usefulness of our boun d by evaluating it f or a p articular relay channel with binary multip licative states and b inary additive noise f or which the channel is given as Y = T X + N . W e show t hat our upper bound is strictl y better than the cut -set upper bound for certain values of R 0 but it lies strictly above the rates yield ed by th e CAF achievability scheme. I . I N T RO D U C T I O N The relay chan nel is one of the simplest, yet arguably among the least u nderstoo d multi-user channels in inf ormation theory . A special class of discrete memor yless r elay channe l is the primitive re lay cha nnel [ 1]. For th is class, the chan nel is defined by a chan nel input X , a channel output Y and a relay output T , and a set of probability func tions p ( y , t | x ) f or all x ∈ X . In this setting, the re lay d oes not h av e an explicit coded input for the ch annel. Moreover , it is a lso assumed th at there is an orthogo nal lin k of fin ite cap acity R 0 , f rom th e r elay to th e receiver . Zhang [4] consid ered this r elay channel an d o btained a partial converse for a degraded ca se. For a co mpreh ensiv e survey on related work on primitive relay channels, see [5]. Recently , Kim [ 1] established the capacity of a class of semi-determin istic primitive relay channels, for wh ich the relay o utput T can be expr essed as a d eterministic fun ction This work was supported by NSF Grants CCF 04 - 47613 , CCF 05 - 14846 , CNS 07 - 16311 and CCF 07-29127. of the ch annel in put X and the ch annel o utput Y , i.e., T = g ( Y , X ) . The cut-set up per b ound [6] was shown to be the capacity thro ugh an algebraic red uction of the compr ess-and- forward (CAF) achiev able rate [ 3] to the cut-set up per b ound . This w as th e first instanc e wher e the CAF achie vability scheme was shown to be capa city achieving for any relay ch annel. In th is pap er , we consider a subclass of the primitiv e relay channel. In this subclass, the r elay ob serves an i.i.d . sequence T which is inde penden t of the chan nel in put X and the chan nel output Y is given by the set o f p robability transition functions p ( y | x, t ) for all ( x, t, y ) ∈ X × T × Y . A lternatively , this chan- nel can b e interpreted as a state dependen t discrete memoryless channel with rate- limited state infor mation a vailable at the receiver ( Figure 1 ) . T his chan nel was also studied in [7] with various modification s regarding the rate-limited knowledge of the ch annel state T at the tran smitter and the receiver . A CAF ach iev ability sch eme for this state depend ent chan nel was given b y Ahlswede an d H an in [8] and it was co njectured to be the capacity for this class of chann els. In fact, the same achiev able rates for th is chan nel wer e o btained in [7 ] and can also be obtained via The orem 6 of [3] . It follows from the r esult of [1] that this conjectu re is true for the subclass when the state T can be expressed as a deter ministic functio n of X and Y , i. e., T = g ( X , Y ) . An example of such a chan nel is the case w hen X , T and Y are all binary , T ∼ Ber ( δ ) and in depen dent of X , an d the ch annel is given by Y = X ⊕ T , where ⊕ den otes modulo - 2 ad dition. Note that, in this c ase, T is a deter ministic function of X and Y , since T = X ⊕ Y . A capacity resu lt following up on the aforem entioned modulo- additive n oise channel was obtained in [2 ], wh ere it was assumed that the receiver ob serves Y = X ⊕ Z and the relay o bserves a n oisy version of th e forward n oise, i.e., T = Z ⊕ ˜ Z . Clearly , if ˜ Z = 0 , then this ch annel redu ces to the class studied in [1]. Howe ver, when ˜ Z 6 = 0 , T cann ot b e written as a deterministic function of X a nd Y , an d this mod ulo-add itiv e class lies outside o f the class of channels co nsidered in [1] . By proving a c onv erse, it was shown in [2] that CAF scheme is capacity achieving f or this mod ulo-ad ditiv e case. The remarkable fact was that the c apacity was shown to be strictly less than the cut-set upper bound for certain values of R 0 . Howe ver, it is worth notin g that the conv erse proved in [ 2] relied heavily on the mo dulo-ad ditive na ture of the fo rward chann el. In this paper, we obtain a new up per bou nd on the capacity of the state- depend ent discrete m emoryle ss channel, where the p ( y | x, t ) X T Y R 0 T ransmitter Receiver Channel state/Rela y Figure 1 : Chann el with rate-limited state informatio n. states are i.i. d. and the state in formatio n is a v ailable to the receiver throu gh a noiseless link of finite capa city R 0 . Our upper b ound serves a dual pu rpose. Firstly , using our u pper bound , we recover the capac ity results obtained in [1] for the case where T = g ( X , Y ) an d the cap acity result ob tained in [2] f or th e modulo-ad ditiv e noise case. Secondly , w e c onfirm the validity of the con jecture due to Ahlswede-Han [8] for another class of channels which d oes not fall in to any of the cases co nsidered in [1] an d [2 ]. T o f urther illustrate the app lication of ou r upper b ound , we consider a channel wh ere X , T , N are bin ary and Y is tern ary and the cha nnel is given b y Y = T X + N , i. e., when the state sequence is binary and multiplicative and there is additi ve binary noise at th e receiver . This chann el ca n be inter preted as the discrete a nalogue of a fast fading ch annel with fade informa tion a vailable in a rate-limited fashion at the re ceiv er . This channel does not fall into a ny class for which cap acity is known. W e evaluate o ur upper bound for this chan nel and show tha t it is strictly less than the cut-set b ound for certain values of R 0 although our up per bou nd is strictly larger th an the rates yielded by the CAF schem e. I I . R E L A Y C H A N N E L M O D E L W e co nsider a re lay c hannel with finite inp ut a lphabet X , finite output alphabet Y an d finite r elay output alphabet T . Moreover , the r elay observes an i.i.d. state sequence T n ∈ T n with some giv en probab ility d istribution p ( t ) . Th e relay chan- nel is described by the set of tran sition prob abilities p ( y | x, t ) which are defined fo r all ( x, t, y ) ∈ X × T × Y . Furthermor e, there is a finite-cap acity noiseless link o f capacity R 0 from th e relay to the receiver . This relay channel can also be thought of as a state- depend ent single- user chann el with rate-limited state inf ormation available at the receiver ( see Figu re 1 ). An ( n, M , P e ) co de fo r this relay chan nel consists of the set o f integers M = { 1 , 2 , . . . , M } and th e following: f t : M → X n f r : T n → { 1 , 2 , . . . , L } φ : Y n × { 1 , 2 , . . . , L } → M (1) where f t is the transmitter encoding fu nction, f r is the relay encodin g func tion and g is t he decoding fu nction. Furth ermore, as the relay to receiv er link is of limited capacity R 0 , we hav e L ≤ 2 nR 0 (2) For a distribution p ( w ) on M , the joint pro bability distrib ution on M × X n × T n × Y n is given as p ( w, x n , t n , y n ) = p ( w ) p ( x n | w ) n Y i =1 p ( t i ) n Y i =1 p ( y i | x i , t i ) (3) For a un iform distribution p ( w ) on M , the a verage pro bability of error is giv en a s, P e = Pr ( φ ( Y n , f r ( T n )) 6 = W ) . A rate R is ac hiev able if f or a ny ǫ > 0 and all n sufficiently large, ther e exists an ( n, M , P e ) co de such th at P e ≤ ǫ and M ≥ 2 nR . The cap acity o f the relay channel is the supremum of the set of all achiev able rates. I I I . A N E W U P P E R B O U N D O N T H E C A PAC I T Y W e will denote by U n as the o utput of the finite capacity link R 0 , i.e. , U n = f r ( T n ) . W e will now obtain an upp er bound on the rate as follows, nR = H ( W ) (4) = I ( W ; Y n , U n ) + H ( W | Y n , U n ) (5) ≤ I ( W ; Y n , U n ) + nǫ n (6) ≤ I ( X n ; Y n , U n ) + nǫ n (7) = I ( X n ; Y n | U n ) + nǫ n (8) = n X i =1 I ( X n ; Y i | U n , Y i − 1 ) + nǫ n (9) = n X i =1 h H ( Y i | U n , Y i − 1 ) − H ( Y i | U n , Y i − 1 , X n ) i + nǫ n (10) ≤ n X i =1 h H ( Y i ) − H ( Y i | U n , Y i − 1 , X n ) i + nǫ n (11) ≤ n X i =1 h H ( Y i ) − H ( Y i | U n , T i − 1 , Y i − 1 , X n ) i + nǫ n (12) = n X i =1 h H ( Y i ) − H ( Y i | U n , T i − 1 , X n ) i + nǫ n (13) = n X i =1 h H ( Y i ) − H ( Y i | U n , T i − 1 , X i ) i + nǫ n (14) = n X i =1 I ( X i , U n , T i − 1 ; Y i ) + nǫ n (15) = n X i =1 I ( X i , V i ; Y i ) + nǫ n (16) = nI ( X , V ; Y ) + nǫ n (17) where (6) follows by Fano’ s ineq uality [6], (7) fo llows from the d ata processing in equality , (8) fo llows from th e fact that X n is indepen dent of T n and is hence ind epend ent of U n , (11) follows f rom the fact that conditionin g red uces entropy and hence we up per b ound by drop ping ( U n , Y i − 1 ) from the first term. Ne xt, (12) follo ws b y adding T i − 1 in the conditional entropy in the second term and o btaining an upper bou nd, (13) follows from the mem oryless pro perty of the channel, i.e., giv en ( X i − 1 , T i − 1 ) , the chann el output Y i − 1 is indepen dent of ev erything else and (14) f ollows fr om the following Markov chain, X n \ X i → ( X i , U n , T i − 1 ) → Y i . T he proo f of th is Markov ch ain is gi ven at the beginnin g of next pa ge. Finally , (16) follows by defining V i = ( U n , T i − 1 ) , an d we introd uce a rand om variable Q , un iform on { 1 , 2 , . . . , n } to define X = ( X i , Q ) , Y = ( Y i , Q ) and V = ( V i , Q ) to arri ve at (17). W e ob tain the Markov chain by showing the following, Pr ( Y i , X − i | X i , U n , T i − 1 ) = Pr ( Y i , X − i , X i , U n , T i − 1 ) Pr ( X i , U n , T i − 1 ) (18) = P t i P ( t i ) Pr ( Y i , X − i , X i , U n , T i − 1 | t i ) Pr ( X i , U n , T i − 1 ) (19) = P t i P ( t i ) Pr ( X i , U n , T i − 1 | t i ) Pr ( Y i , X − i | X i , t i , U n , T i − 1 ) Pr ( X i , U n , T i − 1 ) (20) = P t i P ( t i ) Pr ( X i , U n , T i − 1 | t i ) Pr ( X − i | X i , t i , U n , T i − 1 ) Pr ( Y i | X i , t i , U n , T i − 1 , X − i ) Pr ( X i , U n , T i − 1 ) (21) = P t i P ( t i ) Pr ( X i , U n , T i − 1 | t i ) Pr ( X − i | X i ) Pr ( Y i | X i , t i , U n , T i − 1 ) Pr ( X i , U n , T i − 1 ) (22) = Pr ( X − i | X i ) P t i P ( t i ) Pr ( X i , U n , T i − 1 | t i ) Pr ( Y i | X i , t i , U n , T i − 1 ) Pr ( X i , U n , T i − 1 ) (23) = Pr ( X − i | X i ) X t i P ( t i | X i , U n , T i − 1 ) Pr ( Y i | X i , U n , T i − 1 , t i ) (24) = Pr ( X − i | X i ) Pr ( Y i | X i , U n , T i − 1 ) (25) where we have defined X − i , ( X 1 , . . . , X i − 1 , X i +1 , . . . , X n ) . In addition to ( 17), we also need the following tr ivial u pper bound on the rate, nR ≤ I ( X n ; Y n , T n ) + nǫ n (26) = I ( X n ; Y n | T n ) + nǫ n (27) = n X i =1 I ( X n ; Y i | T n , Y i − 1 ) + nǫ n (28) = n X i =1 h H ( Y i | T n , Y i − 1 ) − H ( Y i | T n , Y i − 1 , X n ) i + nǫ n (29) = n X i =1 h H ( Y i | T i ) − H ( Y i | T n , Y i − 1 , X n ) i + nǫ n (30) = n X i =1 h H ( Y i | T i ) − H ( Y i | T i , X i ) i + nǫ n (31) = n X i =1 I ( X i ; Y i | T i ) + nǫ n (32) = nI ( X ; Y | T ) + nǫ n (33) where (26) follows by Fano’ s in equality , (27) fo llows be- cause X n is in depend ent of T n , (3 0) follows by dro pping ( Y i − 1 , T n \ T i ) fr om th e co nditionin g in th e first term, (31) f ollows from the memor yless prop erty of the chan nel, i.e., given ( X i , T i ) , th e ch annel ou tput Y i is ind ependen t o f ev erything else. W e now obtain a bound on the allow able d istributions of the inv olved rand om variables. Using the fact that the side informa tion is limited by the rate R 0 , we have that nR 0 ≥ I ( T n ; U n ) (34) = n X i =1 I ( T i ; U n | T i − 1 ) (35) = n X i =1 I ( T i ; U n , T i − 1 ) (36) = nI ( T ; V ) (37) where ( 36) fo llows f rom the fact tha t T i are i.i.d . Combining (17), (3 3) and (37), we h av e an upper bou nd on the cap acity of the relay chann el as U B = sup min { I ( X , V ; Y ) , I ( X ; Y | T ) } s.t. R 0 ≥ I ( T ; V ) over p ( x ) p ( t ) p ( v | t ) (38) where the supremum can be restricted over those V such that |V | ≤ |T | + 2 . I V . C O M PA R I S O N W I T H T H E C U T - S E T B O U N D The b est known upper bou nd for th e relay channel is the cut-set boun d [6] , which reduces f or the relay chan nel in consideratio n to [1 ], [5] C S = max p ( x ) min { I ( X ; Y ) + R 0 , I ( X ; Y | T ) } (39) On compar ing with the cut- set bound , it can be o bserved that our boun d dif fers from the cut-set boun d in the multiple access cut. W e will show next that our upper bo und is in general smaller than the cut-set boun d. W e start by upp er boun ding the expression I ( X, V ; Y ) as follows, I ( X , V ; Y ) = I ( X ; Y ) + I ( V ; Y | X ) (40) = I ( X ; Y ) + H ( V | X ) − H ( V | Y , X ) (41) = I ( X ; Y ) + H ( V ) − H ( V | Y , X ) (42) ≤ I ( X ; Y ) + H ( V ) − H ( V | T , Y , X ) (43) = I ( X ; Y ) + H ( V ) − H ( V | T ) (44) = I ( X ; Y ) + I ( T ; V ) (45) ≤ I ( X ; Y ) + R 0 (46) where (42) follows fro m the fact that V is indepen dent of X , (43) follows from th e fact th at cond itioning reduces entro py , (44) follows from the Markov chain ( X , Y ) → T → V and (45) fo llows by u sing the fact that I ( T ; V ) ≤ R 0 . Using (46) and ( 33), we have the following U B ≤ max p ( x ) min { I ( X ; Y ) + R 0 , I ( X ; Y | T ) } (47) Thus, o ur up per b ound is in g eneral sma ller th an the cut-set bound g iv en in ( 39). It was shown in [1 ] that the cut-set bo und is tig ht f or the case when T = g ( X, Y ) a nd is ach iev ed by the CAF achiev ability scheme. Note the fact that for this special subclass, the inequality in (4 3) is in fact an equality and our bound exactly eq uals the cut-set b ound . V . R E C OV E R I N G T H E C A PAC I T Y O F M O D U L O - A D D I T I V E R E L A Y C H A N N E L A specific mod ulo-add itiv e relay channel was considered in [2] f or wh ich th e chan nel is giv en as, Y = X ⊕ Z (48) T = Z ⊕ ˜ Z (49) where X , Y , T , Z and ˜ Z are all bin ary and Z ∼ Ber ( δ ) , ˜ Z ∼ Ber ( ˜ δ ) . Clearly this cha nnel does not fall into the class of channels studied in [1 ], wh ere T can be written as a deterministic function of X and Y . It was shown that the capacity of this channel is giv en by [2, Theo rem 1 ] C = max p ( v | t ): I ( T ; V ) ≤ R 0 1 − H ( Z | V ) (50) W e will sho w that o ur bou nd is equal to the capacity fo r this class of chann els. First, note that I ( X , V ; Y ) = H ( Y ) − H ( Y | X , V ) (51) = H ( Y ) − H ( Z | V ) (52) ≤ 1 − H ( Z | V ) (53) where (53) follows by the fact th at the en tropy o f a binary random variable is upp er bounded by 1 . Next, consider the other cut, I ( X ; Y | T ) = H ( Y | T ) − H ( Y | X , T ) (54) = H ( Y | T ) − H ( Z | T ) (55) ≤ 1 − H ( Z | T ) (56) W e n ote that (5 3) and ( 56) a re achiev ed with equality for a unifor m X . Mo reover , fr om ( 53) a nd (56), it c an be observed that the bound I ( X ; Y | T ) is redu ndant since V → T → Z implies H ( Z | T ) ≤ H ( Z | V ) . Hence, our upper b ound reduces to U B = max p ( v | t ): I ( T ; V ) ≤ R 0 1 − H ( Z | V ) (57) W e should re mark that the conv erse o btained in [2] for th is channel utilized the modu lo-add iti ve nature o f the channel. For such a cha nnel, a unifo rm distribution on X makes the channel outpu t Y indepen dent of n oise Z , ther eby making the proceed ings in the co n verse e asier . Our upper boun d does not rely o n the nature o f the channel and holds fo r any p ( y | x, t ) . W e have thus shown that fo r all the cases wher e the capacity is established, our bou nd is tig ht. T o illustrate the usefulness of our b ound , we will consider a channel which doe s not fall into any of these classes. V I . C A P AC I T Y R E S U LT F O R A S Y M M E T R I C B I N A RY E R A S U R E C H A N N E L W I T H T W O S TA T E S W e will show that fo r a p articular symmetr ic binar y input erasure chan nel with two states, our upp er boun d yields the capacity which turns out to be strictly less than the cut- set bound . Th e state T is binary with Pr ( T = 0) = α . The channel input X is bin ary and ch annel outpu t Y is ternary . For chan nel states T = 0 , 1 , the transition m atrices p ( y | x, t ) are given as (Figure 2 ) , W 0 =  0 1 − ǫ ǫ ǫ 1 − ǫ 0  W 1 =  ǫ 1 − ǫ 0 0 1 − ǫ ǫ  It should be noted th at this class of chan nels does not fall into the class of ch annels co nsidered in [1] since T canno t be obtained as a deterministic function of X an d Y . Moreover , the channel output Y cann ot be expressed in the form as Y = X ⊕ Z , for some p ( t | z ) , where ⊕ is m odulo - 2 addition , since the cardinality of Y is different from the card inality of X . Henc e, the conv erse techniq ue dev eloped in [2] f or modulo- additive relay channels does not app ly to th is chann el. Howe ver , our upper bound holds for any p ( y | x, t ) . W e begin by e v aluating the ach iev ab le rates gi ven by the CAF sch eme, C ≥ s up I ( X ; Y | V ) s.t. I ( T ; V | Y ) ≤ R 0 for so me p ( x, t, v ) = p ( x ) p ( t ) p ( v | t ) (58) Throu ghout this pa per, we denote the entropy functio n as h ( k ) ( s 1 , . . . , s k ) = − k X i =1 s i log ( s i ) (59) where s i ≥ 0 , i = 1 , . . . , k an d P i s i = 1 . W e will deno te the b inary entr opy function as h ( s ) . W e first define Pr ( X = 0) = p an d o btain th e inv olved probabilities, p ( Y = 0 ) = ǫ ( α ∗ p ) (60) p ( Y = 1 ) = 1 − ǫ (61) p ( Y = 2 ) = ǫ (1 − α ∗ p ) (62) and p ( Y = 0 | T = 0) = ǫ (1 − p ) (63) p ( Y = 1 | T = 0) = 1 − ǫ (64) p ( Y = 2 | T = 0) = ǫp (65) and p ( Y = 0 | T = 1) = ǫp (66) p ( Y = 1 | T = 1) = 1 − ǫ (67) p ( Y = 2 | T = 1) = ǫ (1 − p ) (68) T = 1 1 0 1 2 X 0 1 − ǫ 1 − ǫ ǫ ǫ 0 2 ǫ X 0 1 1 ǫ T = 0 1 − ǫ 1 − ǫ Y Y Figure 2 : A symmetr ic binary erasure chann el with tw o states. where we have defined a ∗ b = a (1 − b ) + b (1 − a ) (69 ) Furthermo re, we a lso note the f ollowing inequality , h (3) ( a, b, c ) = 1 2 h (3) ( a, b, c ) + 1 2 h (3) ( c, b, a ) (7 0) ≤ h (3)  a + c 2 , b, a + c 2  (71) = h ( b ) + 1 − b (72) Using this fact, we have H ( Y ) = h (3) ( ǫ ( α ∗ p ) , 1 − ǫ, ǫ (1 − α ∗ p )) (73) ≤ h ( ǫ ) + ǫ (74) Also, a uniform distribution on X , yield s the m aximum entropy f or Y , and ma kes Y and T ind ependen t. Note that the maximum entro py o f Y in this case is h ( ǫ ) + ǫ which is strictly less than log (3) fo r all ǫ ∈ [0 , 1] . Hence, fo r a uniform X , we hav e H ( Y | V ) = H ( Y ) (75) = h ( ǫ ) + ǫ (76) W e also define, η v = Pr ( T = 1 | V = v ) , v = 1 , . . . , |V | (77) Using this definition, we can write H ( Y | X , V ) fo r any d istri- bution p ( x ) o n X as follows, H ( Y | X , V ) = X v p ( v ) X x p ( x ) H ( Y | X = x, V = v ) (78) = X v p ( v ) h (3) ( η v ǫ, 1 − ǫ, (1 − η v ) ǫ ) (79) = H ( U | V ) (80) where we have defin ed a random variable U with |U | = 3 and p ( u | t ) , expressed as a stochastic matr ix W which is given as W =  ǫ 1 − ǫ 0 0 1 − ǫ ǫ  (81) Thus, H ( Y | X , V ) is inv ariant to th e d istribution of X . More- over , by con struction, the rand om variables ( T , U , V ) satisfy the Markov chain V → T → U . W e n ow return to the evaluation of th e ra tes gi ven by the CAF schem e given in (58). Using (76) and (8 0), we ha ve f or a u niform distribution on X , I ( X ; Y | V ) = H ( Y | V ) − H ( Y | X , V ) (82) = h ( ǫ ) + ǫ − H ( U | V ) (83) Furthermo re, for uniform X , we have I ( T ; V | Y ) = I ( T ; V ) , thus the constraint in ( 58) simplifies to I ( T ; V ) ≤ R 0 . For simplicity , defin e the set L ( γ ) = { p ( v | t ) : H ( T | V ) ≥ γ ; V → T → U } ( 84) Using (8 3) and (8 4), we obtain a lower b ound on the capacity as C ≥ h ( ǫ ) + ǫ − inf p ( v | t ) ∈ L ( h ( α ) − R 0 ) H ( U | V ) (85) W e now evaluate our upper boun d. Using the following fact, min( I ( X , V ; Y ) , I ( X ; Y | T )) ≤ I ( X , V ; Y ) (86) we o btain a weaker version of our u pper b ound in (38) as C ≤ sup I ( X , V ; Y ) (87) = s up( H ( Y ) − H ( Y | X , V )) (88) ≤ s up( h ( ǫ ) + ǫ − H ( Y | X , V )) (89) = h ( ǫ ) + ǫ − inf H ( Y | X , V ) (90) = h ( ǫ ) + ǫ − inf p ( v | t ) ∈ L ( h ( α ) − R 0 ) H ( U | V ) (91) where (89) follows from (74), and the sup in ( 87)-(89) is taken over all p ( x ) and th ose p ( v | t ) wh ich satisfy I ( T ; V ) ≤ R 0 . Hence, f rom (8 5) and (91), the capacity is given b y C = h ( ǫ ) + ǫ − inf p ( v | t ) ∈ L ( h ( α ) − R 0 ) H ( U | V ) (92) W e will now explicitly e v aluate the capacity expression ob- tained in (92) and comp are it with th e cut-set boun d. For this purpo se, we nee d a resu lt o n the con ditional en tropy of depe ndent random variables [9]. Let T , U be a pair o f depend ent ran dom variables with a joint d istribution p ( t, u ) . For 0 ≤ γ ≤ H ( T ) , de fine the functio n G ( γ ) as the infimum of H ( U | V ) , with r espect to all discrete random variables V such that H ( T | V ) = γ an d the rando m variables V and U are con ditionally indepen dent giv en T . For the case when T is binar y and p ( u | t ) , expressed as a stoch astic matr ix W , takes the fo rm in (81), we have f rom [9], G ( γ ) = inf p ( v | t ) ∈ L ( γ ) H ( U | V ) (93 ) = h ( ǫ ) + ǫγ (94) W e will use this result from [9] in explicitly ev aluating the capacity in (92). First no te that, if R 0 ≥ h ( α ) , then G ( h ( α ) − R 0 ) = G (0) = h ( ǫ ) ( 95) whereas, if R 0 < h ( α ) , then G ( h ( α ) − R 0 ) = h ( ǫ ) + ǫ ( h ( α ) − R 0 ) (96) Using (9 5) an d (96), the capacity expression in (92) ev aluates to, C ( R 0 ) =  ǫ, R 0 ≥ h ( α ) ǫ (1 − h ( α )) + ǫR 0 , R 0 < h ( α ) (97) which ca n be written in a compact fo rm as, C ( R 0 ) = min( ǫ (1 − h ( α )) + ǫR 0 , ǫ ) (98) The cut-set bo und is ob tained by ev aluating (39) for the channel in consideration. Evaluation of th e cut-set b ound is straightfor ward by noting that I ( X ; Y ) and I ( X ; Y | T ) are both m aximized b y a u niform p ( x ) . For a u niform distribution on X , we have the fo llowing equalities, I ( X ; Y ) = ǫ (1 − h ( α )) (99) I ( X ; Y | T ) = ǫ (100) Hence, th e cut- set b ound is giv en as, C S ( R 0 ) = min( ǫ (1 − h ( α )) + R 0 , ǫ ) (101) The d ifference between the capacity and the cut-set b ound is evident f rom the first term in the min operation , i.e., the capa city exp ression in (98) has a n ǫR 0 appearin g in the minimum, as opposed to R 0 appearin g in the cut- set boun d a t the c orrespon ding place in (101). T he cut-set bo und and the capacity ar e shown in Figure 3 as functio ns o f R 0 for α = 0 . 3 and ǫ = 0 . 4 . In conclu sion, for this channel which doe s not fall into the classes of cha nnels studie d in [1] and [2], ou r upp er bo und equals the CAF achiev ab le rate, thu s yielding the cap acity , which is strictly less than the cut- set bo und for R 0 < h ( α ) . V I I . A C H A N N E L W I T H B I N A RY M U LT I P L I C A T I V E S TA T E A N D B I N A RY A D D I T I V E N O I S E W e will ev alu ate our upper bou nd and compare it with the cut-set bound fo r the case when X , T and N are binary and the channel is gi ven as, Y = T X + N (102) The channel o utput Y tak es values in the set { 0 , 1 , 2 } . The random variables T an d N are distributed as T ∼ Ber ( α ) and N ∼ Ber ( δ ) . This relay ch annel d oes n ot fall into the subc lass of channels conside red in [1]. Moreover, the con verse o btained in [2] does not app ly to this chann el since the output cannot be written as a mod ulo-sum . T o ev aluate our upper bou nd, let us define Pr ( X = 1) = p (103) Pr ( T = 1 ) = α (104) Pr ( N = 1) = δ (105) W e then obtain H ( Y ) as follows H ( Y ) = h (3) ( P Y (0) , P Y (1) , P Y (2)) (106) where P Y (0) = p (1 − α )(1 − δ ) + (1 − p )(1 − δ ) (107 ) P Y (1) = (1 − p ) δ + p [(1 − α ) δ + α (1 − δ )] (108) P Y (2) = pαδ (109) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 R 0 Cut−set bound Capacity Figure 3 : Capacity of the binary sy mmetric erasure chan nel for α = 0 . 3 an d ǫ = 0 . 4 . and H ( Y | X ) is obtained as, H ( Y | X ) = (1 − p ) H ( N ) + pH ( T + N ) (110) = (1 − p ) h ( δ ) + ph (3) ((1 − α )(1 − δ ) , α ∗ δ, αδ ) (111) The b roadcast cut is o btained as, I ( X ; Y | T ) = H ( Y | T ) − H ( Y | X , T ) (112) = (1 − α ) h ( δ ) + αh (3) ((1 − p )(1 − δ ) , p ∗ δ, pδ ) − h ( δ ) (113) The cu t-set bo und is gi ven by , C S = max p min { I ( X ; Y ) + R 0 , I ( X ; Y | T ) } (114) W e now ev aluate our bound by first c onsidering , I ( X , V ; Y ) = H ( Y ) − H ( Y | X , V ) (115) W e have already ev aluated H ( Y ) in (106). Con sider H ( Y | X , V ) : H ( Y | X , V ) = X ( x,v ) P X ( x ) P V ( v ) H ( Y | X = x, V = v ) (116) = X v P V ( v )  (1 − p ) H ( Y | X = 0 , V = v ) + pH ( Y | X = 1 , V = v )  (117) = X v P V ( v )  (1 − p ) H ( N ) + pH ( T + N | V = v )  (118) = X v P V ( v )  (1 − p ) h ( δ ) + pH ( T + N | V = v )  (119) = X v P V ( v )  (1 − p ) h ( δ ) + pH ( W | V = v )  (120) = (1 − p ) h ( δ ) + pH ( W | V ) (121) where we h av e defined ano ther rando m v ariable W as fo llows, W = T + N (122) W e are interested in lower bound ing H ( W | V ) . W e also k now that any permissible c ondition al distribution p ( v | t ) satisfies the constra int I ( T ; V ) ≤ R 0 . Using this, we also h av e the following, H ( T | V ) ≥ h ( α ) − R 0 (123) Let us also d efine, P T | V ( T = 1 | V = v ) = η v , v ∈ 1 , . . . , |V | (124) W e now retu rn to calculating H ( W | V ) Pr ( W = w | V = v ) = X t P T | V ( t | v ) P W | T ,V ( w | t, v ) (1 25) = (1 − η v ) P ( w | T = 0 , V = v ) + η v P ( w | T = 1 , V = v ) (126) Since the random variable W takes values in the set { 0 , 1 , 2 } , we o btain, Pr ( W = 0 | V = v ) = (1 − η v )(1 − δ ) (127 ) Pr ( W = 1 | V = v ) = η v ∗ δ (128) Pr ( W = 2 | V = v ) = η v δ (129) W e finally obtain, H ( W | V ) = X v P V ( v ) h (3) ((1 − η v )(1 − δ ) , η v ∗ δ, η v δ ) (130) For th e specia l case when the ad ditiv e noise is N ∼ Ber (1 / 2) , the above expression simplifies to H ( W | V ) = X v P V ( v ) h (3)  (1 − η v ) 2 , 1 2 , η v 2  (131) = X v P V ( v ) 1 2 h ( η v ) + 1 ! (132) = 1 2 H ( T | V ) + 1 (133) ≥ 1 2 ( h ( α ) − R 0 ) + 1 (134) where (13 4) follows f rom (123). Sub stituting (134) in (121) we o btain H ( Y | X , V ) = (1 − p ) h ( δ ) + p H ( W | V ) (135) ≥ (1 − p ) h ( δ ) + p 1 2 ( h ( α ) − R 0 ) + 1 ! (136) Continuing from (115), we obtain an upp er b ound on I ( X , V ; Y ) as follows, I ( X , V ; Y ) = H ( Y ) − H ( Y | X , V ) (137 ) ≤ H ( Y ) − 1 − p 2 ( h ( α ) − R 0 ) (138) Moreover , th e first term ap pearing in th e cu t-set bound sim- plifies to I ( X ; Y ) + R 0 = H ( Y ) − H ( Y | X ) + R 0 (139) = H ( Y ) − 1 − p 2 h ( α ) + R 0 (140) W e thus obtain o ur upp er boun d as, U B = ma x p ∈ [0 , 1] min h H ( Y ) − 1 − p 2 h ( α ) + pR 0 , I ( X ; Y | T ) i (141) whereas th e cut-set boun d is, C S = max p ∈ [0 , 1] min h H ( Y ) − 1 − p 2 h ( α ) + R 0 , I ( X ; Y | T ) i (142) The difference between the cut- set bound and ou r uppe r bound is evident from the first term in th e min operation, i.e., ou r upper bound has a pR 0 term in (141), as opposed to R 0 at the correspo nding place in (142). Both these b ound s alo ng with the CAF ra te are illustra ted in Fig ure 4 as a fun ction of R 0 for the c ase when α = 1 / 2 and δ = 1 / 2 . W e should remar k here that altho ugh ou r boun d is strictly smaller th an th e cut-set bo und f or certain values of R 0 , it is strictly larger than the rates g iv en by the CAF scheme. He re, the CAF rates are ev aluated by restricting V to b e binar y , i.e., by co nsidering all conditio nal distributions p ( v | t ) , such that, |V | = 2 . T herefor e, the CAF rates plo tted in Figure 4 are potentially suboptima l an d can be potentially improved upon by increasing the cardin ality of V . V I I I . D I S C U S S I O N Let us recall our up per bou nd obtained in (38), U B = sup min { I ( X , V ; Y ) , I ( X ; Y | T ) } s.t. R 0 ≥ I ( T ; V ) over p ( x ) p ( t ) p ( v | t ) (143) Using the fact that min( I ( X , V ; Y ) , I ( X ; Y | T )) ≤ I ( X , V ; Y ) (144 ) and o bserving that I ( X , V ; Y ) = I ( V ; Y ) + I ( X ; Y | V ) (1 45) it can be n oted that our upper bou nd in (143) can b e further upper b ound ed as C ≤ s up I ( V ; Y ) + I ( X ; Y | V ) (146) s.t. I ( T ; V ) ≤ R 0 (147) for so me p ( x ) p ( v | t ) (148) On the other han d, the cap acity is alw ays lower bound ed by the CAF rate, C ≥ s up I ( X ; Y | V ) (1 49) s.t. I ( T ; V | Y ) ≤ R 0 (150) for so me p ( x ) p ( v | t ) (151) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.16 0.17 0.18 0.19 0.2 0.21 0.22 0.23 0.24 0.25 0.26 R 0 Cut−set bound Our upper bound CAF rate Figure 4 : Comparison of our up per bou nd with the cut- set bound when T ∼ Ber (1 / 2) and N ∼ Ber (1 / 2 ) . Now using the following fact, I ( T ; V | Y ) = H ( V | Y ) − H ( V | T ) (152) = I ( T ; V ) − I ( V ; Y ) (1 53) we c an rewrite the CAF lower bo und o n the capacity as C ≥ s up I ( X ; Y | V ) (154) s.t. I ( T ; V ) − I ( V ; Y ) ≤ R 0 (155) for so me p ( x ) p ( v | t ) (156) W e can see th at the CAF lower bou nd on the capa city inv o lves taking a supremum of I ( X ; Y | V ) subject to the con straint I ( T ; V ) − I ( V ; Y ) ≤ R 0 whereas ou r upper b ound inv olves taking a supremu m of a larger quan tity I ( V ; Y ) + I ( X ; Y | V ) subject to a stricter constrain t I ( T ; V ) ≤ R 0 . Although th ese two maximization p roblems are different, for the class of channels for which capacity was obtain ed, at th e capa city achieving input d istribution p ( x ) , we had I ( V ; Y ) = 0 . Moreover, the same input distribution p ( x ) yielded the maxim um fo r both maximization problem s. Thus, for th e class of ch annels c onsidered in Section VI, these two maximization p roblems are eq uiv alent. This observation yields a heuristic explanation as to why we were able to obtain the capacity results for th ese classes of channels. I X . A N E W L O W E R B O U N D O N C R I T I C A L R 0 In [10] , Cover posed a slightly different pr oblem regarding the general prim itiv e re lay channel. Considering the capa city as a fun ction of R 0 , i. e., C ( R 0 ) , first obser ve the fo llowing facts, C (0) = sup p ( x ) I ( X ; Y ) (157) C ( ∞ ) = sup p ( x ) I ( X ; Y | T ) (158) Moreover , C ( R 0 ) is a n ondecr easing functio n of R 0 . Cover posed the follo wing q uestion in [10]: what is the smallest v alue of R 0 , say R ∗ 0 , for wh ich C ( R ∗ 0 ) = C ( ∞ ) ? As an applicatio n of our upper boun d, we implicitly p rovide a new lower bo und on R ∗ 0 for th e class of p rimitive r elay channels studied in th is paper . For the class o f ch annels con sidered in Section VI, we obtained the capacity . As a conseq uence, we can explicitly characterize R ∗ 0 for this class of ch annels as h ( α ) . Further- more, fo r the class o f c hannels con sidered in Section VI I, ou r upper bound on the capa city yields an improved lower bo und on R ∗ 0 than the o ne pr ovided by th e cu t-set bou nd, which is clearly evident in Figure 4 . X . C O N C L U S I O N S W e ob tained a new uppe r bo und for a class of primitive relay channels. T he prim iti ve relay chan nel studied in this p aper can also be considere d as a state-de penden t discrete memory less channel, with rate-limited state information av ailable at the receiver and no state inform ation a vailable at th e transmitter . Using our upp er bo und, we first recover all previously known cap acity results f or such channels. Further more, we explicitly char acterize the capacity of a new subclass of these primitive re lay channels which does not overlap with the classes p reviously studied in [1] , [2] . In pa rticular, for this class o f c hannels, it is assumed that the re ar e two chan nel states, an d fo r each channel state, there is an era sure ch annel from X to Y . W e show that the capacity for such channels is strictly smaller than th e cu t-set boun d for certain values of R 0 . This capacity result v alidates a co njecture due to Ahlswede and Ha n [8 ] fo r this class of channels. Moreover , we also evaluated our upper bo und for a c ase where Y = T X + N , wher e T , X and N are binar y . T his channel does n ot fall into any of the classes studied in [1], [2] an d ne ither do es it fall into the aforementio ned class of channels. W e show that our u pper bound strictly improves upon the cut-set boun d f or certain v alues of R 0 , altho ugh, our upper bo und is strictly larger than the rates yield ed by a potentially sub optimal evaluation of the CAF schem e. R E F E R E N C E S [1] Y -H. Kim. Capacity of a class of determini stic rela y channels. IEEE T rans. on Information Theory , 54(3):1328–1329 , Mar . 2008. [2] M. Aleksic, P . Razag hi, and W . Y u. Capacity of a class of modulo-sum relay channels. Submitted to IEEE T rans. on Information Theory , 2007. [3] T . M. Cover and A. E l Gamal. Capacity theore ms for the relay channel . IEEE T rans. on Informati on Theory , 25(5):572 –584, September 1979. [4] Z. Zhang. P artia l conv erse for the rela y channel. IEEE T rans. on Informatio n Theory , 34(5):1106–1 110, September 1988. [5] Y -H. Kim. Coding techniques for primiti v e relay channels. Proc. Annual Allerton Confer ence on Communication, Contr ol and Computing , pages 129–135, 2007. [6] T . M. Cove r and J. A. Thomas. Elements of Information Theory . Ne w Y ork :W ile y , 1991. [7] C. Heegar d and A. El Gamal. On the capa city of computer m emory with defects. IEEE T rans. on Informati on Theory , 29(5):731–73 9, Sep. 1983. [8] R. Ahlswede and T . S. Han. On source coding with side information via a multiple -acce ss channel and rela ted problems in multi-use r information theory . IEEE T rans. on Information Theory , 29(3):396 –412, May 1983. [9] H. S. W itsenh ausen and A. D. W yner . A conditiona l entropy bound for a pair of discrete random vari ables. IEEE T rans. on Information Theory , 21(5):493– 501, September 1975. [10] T . M. Cov er . The capacity of the relay channel. In Open Probl ems in Communicat ion and Computa tion , pages 72–73, 1987.