Outer Bounds for Multiple Access Channels with Feedback using Dependence Balance

Outer Bounds for Multiple Access Channels with F eedbac k using Dep endence Balance ∗ Ra vi T andon Senn ur Ulukus Departmen t of Electrical and Computer Engineering Univ ersit y of Maryland, College P ark, MD 20 7 42 r avit@umd.e du ulukus@umd.e du Octob er 29 , 20 1 8 Abstract W e use the idea of d ep endence balance [1] to obtain a new outer b ound for th e ca- pacit y region of the discrete memoryless multiple access channel with noiseless feedbac k (MA C-FB). W e consider a binary add itiv e noisy MA C-FB whose feedbac k capacit y is not kno wn. The binary additive noisy MAC consid ered in this p ap er can b e view ed as the discrete coun terpart of the Gaussian MA C-FB. Ozaro w [2] established that th e capacit y r egion of the tw o-user Gaussian MA C-FB is giv en by the cut-set b oun d. O ur result sho ws that for the discrete v ersion of the c hannel considered by Ozaro w, this is not the case. Direct ev aluation of our outer b ound is intracta ble due to an inv olve d auxiliary random v ariable wh ose large cardinalit y prohibits an exhau s tiv e searc h. W e o v ercome this difficult y b y usin g f unctional an alysis to explicitly ev aluate our outer b ound. Ou r outer b ound is s tr ictly less than the cut-set b ound at all p oin ts on the capacit y region where feedb ac k in creases capacit y . In addition, we explicitly ev aluate the Co v er-Leung ac hiev able rate region [3] for the binary additiv e noisy MA C-FB in consideration. F u r thermore, u s ing th e to ols d ev eloped for the ev aluation of our outer b ound, we also explicitly c haracterize the b oundary of the feedbac k capacit y region of the bin ary erasure MA C, for w hic h the Co v er-Leung ac hiev able r ate region is known to b e tigh t. This last result confirms that the feedbac k strategies develo p ed in [4] for the b inary erasur e MA C are capacit y ac hieving. ∗ This work was suppo rted by NSF Gra n ts CCF 0 4-47613 , CCF 05-14 8 46, CNS 07-1 6311 and CCF 07- 29127 . 1 1 In tro duc tion Noiseless feedbac k can increase the capacity region o f the discrete memoryless MA C, unlik e for the single-user discrete memoryless channe l. This was sho wn b y Gaa r der and W olf in [5] for the binar y erasure MAC , which is defined as Y = X 1 + X 2 . Ozaro w sho w ed in [2] that feedbac k can also increase the capacity region of a tw o- user Gaussian MAC -FB. A constructive ac hiev abilit y sche me based on the classical Kailath-Schalkw ijk [6 ] feedbac k sc heme w as show n to b e o ptimal for the tw o-user G aussian MAC -FB. Moreo v er, t he cut-set outer b ound w as sho wn to b e tight in this case. Subsequen tly , Cov er and Leung o btained an ac hiev able rate region for the general MA C- FB based on blo c k Mark ov sup erp osition co ding [3 ]. Ev en though this region is in general larger tha n the capacit y region of the MA C without feedbac k, it is not optimal for the t w o-user G a ussian MA C-FB, as was sho wn in [2]. Kramer [7] used the notion of directed information to obtain an expression for t he capacit y regio n of the discrete memoryless MA C- FB. Unfortunately , this expression is in an incomputable non-single-letter form. Recen tly , Bross and Lapidoth [8] pro po sed an achiev able rate r egio n for the tw o- user discrete memo- ryless MA C-FB and show ed that their region includes the Co v er-Leung region, t he inclusion b eing strict for some c hannels. F o r a sp ecific class o f MAC-FB, Willems [9] dev elop ed an outer b ound that equals the Co v er-Leung achie v able ra t e regio n. F o r this class of MA C-F B, eac h c hannel input (sa y X 1 ) should b e expressible as a deterministic function of the other channel input ( X 2 ) and the ch annel o utput ( Y ). The binary erasure MA C considered b y Gaarder and W olf, where Y = X 1 + X 2 , fa lls into this class of c hannels. T herefore, Co v er-Leung region is the feedbac k capacit y region fo r the binary erasure MAC. A general outer b ound for MA C-FB is the cut-set b ound. Although the cut-set b ound was sho wn to b e tigh t for the tw o- user G a ussian MAC -FB, it is in general lo ose. An intuitiv e reason for the cut-set b ound to b e lo ose for the general MA C-FB is its p ermissibilit y of arbitrary input distributions, some of which yielding ra tes whic h may not b e ac hiev able. F or instance, ev en though Co v er-Leung achie v ability sc heme intro duces correlation b etw een X 1 and X 2 , it is a limited f orm of correlation, as the channe l inputs are conditionally indep enden t giv en an auxiliary random v ariable, whereas the cut-set b ound allows all p ossible correlations. The idea of dep endence bala nce w as in tro duced by Hekstra and Willems in [1] to obtain an o uter b ound on the capacit y region of the single-output tw o-wa y c hannel. The basic idea b ehind this o uter b o und is to restrict the set of allow able input distributions, consequen tly restricting arbitrary correlation b et w een channel inputs. The autho r s also deve lop ed a par- allel c hannel extension for the dep endence balance b ound. The parallel c hannel extension can b e interpreted as follo ws: the parallel c hannel output can b e considered as a g enie aided information whic h is made a v ailable at b oth tra nsmitters a nd the receiv er and it also effects the set of allow a ble input distributions through the dep endence ba lance b ound. D epending on the choice of the genie information (whic h is equiv alen t to c ho osing a parallel c hannel), 2 there is an inheren t tra deoff b etw een the set of allo w able input distributions and the exces- siv e m utual info rmation r a te terms whic h app ear in the rate expres sions as a conseque nce of the parallel channel output. W e will exploit t his tradeoff provided by the parallel channel extension of the dep endenc e balance b ound t o obtain a strict impro v emen t o v er the cut- set b ound f o r a par ticular MA C whose feedbac k capacit y is not known . T o mot iv ate the c hoice of our MA C, consider the binary erasure MA C used by Gaar der and W olf given by Y = X 1 + X 2 . If w e introduce binary additive noise at the c hannel output, then the c hannel b ecomes Y = X 1 + X 2 + N , where all X 1 , X 2 and N are binary and N has a uniform distribution. This is a non-deterministic no isy MA C whic h do es not fall in to an y class of channels fo r which the feedbac k capacit y is kno wn. W e should men tion t ha t this particular MAC w as extensiv ely studied b y Kramer in [7 , 10], where the first improv emen t o v er the Co v er-Leung ac hiev able rate region w as obta ined. W e extend the idea of dep endence balance to obtain an o uter b ound for the en tire capacity region of this binary additive noisy MA C-FB. Direct ev a luation of the parallel channe l based dep endence balance b ound is intractable due to an inv olv ed auxiliary random v ariable whose large cardinality prohibits an exhaustiv e searc h. W e use comp osite functions and their prop erties to obtain a simple ch aracterization for our b ound. Our outer b ound strictly impro v es up on the cut-set b ound at all p oints on the b oundary where feedbac k incre ases capacit y . In addition, w e explicitly ev aluat e the Co v er-Leung achiev able rate region for our binary a dditiv e noisy MAC -FB. W e particularly fo cus on t he symmetric-rate 1 p oin t on the feedbac k capacit y regio n of this c hannel. Co v er-Leung’s ac hiev able symmetric-rate for this channel w as obtained in [10] as 0 . 43621 bits/transmission. In [1 0], Kramer obtained an improv ed symmetric-rate inner b ound as 0 . 4387 9 bits/transmission by using superp osition co ding a nd binning with co de trees. The cut-set upp er b ound on the symmetric-rate w as obtained in [10] as 0 . 45915 bits/transmission. W e obtain a symmetric-rate upp er b ound of 0 . 45330 bits/tra nsmiss ion whic h strictly impro v es up on the cut-set b ound. F urthermore, we also sho w tha t a binary and uniform selection of the in v olv ed auxiliary random v ariable is sufficien t to obtain our symmetric-rate upp er b ound. It should b e remarked that the channel w e consider in this pap er can b e t hough t of as the discrete coun terpart of the c hannel considered b y Ozarow [2]. Although the cut-set b o und w as show n to b e tight f or the tw o- user Gaussian MA C-FB, o ur result sho ws that the cut-set b ound is not tigh t for the discrete v ersion of the additiv e noisy MA C-FB. As an a pplication o f the prop erties of the comp osite functions deve lop ed in this pa p er, w e are a ble to obtain t he en tire b oundary of the capacity region o f the binary erasure MAC - FB. The ev alua tion of the asymmetric rate pairs o n the b oundary of the feedbac k capacit y region of the binary erasure MAC w as men tioned as a n op en problem in [11]. It w as sho wn 1 By s ymmetric-rate p oint , w e refer to the maximum rate R such that the r ate pair ( R , R ) lies in the capacity r e gion of MAC-FB. 3 in [12] tha t a binary and uniform auxiliary random v ariable T is sufficie n t to attain the sum-rate p oin t on the capacit y region of the binary erasure MA C-FB. W e show here that this is also the case for an y asymmetric r ate p oint o n the b oundary of the feedbac k capacit y region. This result also complemen ts the work of Kramer [4], where feedbac k strategies w ere dev elop ed for the binary erasure MA C-FB and it w as shown that these strategies achie v e all rates yielded b y a binary selection of t he auxiliary random v a riable T in the capacity region. Our result henc e sho ws in effect that the feedbac k strategies dev elop ed in [4 ] fo r binary erasure MAC are optimal and capacit y achie ving. 2 System Mo del A discrete memoryle ss t w o-user MAC -FB (see Figure 1) is defined b y t he following: t wo input alphab ets X 1 and X 2 , an output alphab et Y , and the c hannel defined b y a probability transition function p ( y | x 1 , x 2 ) for all ( x 1 , x 2 , y ) ∈ X 1 × X 2 × Y . A ( n, M 1 , M 2 , P e ) co de for the MA C-FB consists of tw o sets of enco ding f unctions f 1 i , f 2 i for i = 1 , . . . , n and a decoding function g f 1 i : M 1 × Y i − 1 → X 1 , i = 1 , . . . , n f 2 i : M 2 × Y i − 1 → X 2 , i = 1 , . . . , n g : Y n → M 1 × M 2 The t w o transmitters pro duce indep enden t and uniformly distributed messages W 1 ∈ { 1 , . . . , M 1 } and W 2 ∈ { 1 , . . . , M 2 } , resp ectiv ely , and transmit them through n c hannel uses. The a v erage error probability is defined as P e = P r ( g ( Y n ) 6 = ( W 1 , W 2 )). A rate pair ( R 1 , R 2 ) is said to b e achie v able for MA C-FB if for any ǫ ≥ 0, there exists a pair of n enco ding functions { f 1 i } n i =1 , { f 2 i } n i =1 , and a deco ding function g suc h that R 1 ≤ log( M 1 ) /n , R 2 ≤ log( M 2 ) /n and P e ≤ ǫ for sufficien tly larg e n . T he capacity region of MA C-FB is the closure of the set of all achiev a ble ra te pairs ( R 1 , R 2 ). 3 Cut-Se t Outer Bound for MAC-FB By applying Theorem 14.10.1 in [13], the cut-set outer b ound on the capacity region of MA C-FB can b e obtained as: C S = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y | X 2 ) (1) R 2 ≤ I ( X 2 ; Y | X 1 ) (2) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (3) 4 Encoder 1 W 1 W 2 Encoder 2 X 1 X 2 Y p ( y | x 1 , x 2 ) Decoder ˆ W 1 ˆ W 2 Figure 1: Th e multiple access c hannel with noiseless feedbac k (MA C-FB). where t he random v ariables ( X 1 , X 2 , Y ) ha v e the join t distribution p ( x 1 , x 2 , y ) = p ( x 1 , x 2 ) p ( y | x 1 , x 2 ) (4) The cut-set outer b ound allo ws all input distributions p ( x 1 , x 2 ), whic h mak es it seemingly lo ose since an ac hiev able sc heme migh t not ac hiev e arbitrary correlation and rates giv en b y the cut-set b ound. Our aim is to r estrict the set of allo w able input distributions by using a dep endence bala nce approach. 4 Dep e ndence Balance Oute r B ound for MAC-FB Hekstra and Willems [1] sho w ed t hat the capa city region of MA C-FB is con tained within D B , where D B = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y | X 2 , T ) (5) R 2 ≤ I ( X 2 ; Y | X 1 , T ) (6) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (7) where t he random v ariables ( X 1 , X 2 , Y , T ) hav e the join t distribution p ( t, x 1 , x 2 , y ) = p ( t ) p ( x 1 , x 2 | t ) p ( y | x 1 , x 2 ) (8) and also satisfy the follo wing dep endence balance b ound I ( X 1 ; X 2 | T ) ≤ I ( X 1 ; X 2 | Y , T ) (9) where T is sub ject to a cardinality constrain t of |T | ≤ |X 1 ||X 2 | + 2. The dep endence balance b ound restricts the set of input distributions in the sense that it allo ws only those input distributions p ( t, x 1 , x 2 ) whic h satisfy ( 9). It should b e noted tha t b y ignoring the constrain t in ( 9 ), one obtains t he cut-set b ound. 5 5 Adaptiv e P arallel C hannel Extension of the Dep en - dence Balance Bound In [1], Hekstra and Willems also deve lop ed an adaptive parallel c hannel extension f o r the dep endence balance b ound whic h is giv en as follow s: Let ∆( U ) denote the set of all distri- butions of U and ∆( U |V ) denote the set of all conditional distributions of U giv en V . Then for any mapping F : ∆( X 1 × X 2 ) → ∆( Z |X 1 × X 2 × Y ), the capacity r egio n o f the MA C- F B is contained in D B P C = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y , Z | X 2 , T ) (10) R 2 ≤ I ( X 2 ; Y , Z | X 1 , T ) (11) R 1 ≤ I ( X 1 ; Y | X 2 ) (12) R 2 ≤ I ( X 2 ; Y | X 1 ) (13) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) (14) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y , Z | T ) o (15) where t he random v ariables ( X 1 , X 2 , Y , Z , T ) hav e t he joint distribution p ( t, x 1 , x 2 , y , z ) = p ( t ) p ( x 1 , x 2 | t ) p ( y | x 1 x 2 ) p + ( z | x 1 , x 2 , y , t ) (16) suc h that for all t p + ( z | x 1 , x 2 , y , t ) = F ( p X 1 X 2 ( x 1 , x 2 | t )) (17) and suc h that I ( X 1 ; X 2 | T ) ≤ I ( X 1 ; X 2 | Y , Z , T ) (18) where T is sub ject to a cardinalit y b ound o f |T | ≤ |X 1 ||X 2 | + 3. W e should remark that the parallel c hannel (defined by p + ( z | x 1 , x 2 , y , t )) is selected apri- ori, and for every choice of the parallel c hannel, one obtains an outer b o und o n the capacit y region of MAC-FB, whic h is in general tigh ter tha n the cut-set b ound. The set of allo w able input distributions p ( t, x 1 , x 2 ) are those whic h satisfy the constraint in (18). Also note t ha t only the righ t hand side of (18), i.e., only I ( X 1 ; X 2 | Y , Z , T ), dep ends on the c hoice of the parallel c hannel. By carefully selecting p + ( z | x 1 , x 2 , y , t ), one can reduce I ( X 1 ; X 2 | Y , Z , T ), thereb y making the constrain t in (18) more stringen t, consequen tly reducing the set of al- lo w able input distributions. T o obtain a n improv emen t ov er the cut-set b o und, we need to select a “go o d” parallel c hannel suc h that it restricts the input distributions to a small allo w able set a nd yields small v alues of I ( X 1 ; Z | Y , X 2 , T ) and I ( X 2 ; Z | Y , X 1 , T ) at t he same 6 time. These tw o m utual information “ leak” terms ar e the extra terms t ha t app ear in ( 1 0) and (11) relative to the r a tes app earing in (5) and (6), r esp ectiv ely . T o motiv ate the c hoice of o ur particular parallel c hannel, first consider a trivial c hoice of Z : Z = φ (a constant). F or this c hoice of Z , (18) r educes to (9) and we are not restricting the set of a llo w able input distributions any more than the D B b ound. Moreo v er, for a constant selection of Z , (10) and (11) reduce to (5) and (6), resp ectiv ely . Th us, a constan t selection of Z for D B P C is equiv alent to D B itself. Also note that the smallest v alue of I ( X 1 ; X 2 | Y , Z , T ) is zero. Th us, it f o llo ws that if w e select a parallel c hannel suc h that I ( X 1 ; X 2 | Y , Z , T ) = 0 for ev ery input distribution p ( t, x 1 , x 2 ), then I ( X 1 ; X 2 | T ) = 0 b y (18). Hence, the smalles t set of input distributions p ermissable b y D B P C consists of those p ( t, x 1 , x 2 ) for wh ic h X 1 and X 2 are conditionally indep enden t g iven T . F urthermore, for a pa rallel c hannel suc h that I ( X 1 ; X 2 | Y , Z , T ) = 0, the b ound in (15) is redundant. This can b e seen from: 0 = I ( X 1 ; X 2 | T ) − I ( X 1 ; X 2 | Y , Z , T ) = I ( X 1 ; Y , Z | T ) − I ( X 1 ; Y , Z | X 2 , T ) = I ( X 1 , X 2 ; Y , Z | T ) − I ( X 1 ; Y , Z | X 2 , T ) − I ( X 2 ; Y , Z | X 1 , T ) (19) Using (19), it is clear that the sum of constraints (10) and ( 1 1) is at least as strong as the constrain t (15 ) . This sho ws that (15) is redundant for the class of parallel channe ls where I ( X 1 ; X 2 | Y , Z , T ) = 0. 6 Binary Additiv e Noisy MA C- FB In this pap er, w e will consider a binary-input additiv e noisy MA C giv en by Y = X 1 + X 2 + N (20) where N is binary , uniform ov er { 0 , 1 } and is indep enden t of X 1 and X 2 . The channe l output Y tak es v alues from the set Y = { 0 , 1 , 2 , 3 } . This c hannel do es not fall in to a ny class of MAC for whic h the feedbac k capacit y region is kno wn. This channel w as also considered b y Kr a mer in [7 , 10] where it w as sho wn that the Co v er-Leung ac hiev a ble ra te is strictly sub-optimal for the sum-rate. W e selec t a para llel c hannel p + ( z | x 1 , x 2 , y ) suc h that I ( X 1 ; X 2 | Y , Z , T ) = 0. By (1 8), this will imply I ( X 1 ; X 2 | T ) = 0, and hence only distributions of the t yp e p ( t, x 1 , x 2 ) = p ( t ) p ( x 1 | t ) p ( x 2 | t ) will b e allow ed. By doing so, w e restrict the set of allow able input dis- tributions t o be the smallest p ermitted b y D B P C , although w e pa y a p enalt y due to the p ositiv e “leak” t erms I ( X 1 ; Z | Y , X 2 , T ) and I ( X 2 ; Z | Y , X 1 , T ). Tw o simple c hoices of Z whic h yield I ( X 1 ; X 2 | Y , Z , T ) = 0 are Z = X 1 and Z = X 2 . F or 7 eac h o f these c hoices, the corresp onding outer b ounds are, D B (1) P C = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y | X 2 , T ) + H ( X 1 | Y , X 2 , T ) (21) R 2 ≤ I ( X 2 ; Y | X 1 , T ) (22) R 1 ≤ I ( X 1 ; Y | X 2 ) (23) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (24) and D B (2) P C = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y | X 2 , T ) (25) R 2 ≤ I ( X 2 ; Y | X 1 , T ) + H ( X 2 | Y , X 1 , T ) (26) R 2 ≤ I ( X 2 ; Y | X 1 ) (27) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (28) where b oth D B (1) P C and D B (2) P C are ev aluated o v er the set o f input distributions of the fo rm p ( t, x 1 , x 2 ) = p ( t ) p ( x 1 | t ) p ( x 2 | t ). F o r the binary additive noisy MA C-FB in consideration whic h is giv en in (20), the fol- lo wing equalities hold for an y distribution of the form p ( t, x 1 , x 2 ) = p ( t ) p ( x 1 | t ) p ( x 2 | t ), H ( X 1 | Y , X 2 , T ) = 1 2 H ( X 1 | T ) (29) H ( X 2 | Y , X 1 , T ) = 1 2 H ( X 2 | T ) (30) Using (2 9 ) and (30), we can simplify D B (1) P C and D B (2) P C as, D B (1) P C = n ( R 1 , R 2 ) : R 1 ≤ min ( I ( X 1 ; Y | X 2 ) , H ( X 1 | T )) (31) R 2 ≤ 1 2 H ( X 2 | T ) (32) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (33) and D B (2) P C = n ( R 1 , R 2 ) : R 1 ≤ 1 2 H ( X 1 | T ) (34) R 2 ≤ min ( I ( X 2 ; Y | X 1 ) , H ( X 2 | T )) (35) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (36) where b oth b ounds are ev aluat ed o v er the set of distributions of t he form p ( t, x 1 , x 2 ) = p ( t ) p ( x 1 | t ) p ( x 2 | t ) and the auxiliary random v ariable T is sub ject to a cardinality constraint 8 of |T | ≤ |X 1 ||X 2 | + 3. The ev aluation of the ab ov e o uter b ounds is rather cum b ersome b ecause for binary inputs, the b ound on |T | is |T | ≤ 7. T o the b est of our kno wledge, no one has b een able to conduct an exhaustiv e searc h ov er an auxiliary ra ndom v ariable whose cardinalit y is larger than 4. In Section 8 , we will obtain a n alternate c haracterization f or our outer b ounds using comp osite functions and their prop erties. F or tha t, w e will first dev elop some useful prop erties of comp osite functions in the next section. A v alid o uter b ound is give n by the in tersection of D B (1) P C and D B (2) P C , D B P C = D B (1) P C \ D B (2) P C (37) W e will sho w that this o uter b ound is strictly smaller than the cut-set b ound at all p oin ts on the capacit y region where feedbac k increases capacit y . 7 Comp o site F unctions and Th eir Prop ertie s Before obta ining a characterization o f our outer b ounds, w e will define a comp osite function and prov e t w o lemmas regarding its prop erties. Thes e lemmas will b e essen tial in obtaining simple c haracterizations for our outer b ounds and the Co v er-Leung ac hiev able rate region. Throughout the pap er, w e will refer to the en trop y function as h ( k ) ( s 1 , . . . s k ) whic h is defined as, h ( k ) ( s 1 , . . . , s k ) = − k X i =1 s i log( s i ) (38) for s i ≥ 0, i = 1 . . . , k , and P k i =1 s i = 1, where all logarithms are to the base 2. W e will denote h (2) ( s ) simply a s h ( s ). T o c haracterize our b ounds, w e will make use of the follow ing function φ ( s ) = ( 1 − √ 1 − 2 s 2 , f or 0 ≤ s ≤ 1 / 2 1 − √ 2 s − 1 2 , f or 1 / 2 < s ≤ 1 (39) It w as sho wn in [12] that the comp osite function h ( φ ( s )) is symmetric around s = 1 / 2 and conca v e in s for 0 ≤ s ≤ 1. The functions φ ( s ) and h ( φ ( s )) ar e illustrated in Figure 2. F rom the definition o f φ ( s ) in (3 9) it is clear that for an y s ∈ [0 , 1], the function φ ( s ) satisfies the follo wing prop ert y φ (2 s (1 − s )) = min( s, 1 − s ) (40) As a consequence , the follow ing holds as well h ( φ (2 s (1 − s ))) = h ( s ) (41) 9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 s φ (s) h( φ (s)) Figure 2: F unctions φ ( s ) and h ( φ ( s )). F o r any s ∈ [0 , 1], the fo llo wing holds from the definition o f φ ( s ), s = ( φ (2 s (1 − s )) , 0 ≤ s ≤ 1 2 1 − φ (2 s (1 − s )) , 1 2 < s ≤ 1 (42) F o r any x ∈ [0 , 1 2 ] and y ∈ [0 , 1 2 ], let us define a function f ( x, y ) , φ ( x ) + φ ( y ) − 2 φ ( x ) φ ( y ) (43) = 1 − p (1 − 2 x )(1 − 2 y ) 2 (44) F ro m the a bov e definition, it is clear t hat the function f ( x, y ) lies in the ra nge [0 , 1 2 ]. Lemma 1 The variable v = s 1 + s 2 − 2 s 1 s 2 (45) is al ways lower b ounde d by f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) for any s 1 ∈ [0 , 1] , s 2 ∈ [0 , 1] . Pro of: W e will prov e this lemma by considering all four p ossible cases. 1. If s 1 ∈ [0 , 1 2 ] , s 2 ∈ [0 , 1 2 ], then from ( 4 2), s 1 = φ (2 s 1 (1 − s 1 )), s 2 = φ (2 s 2 (1 − s 2 )) a nd hence v = f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) (46) 2. If s 1 ∈ [ 1 2 , 1] , s 2 ∈ [ 1 2 , 1], then from (42), s 1 = 1 − φ (2 s 1 (1 − s 1 )), s 2 = 1 − φ (2 s 2 (1 − s 2 )) 10 and hence v = f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) (47) 3. If s 1 ∈ [0 , 1 2 ] , s 2 ∈ [ 1 2 , 1], then from (42), s 1 = φ (2 s 1 (1 − s 1 )), s 2 = 1 − φ (2 s 2 (1 − s 2 )) and hence v = 1 − f ( 2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) ( a ) ≥ f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) (48) where ( a ) follow s by the fa ct that f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) ≤ 1 2 . 4. If s 1 ∈ [ 1 2 , 1] , s 2 ∈ [0 , 1 2 ], then f rom (4 2), s 1 = 1 − φ (2 s 1 (1 − s 1 )), s 2 = φ (2 s 2 (1 − s 2 )) and hence v = 1 − f ( 2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) ( b ) ≥ f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) (49) where ( b ) follows b y the fact that f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )) ≤ 1 2 . Th us, for an y pair ( s 1 , s 2 ), where s 1 ∈ [0 , 1 ], s 2 ∈ [0 , 1], w e hav e sho wn that v ≥ f (2 s 1 (1 − s 1 ) , 2 s 2 (1 − s 2 )).  Lemma 2 The function f ( x, y ) is joi ntly c onvex in ( x, y ) for 0 ≤ x ≤ 1 2 , 0 ≤ y ≤ 1 2 . Pro of: Showing that the function f ( x, y ) is join tly con v ex in ( x, y ) is equiv alen t to sho wing that the Hessian matrix, H of f ( x, y ) is p ositiv e semi-definite, whic h is equiv alent to show ing that the eigen v alues of H are non-negativ e. The Hessian matrix, H , of f ( x, y ) is H =   √ 1 − 2 y 2(1 − 2 x ) 3 / 2 − 1 2 √ (1 − 2 x )(1 − 2 y ) − 1 2 √ (1 − 2 x )(1 − 2 y ) √ 1 − 2 x 2(1 − 2 y ) 3 / 2   (50) The t w o eigen v alues of H are λ 1 = 0 λ 2 = 1 2 √ 1 − 2 y (1 − 2 x ) 3 / 2 + √ 1 − 2 x (1 − 2 y ) 3 / 2 ! (51) whic h are no n- negativ e for all 0 ≤ x ≤ 1 2 and 0 ≤ y ≤ 1 2 , thus completing the pro of.  11 8 Ev aluation of the De p ende nce Balance Oute r Bound W e will now return to the c haracterization of our upp er b ounds D B (1) P C and D B (2) P C . Let the cardinalit y of the auxiliary r andom v ariable T b e fixed and arbitrary , sa y |T | . Then, t he join t distribution p ( t ) p ( x 1 | t ) p ( x 2 | t ) can b e described by the following v ariables: q 1 t = Pr( X 1 = 0 | T = t ) , t = 1 , . . . , |T | q 2 t = Pr( X 2 = 0 | T = t ) , t = 1 , . . . , |T | p t = Pr( T = t ) , t = 1 , . . . , |T | (52) W e will characterize our outer b ounds in terms of three v ariables u 1 , u 2 and u whic h are functions of p ( t, x 1 , x 2 ), and are defined as, u 1 = X t p t q 1 t (1 − q 1 t ) = X t p t u 1 t (53) u 2 = X t p t q 2 t (1 − q 2 t ) = X t p t u 2 t (54) u = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t ) = X t p t u t (55) where we ha v e defined u 1 t = q 1 t (1 − q 1 t ) (56) u 2 t = q 2 t (1 − q 2 t ) (57) u t = q 1 t + q 2 t − 2 q 1 t q 2 t (58) It should b e no t ed that since 0 ≤ q j t ≤ 1, fo r j = 1 , 2, t = 1 , . . . , |T | , the v a r iables u 1 , u 2 , u 1 t and u 2 t all lie in the range [0 , 1 4 ]. Our outer b ounds D B (1) P C and D B (2) P C are comprised of the follo wing information theoretic en tities: 1. H ( X 1 | T ), H ( X 2 | T ) 2. I ( X 1 ; Y | X 2 ), I ( X 2 ; Y | X 1 ) 3. I ( X 1 , X 2 ; Y ). W e will first obtain upp er b ounds for eac h one of these en tities individually in terms of ( u 1 , u 2 , u ). 12 W e upp er b o und H ( X 1 | T ) as follows , H ( X 1 | T ) = X t p t h ( q 1 t ) (59) = X t p t h ( φ (2 q 1 t (1 − q 1 t ))) (60) = X t p t h ( φ (2 u 1 t )) (61) ≤ h ( φ (2 u 1 )) (62) where (60) follow s due to (41), (61) follows from (56 ), and (62) follow s from the fact t hat h ( φ ( s )) is conca v e in s and the a pplication o f Jensen’s inequalit y [13]. Using a similar set of inequalities for H ( X 2 | T ), we obtain H ( X 2 | T ) ≤ h ( φ (2 u 2 )) (63) W e will now upp er b ound I ( X 1 ; Y | X 2 ) in terms of the v ariable u . F or this purp ose, let us first define a = P X 1 X 2 (0 , 0) = X t p t q 1 t q 2 t (64) b = P X 1 X 2 (0 , 1) = X t p t q 1 t (1 − q 2 t ) (65) c = P X 1 X 2 (1 , 0) = X t p t (1 − q 1 t ) q 2 t (66) d = P X 1 X 2 (1 , 1) = 1 − a − b − c. (67) W e no w pro ceed as, I ( X 1 ; Y | X 2 ) = H ( Y | X 2 ) − H ( Y | X 1 , X 2 ) (68) = H ( Y | X 2 ) − 1 (69) = ( a + c ) h (3)  a 2( a + c ) , 1 2 , c 2( a + c )  + ( b + d ) h (3)  b 2( b + d ) , 1 2 , d 2( b + d )  − 1 (70) ≤ h (3)  a + d 2 , 1 2 , b + c 2  − 1 (71) = 1 2 h ( b + c ) (72) = 1 2 h ( u ) (73) 13 where (71) follows by the concavit y of the en trop y function and the application of Jensen’s inequalit y [13]. Using a similar set o f inequalities, w e also hav e I ( X 2 ; Y | X 1 ) ≤ 1 2 h ( u ) (74) W e will no w obtain an upp er b ound on I ( X 1 , X 2 ; Y ). First note that I ( X 1 , X 2 ; Y ) = H ( Y ) − H ( Y | X 1 , X 2 ) (75) = h (4) ( P Y (0) , P Y (1) , P Y (2) , P Y (3)) − 1 (76) where P Y (0) = X t p t q 1 t q 2 t / 2 (77) P Y (1) = X t p t  q 1 t + q 2 t − q 1 t q 2 t  / 2 (78) P Y (2) = X t p t  1 − q 1 t q 2 t  / 2 (79) P Y (3) = X t p t (1 − q 1 t )(1 − q 2 t ) / 2 (80) Using the following fact, h (4) ( α, β , γ , θ ) = 1 2 h (4) ( α, β , γ , θ ) + 1 2 h (4) ( θ , γ , β , α ) (81) ≤ h (4)  α + θ 2 , β + γ 2 , β + γ 2 , α + θ 2  (82) = h ( α + θ ) + h  1 2  (83) = h (1 − ( β + γ )) + 1 (84) where (82) follows by the concavit y of the en trop y function and the application of Jensen’s inequalit y [13], w e now obtain an upp er b ound on I ( X 1 , X 2 ; Y ) b y contin uing from (76), I ( X 1 , X 2 ; Y ) = h (4) ( P Y (0) , P Y (1) , P Y (2) , P Y (3)) − 1 (85) ≤ h (1 − ( P Y (1) + P Y (2))) + h  1 2  − 1 (86) = h  1 − u 2  (87) where (86) follow s b y (84) and (87) follo ws from the fact that P Y (1) + P Y (2) = (1 + u ) / 2 using (7 8 ) and (79), where u is as defined in (55). 14 8.1 A Set of F easible ( u 1 , u 2 , u ) : P W e ha v e obtained upp er b ounds on the information theoretic en tities whic h comprise our outer b ounds in terms of three v ariables u 1 , u 2 and u . W e will now giv e a feasible region for these triples based o n the structures of these v ariables. First, note that for a ny q 1 t ∈ [0 , 1], the fo llo wing holds: u 1 t = q 1 t (1 − q 1 t ) ≤ 1 4 . Similar ly , u 2 t = q 2 t (1 − q 2 t ) ≤ 1 4 . Hence, w e ha v e 0 ≤ u 1 ≤ 1 4 (88) 0 ≤ u 2 ≤ 1 4 (89) W e no w obtain a low er b ound on u as u = X t p t u t (90) ≥ X t p t f (2 u 1 t , 2 u 2 t ) (91) ≥ f 2 X t p t u 1 t , 2 X t p t u 2 t ! (92) = f (2 u 1 , 2 u 2 ) (93) where (9 1) follow s by Lemma 1 and (92) fo llo ws b y Lemma 2 and the a pplication of Jensen’s inequalit y [13]. W e no w obtain another low er b ound on u , u = X t p t u t (94) = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t ) (95) = X t p t ( q 1 t − q 2 1 t + q 2 t − q 2 2 t + ( q 1 t − q 2 t ) 2 ) (96) ≥ X t p t ( q 1 t − q 2 1 t + q 2 t − q 2 2 t ) (97) = X t p t q 1 t (1 − q 1 t ) + X t p t q 2 t (1 − q 2 t ) (98) = u 1 + u 2 (99) 15 Finally , w e obtain an upp er b ound on u in terms of u 1 and u 2 , u = X t p t u t (100) = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t ) (101) = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t + q 2 1 t + (1 − q 2 t ) 2 − q 2 1 t − (1 − q 2 t ) 2 ) (102) ≤ X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t + q 2 1 t + (1 − q 2 t ) 2 − 2 q 1 t (1 − q 2 t )) (103) = 1 − ( u 1 + u 2 ) (104) where ( 1 03) follows b y the inequalit y q 2 1 t + (1 − q 2 t ) 2 ≥ 2 q 1 t (1 − q 2 t ). By noting f (2 u 1 , 2 u 2 ) − ( u 1 + u 2 ) = 1 − p (1 − 4 u 1 )(1 − 4 u 2 ) 2 − ( u 1 + u 2 ) (105) = (1 − 4 u 1 ) + (1 − 4 u 2 ) − 2 p (1 − 4 u 1 )(1 − 4 u 2 ) 4 (106) = ( √ 1 − 4 u 1 − √ 1 − 4 u 2 ) 2 4 (107) ≥ 0 (108) and using (93), w e note that the low er b ound in (9 9) is r edundant. Therefore, from (93) and (104), w e hav e the follow ing feasible range f or the v ariable u in terms of u 1 and u 2 , f (2 u 1 , 2 u 2 ) ≤ u ≤ 1 − ( u 1 + u 2 ) (109) Com bining (88), (89) and (10 9), a set of feasible ( u 1 , u 2 , u ) is giv en as f o llo ws, P = n ( u 1 , u 2 , u ) : 0 ≤ u 1 ≤ 1 4 ; 0 ≤ u 2 ≤ 1 4 ; f (2 u 1 , 2 u 2 ) ≤ u ≤ 1 − ( u 1 + u 2 ) o (110) It should b e noted that the set P in ( 1 10) may not necess arily b e the smallest f easible set of a ll triples ( u 1 , u 2 , u ). Since we are intereste d in a maximization o v er these set o f triples, a p ossibly larger set P suffices. 8.2 A Simple Characterization of D B (1) P C and D B (2) P C Using the upp er b ounds on H ( X 1 | T ), H ( X 2 | T ), I ( X 1 ; Y | X 2 ), I ( X 2 ; Y | X 1 ) and I ( X 1 , X 2 ; Y ) in (62), (63), (73), (74 ) and (87) in terms of ( u 1 , u 2 , u ) along with a feasible set of triples P in (110), we obtain the fo llo wing tw o outer b ounds on the capacity region of the binary 16 additiv e noisy MAC -FB, starting fro m (31)-(3 3) and (3 4)-(36), D B (1) P C = [ ( u 1 ,u 2 ,u ) ∈P ( ( R 1 , R 2 ) : R 1 ≤ min  1 2 h ( u ) , h ( φ (2 u 1 ))  R 2 ≤ 1 2 h ( φ (2 u 2 )) R 1 + R 2 ≤ h  1 − u 2  ) (111) and D B (2) P C = [ ( u 1 ,u 2 ,u ) ∈P ( ( R 1 , R 2 ) : R 1 ≤ 1 2 h ( φ (2 u 1 )) R 2 ≤ min  1 2 h ( u ) , h ( φ (2 u 2 ))  R 1 + R 2 ≤ h  1 − u 2  ) (112) W e will plot these outer b ounds and their in tersection in Figure 4. In next section, w e will explicitly ch aracterize our upp er b ounds fo r the symmetric-rate p oin t on the capacity regio n of the binary additiv e noisy MA C-FB in consideration. 9 Explicit C haracterization of the Symmetric-rate Up- p er Bound F o r the binary additive noisy MA C-FB in consideration, it w as sho wn b y Kra mer [7] that t he symmetric-rate cut-set b ound is 0 . 4591 5 bits/transmission. It w as also sho wn in [7 ] that the Co v er-Leung achie v able symmetric-rate is 0 . 43 6 21 bits/transmission and it w as impro v ed to 0 . 43879 bits/transmission b y using superp osition co ding and binning with co de trees. F or completeness and comparison with existing b ounds, w e will first completely c haracterize our outer bound for the symmetric-rate b y pro viding the input distribution p ( t ) p ( x 1 | t ) p ( x 2 | t ) whic h ach iev es it. By symmetric-rate w e mean a rate R suc h that the rate pair ( R, R ) lies in the capacit y regio n of MA C- F B. F or the symmetric-rate, b oth D B (1) P C and D B (2) P C will yield the same upp er b ound. Hence, w e will f o cus on D B (1) P C . Using (111), w e are in terested in 17 obtaining the la rgest R o v er all ( u 1 , u 2 , u ) ∈ P suc h that R ≤ min  1 2 h ( u ) , h ( φ (2 u 1 ))  (113) R ≤ 1 2 h ( φ (2 u 2 )) (114) 2 R ≤ h  1 − u 2  (115) W e will sho w that a seemingly w eak er v ersion of the ab ov e b ound will impro v e up on the symmetric-rate cut-set b ound. W e will also sho w that the w eak er b ound is in fact the same as the ab o v e b ound, and its sole purp ose is the simplicit y of ev alua t io n and insigh t into the input distribution that att a ins it. W e first obta in a w eak ened v ersion of (113) as R ≤ min  1 2 h ( u ) , h ( φ (2 u 1 ))  ≤ h ( φ (2 u 1 )) (116) Next, consider (115) 2 R ≤ h  1 − u 2  (117) = h  1 2 − u 2  (118) ≤ h  1 2 − f (2 u 1 , 2 u 2 ) 2  (119) where (119) follo ws from (93 ) a nd the fact that the binary en t rop y function h ( s ) is mono- tonically increasing in s fo r s ∈ [0 , 1 2 ]. Com bining (114), (116) and (119), w e are in terested in t he largest R suc h that R ≤ max u 1 ,u 2 ∈ [0 , 1 4 ] min  h ( φ (2 u 1 )) , 1 2 h ( φ (2 u 2 )) , 1 2 h  1 2 − f (2 u 1 , 2 u 2 ) 2  (120) W e note that this upp er b ound on the symmetric-rate dep ends only on u 1 and u 2 , a nd therefore, we replace the f easible set P with u 1 , u 2 ∈ [0 , 1 4 ]. W e kno w that h ( φ ( s )) is concav e in s for s ∈ [0 , 1]. Hen ce, it follo ws that b oth h ( φ (2 u 1 )) and 1 2 h ( φ (2 u 2 )) are conca v e in u 1 and u 2 , resp ectiv ely , and hence concav e in the pair ( u 1 , u 2 ). W e also ha v e the follo wing lemma. Lemma 3 The function g ( u 1 , u 2 ) = 1 2 h  1 − f (2 u 1 , 2 u 2 ) 2  (121) is m onotonic al ly de cr e asing and jointly c onc ave in the p air ( u 1 , u 2 ) for u 1 , u 2 ∈ [0 , 1 4 ] . 18 Pro of: It suffices to show that for a fixed u 2 , the function g ( u 1 , u 2 ) is monoto nically decreas- ing in u 1 . Substituting the v a lue of f (2 u 1 , 2 u 2 ), w e ha v e g ( u 1 , u 2 ) = 1 2 h  1 − ( φ (2 u 1 ) + φ (2 u 2 ) − 2 φ (2 u 1 ) φ (2 u 2 )) 2  (122) = 1 2 h  1 2 − φ (2 u 2 ) 2 − φ (2 u 1 )(1 − 2 φ (2 u 2 )) 2  (123) No w using the fact that φ (2 s ) is increasing in s fo r s ∈ [0 , 1 4 ], we hav e that for u ′ 1 ≥ u 1 , φ (2 u ′ 1 ) ≥ φ (2 u 1 ). Moreo ve r, the following holds φ (2 u ′ 1 )(1 − 2 φ (2 u 2 )) 2 ≥ φ (2 u 1 )(1 − 2 φ (2 u 2 )) 2 (124) since φ (2 u 2 ) ≤ 1 2 . No w using the ab o v e inequalit y along with the fact that the binary en trop y function h ( s ) is increasing for 0 ≤ s ≤ 1 2 , w e hav e that for u ′ 1 ≥ u 1 , 1 2 h  1 2 − φ (2 u 2 ) 2 − φ (2 u 1 )(1 − 2 φ (2 u 2 )) 2  ≥ 1 2 h  1 2 − φ (2 u 2 ) 2 − φ (2 u ′ 1 )(1 − 2 φ (2 u 2 )) 2  (125) This show s that fo r a fixed u 2 , the function g ( u 1 , u 2 ) is monot onically decreasing in u 1 . As the function is symmetric in u 1 and u 2 , the monotonicit y of g ( u 1 , u 2 ) in ( u 1 , u 2 ) follows. T o show the concavit y of g ( u 1 , u 2 ) in the pa ir ( u 1 , u 2 ), w e first note from Lemma 2 that f (2 u 1 , 2 u 2 ) is join tly conv ex in the pair ( u 1 , u 2 ). W e define another function ξ ( u 1 , u 2 ) = 1 − f (2 u 1 , 2 u 2 ) 2 (126) Note that ξ ( u 1 , u 2 ) is join tly conca v e in t he pair ( u 1 , u 2 ). F urthermore, the binary en tropy function h ( s ) is conca ve and no ndecreas ing for s ∈ [0 , 1 2 ]. Hence, rewriting t he function g ( u 1 , u 2 ) as a comp osition of tw o functions, w e obtain g ( u 1 , u 2 ) = 1 2 h ( ξ ( u 1 , u 2 )) (127) F ro m the theory of comp osite functions [14], we kno w that a comp osite function f 1 ( f 2 ( s )) is conca v e in s if f 1 ( . ) is conca v e and nondecreasing and f 2 ( s ) is conca v e in s . Iden tifying f 1 ( . ) with h ( . ) and f 2 ( u 1 , u 2 ) with ξ ( u 1 , u 2 ), the conca vit y of g ( u 1 , u 2 ) in the pair ( u 1 , u 2 ) is established.  Therefore, all three functions in the min( . ) in (120) are conca v e in ( u 1 , u 2 ). In v oking the fact that the minimum of conca v e functions is concav e, we conclude that the maximum in (120) is unique. W e will no w sho w that the unique pair ( u ∗ 1 , u ∗ 2 ) that attains this maxim um satisfies the prop erty that h ( φ (2 u ∗ 1 )) = 1 2 h ( φ (2 u ∗ 2 )) = g ( u ∗ 1 , u ∗ 2 ). 19 F o r this purp ose, w e first c haracterize those pairs ( ˜ u 1 , ˜ u 2 ) suc h that the following holds, h ( φ (2 ˜ u 1 )) = 1 2 h ( φ (2 ˜ u 2 )) = g ( ˜ u 1 , ˜ u 2 ) (128) By using (128), w e obtain t w o equations f or ˜ u 1 and ˜ u 2 , as h ( φ (2 ˜ u 1 )) = 1 2 h  1 − φ (2 ˜ u 1 ) 3 − 2 φ (2 ˜ u 1 )  (129) φ (2 ˜ u 2 ) = 1 − φ (2 ˜ u 1 ) 3 − 2 φ (2 ˜ u 1 ) (130) F ro m (129), one can see that 2 ˜ u 1 is the unique solution s ∈ [0 , 1 2 ] of the equation h ( φ ( s )) = 1 2 h  1 − φ ( s ) 3 − 2 φ ( s )  (131) Obtaining t he optima l ˜ u 1 from the ab ov e equation is illustrated in F igure 3. The unique solutions ( ˜ u 1 , ˜ u 2 ) of (129) and (130) a r e ˜ u 1 = 0 . 086 063 , ˜ u 2 = 0 . 218333 (13 2) W e will no w show that this pair ( ˜ u 1 , ˜ u 2 ) yields the maxim um in (120). Returning to the maximization problem ( 120), first denote S as the region of allow able ( u 1 , u 2 ), S = n ( u 1 , u 2 ) : 0 ≤ u 1 ≤ 1 4 ; 0 ≤ u 2 ≤ 1 4 o (133) Also define a subset o f this region ˜ S = n ( u 1 , u 2 ) : u 1 ∈ ( ˜ u 1 , 1 4 ]; u 2 ∈ ( ˜ u 2 , 1 4 ] o (134) where ( ˜ u 1 , ˜ u 2 ) is giv en b y (132). W e will now sho w tha t the pair ( ˜ u 1 , ˜ u 2 ) yields the solution of the maximization problem in (120). Consider the followin g t w o cases, 1. If ( u 1 , u 2 ) ∈ ˜ S , then by Lemma 3, w e hav e that g ( u 1 , u 2 ) ≤ g ( ˜ u 1 , ˜ u 2 ), using whic h we obtain, min  h ( φ (2 u 1 )) , 1 2 h ( φ (2 u 2 )) , g ( u 1 , u 2 )  ≤ g ( u 1 , u 2 ) ≤ g ( ˜ u 1 , ˜ u 2 ) (135) 2. If ( u 1 , u 2 ) ∈ S \ ˜ S , w e either ha v e u 1 ≤ ˜ u 1 or u 2 ≤ ˜ u 2 or b o t h. Using this along with 20 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 s h((1− φ (s))/(3−2 φ (s)))/2 h( φ (s)) 2u 1 * Figure 3: Ch aracterization of the optimal u ∗ 1 . the fa ct that h ( φ (2 s )) is monoto nically increasing in s for s ∈ [0 , 1 4 ], w e obtain min  h ( φ (2 u 1 )) , 1 2 h ( φ (2 u 2 )) , g ( u 1 , u 2 )  ≤ h ( φ (2 ˜ u 1 )) (136) The ab o v e tw o cases sho w the follo wing, max u 1 ∈ [0 , 1 4 ] ,u 2 ∈ [0 , 1 4 ] min  h ( φ (2 u 1 )) , 1 2 h ( φ (2 u 2 )) , g ( u 1 , u 2 )  = h ( φ (2 ˜ u 1 )) (137) = 1 2 h ( φ (2 ˜ u 2 )) (138) = g ( ˜ u 1 , ˜ u 2 ) (139) Th us, the maxim um in (120) is obtained at ( u ∗ 1 , u ∗ 2 ) = ( ˜ u 1 , ˜ u 2 ). W e no w obtain a distribution p ( t ) p ( x 1 | t ) p ( x 2 | t ) whic h attains this symmetric-rate upp er b ound. Fix T to b e binary , and select the inv olv ed proba bilities as p 0 = p 1 = 1 2 (140) q 10 = 1 − q 11 = φ (2 u ∗ 1 ) (141) q 20 = 1 − q 21 = φ (2 u ∗ 2 ) (142) The reason for constructing suc h an input distribution is that, at this sp ecific distribution, 21 w e ha v e the follo wing exact equalities, H ( X 1 | T ) = h ( φ (2 u ∗ 1 )) (143) 1 2 H ( X 2 | T ) = 1 2 h ( φ (2 u ∗ 2 )) (144) 1 2 I ( X 1 , X 2 ; Y ) = g ( u ∗ 1 , u ∗ 2 ) (145) and w e a c hieve the outer b ound we dev elop ed with equalit y . Substituting the v alues of ( u ∗ 1 , u ∗ 2 ), w e obtain a distribution giv en b y , p 0 = p 1 = 1 2 (146) q 10 = 1 − q 11 = 0 . 095 109 (147) q 20 = 1 − q 21 = 0 . 322 050 (148) The ab ov e input distribution yields a symmetric-rate o f 0 . 4533 0 bits/transmission. Mor eov er, the u ∗ corresp onding to this distribution is giv en b y u ∗ = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t ) (149) = f (2 u ∗ 1 , 2 u ∗ 2 ) (150) = 0 . 355899 (151) where (150) is by construction of the input distribution p ( t, x 1 , x 2 ) and (15 1) is obtained b y substituting the distribution sp ecified in ( 1 46)-(148). Moreo v er, φ (2 u ∗ 2 ) < u ∗ < 1 2 , hence w e also ha v e that 1 2 h ( u ∗ ) ≥ 1 2 h ( φ (2 u ∗ 2 )) = h ( φ (2 u ∗ 1 )) (152) This sho ws that the w eak ened vers ion of the upp er b ound obtained in (120) is indeed tight and a binary auxiliary random v ariable T with uniform distribution ov er { 0 , 1 } is sufficien t to att a in this symmetric-rate upp er b ound. 10 Ev aluation of the Co v er-Leung Ac hiev able Rate Re- gion F o r completeness we will also obtain a simple c haracterization of the Co v er-Leung inner b ound for our binary a dditiv e noisy MA C-FB. F or this purp ose, we follo w a t w o-step ap- proac h. In the first step, we first obtain a n outer b o und on the ac hiev able rate region in terms of tw o v ariables ( u 1 , u 2 ). In the second step, w e sp ecify an input distribution, as a 22 function of ( u 1 , u 2 ), wh ic h ac hieve s the outer b ound. W e therefore arriv e a t an a lternate c haracterization of the Cov er-Leung ac hiev able rate region in terms of the v ariables ( u 1 , u 2 ). The Co v er-Leung ac hiev able rate region [3] is give n as, C L = n ( R 1 , R 2 ) : R 1 ≤ I ( X 1 ; Y | X 2 , T ) (15 3) R 2 ≤ I ( X 2 ; Y | X 1 , T ) (15 4) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) o (155) where t he random v ariables ( T , X 1 , X 2 , Y ) ha v e the join t distribution, p ( t, x 1 , x 2 , y ) = p ( t ) p ( x 1 | t ) p ( x 2 | t ) p ( y | x 1 , x 2 ) (156) and the ra ndom v ariable T is sub ject to a cardinality constraint of |T | ≤ min( |X 1 ||X 2 | + 1 , |Y | + 2). F or the binary , a dditive noisy MA C in consideration, the constraints in (153)-(15 5) b ecome, R 1 ≤ 1 2 H ( X 1 | T ) (157) R 2 ≤ 1 2 H ( X 2 | T ) (158) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y ) (159) W e will first o btain an o uter b ound on the region specified by (157)-(159) in terms of t w o v ariables ( u 1 , u 2 ). F or ev ery pair ( u 1 , u 2 ), w e will then sp ecify an input distribution whic h will attain this outer b ound. Note that the three constraints (157)-(159) are of similar form as in the case of D B (1) P C and D B (2) P C , a nd we pro ceed in a similar manner to obtain upp er b ounds on the three terms a bov e in terms o f u 1 and u 2 as, R 1 ≤ 1 2 h ( φ (2 u 1 )) (160) R 2 ≤ 1 2 h ( φ (2 u 2 )) (161) R 1 + R 2 ≤ h  1 − f (2 u 1 , u 2 ) 2  (162) where t he v ar iables ( u 1 , u 2 ) b elong to the set S defined in (1 3 3). Hence, an outer b ound on 23 the ra te region sp ecified b y (15 7)-( 1 59) is giv en a s O , where O = [ ( u 1 ,u 2 ) ∈S ( ( R 1 , R 2 ) : R 1 ≤ 1 2 h ( φ (2 u 1 )) R 2 ≤ 1 2 h ( φ (2 u 2 )) R 1 + R 2 ≤ h  1 − f (2 u 1 , u 2 ) 2  ) (163) Let ( u 1 , u 2 ) b e an y arbitr a ry pair whic h b elongs to S . Consider an input distribution f or whic h |T | = 2, and T is uniform ov er { 0 , 1 } and, p 0 = p 1 = 1 2 (164) q 10 = 1 − q 11 = φ (2 u 1 ) (165) q 20 = 1 − q 21 = φ (2 u 2 ) (166) F o r this input distribution, w e obtain the following exact equalities H ( X 1 | T ) = h ( φ (2 u 1 )) (167) H ( X 2 | T ) = h ( φ (2 u 2 )) (168) I ( X 1 , X 2 ; Y ) = h  1 − f (2 u 1 , 2 u 2 ) 2  (169) W e hav e thus sho wn that the outer b ound we obtained on the a chiev able rate region in terms of ( u 1 , u 2 ) can b e atta ined by a set of input distributions f or which t he inv olved auxiliary random v ariable T is binary and uniform. This in turn implies that a bina r y and uniform random v ariable T is sufficien t to c haracterize the entire Co v er-Leung achie v able rate region for the binary additiv e noisy MA C-FB. By v a rying ov er all suc h input distributions, or equiv alen tly , b y v arying ( u 1 , u 2 ) in the set S , w e obt a in the en tire Co v er-Leung ac hiev able rate region. W e should remark here that when ev a luating the D B P C b ound in the previous section for Z = X 1 and Z = X 2 , it w as not necessary to specify the distribution whic h ac hiev es the b ound, since it w as an outer b ound. On the other hand, when ev alua ting the Co v er-Leung b ound, since it is an a c hiev a bilit y , it is necessary to give a distribution whic h ac hiev es the b ound. The dependence bala nce b ounds corresp onding to t he parallel c hannel c hoices Z = X 1 and Z = X 2 , along with t he cut-set upp er b ound and the Co v er-Leung achie v able rate region are show n in Figure 4. It is in teresting to not e that our b ound impro v es up on the cut-set b ound a t all p oints where the Cov er-Leung ach iev able ra te region is strictly larger than the capacit y region without feedbac k. In other w ords, our b ound impro ves up on the cut-set b ound a t all p oints where feedbac k increases capacit y . 24 W e should remark that our c hoices of parallel c hannels; namely , Z = X 1 and Z = X 2 are the simplest ones whic h ensure that I ( X 1 ; X 2 | Y , Z , T ) = 0 but they yield fixed information leaks. W e b eliev e that by a more elab ora te choice of a parallel c hannel, i.e., by carefully selecting a parameterized para llel channel p + ( z | x 1 , x 2 , y , t ) suc h that I ( X 1 ; X 2 | Y , Z , T ) = 0, one w ould still b e able to restrict the input distributions to a conditionally indep enden t form and then optimize t he parameters of the pa r allel channel to minimize the information leak terms. This approach can p o t en tia lly improv e up on our outer b ound. 11 The Capacit y Region of the B inary Erasure MAC- FB The capacit y region of a class o f discrete memoryless MA C-F B w as c haracterized in [9] b y establishing a conv erse and it w as show n to b e eq ual to the Cov er-Leung ac hiev able rate region. This class of channe ls satisfy the prop erty that at least one of the c hannel inputs say X 1 , can b e written as a deterministic function of the other c hannel input X 2 and t he ch annel output Y . The binary erasure MAC, where Y = X 1 + X 2 , falls into this class of channe ls. In addition, the binary erasure MA C-FB is the noiseless version o f the binary additiv e noisy MA C-FB studied in this pap er. Willems sho w ed in [1 2] that a binary selection of auxiliary random v aria ble is sufficien t to o btain the sum-rate p oint of the capacity region o f the binary erasure MA C-FB. In this section, we will sho w that by using our results for comp osite functions whic h w ere presen ted in previous sections, it is p ossible to obtain a ll p oin ts on the b oundary of this capacity region using a binary auxiliary random v ariable. The feedbac k capa city region of this c hannel is giv en b y t he Cov er-Leung achiev able rat e region giv en in (15 3)-(155) whic h can b e simplified for the binary erasure MA C-FB as, R 1 ≤ H ( X 1 | T ) (170) R 2 ≤ H ( X 2 | T ) (171) R 1 + R 2 ≤ H ( Y ) (172) W e obtain three upp er b o unds on the expressions app earing in the b ounds (170)-(17 2). F ro m (62), we hav e, H ( X 1 | T ) ≤ h ( φ (2 u 1 )) (173) Similarly , we also ha v e H ( X 2 | T ) ≤ h ( φ (2 u 2 )) (174) 25 W e no w obtain an upp er b ound on H ( Y ), by first noting that, H ( Y ) = h (3) ( P Y (0) , P Y (1) , P Y (2)) (175) where P Y (0) = X t p t q 1 t q 2 t (176) P Y (1) = X t p t ( q 1 t + q 2 t − 2 q 1 t q 2 t ) (177) P Y (2) = X t p t (1 − q 1 t )(1 − q 2 t ) (178) No w, we use the fo llo wing inequality established in [12], h (3) ( a, b, c ) = 1 2 h (3) ( a, b, c ) + 1 2 h (3) ( c, b, a ) (179) ≤ h (3)  a + c 2 , b, a + c 2  (180) = h ( b ) + 1 − b (181) where (180) f ollo ws b y the conca vit y of t he entrop y function and b y the application of Jensen’s inequalit y [1 3]. Using (181) and contin uing f r o m (175), w e obtain H ( Y ) = h (3) ( P Y (0) , P Y (1) , P Y (2)) (182) ≤ h ( P Y (1)) + 1 − P Y (1) (183) = h ( u ) + 1 − u (184) where u is defined in (55). Using (173), (174) and (1 8 4), w e can write an outer b ound O 1 on the capacit y region as follo ws, O 1 = [ ( u 1 ,u 2 ,u ) ∈P O 1 ( u 1 , u 2 , u ) (18 5 ) where O 1 ( u 1 , u 2 , u ) = n ( R 1 , R 2 ) : R 1 ≤ h ( φ (2 u 1 )) R 2 ≤ h ( φ (2 u 2 )) R 1 + R 2 ≤ h ( u ) + 1 − u o (186) and the set P is defined in (110). W e will no w obtain a simpler characterization o f O 1 in 26 terms of tw o v aria bles ( u 1 , u 2 ) by sho wing that O 1 ≡ O 2 , where, O 2 = [ ( u 1 ,u 2 ) ∈S O 2 ( u 1 , u 2 ) (187) where O 2 ( u 1 , u 2 ) = n ( R 1 , R 2 ) : R 1 ≤ h ( φ (2 u 1 )) R 2 ≤ h ( φ (2 u 2 )) R 1 + R 2 ≤ h ( f (2 u 1 , 2 u 2 )) + 1 − f (2 u 1 , 2 u 2 ) o (188) The inclusion O 2 ⊆ O 1 is straightforw ard b y forcing u = f ( 2 u 1 , 2 u 2 ) in O 1 . W e will no w sho w that O 1 ⊆ O 2 . F or this purp ose, w e will need t he following lemma. Lemma 4 The function µ ( s ) = h ( s ) + 1 − s (189 ) is c onc av e in s fo r s ∈ [0 , 1 ] and takes its maximum val ue at s = 1 3 . Mor e over, the function µ ( s ) is incr e asing in s f or s ∈ [0 , 1 3 ] and de cr e asing in s for s ∈ [ 1 3 , 1] . The pro of of this lemma follows from the fact that b oth h ( s ) and − s are concav e in s . No w consider any arbitrary triple ( u 1 , u 2 , u ) ∈ P . W e can classify any suc h triple into one o f the follo wing cases: 1. If f (2 u 1 , 2 u 2 ) ≤ u ≤ 1 2 : f or any suc h ( u 1 , u 2 , u ), there exists a pair ( ¯ u 1 , ¯ u 2 ), suc h that u 1 ≤ ¯ u 1 ≤ 1 4 (190) u 2 ≤ ¯ u 2 ≤ 1 4 (191) u = f (2 ¯ u 1 , 2 ¯ u 2 ) (192) One suc h pair ( ¯ u 1 , ¯ u 2 ) can b e obtained as follo ws. Using the fact that for a fixed u 1 , f (2 u 1 , 2 u 2 ) is increasing in u 2 , w e select ¯ u 1 = u 1 and solve for u 2 ≤ ¯ u 2 ≤ 1 4 for whic h f (2 ¯ u 1 , 2 ¯ u 2 ) = u . The required ¯ u 2 is obta ined as, ¯ u 2 = 1 4  1 − (1 − 2 u ) 2 (1 − 4 u 1 )  (193) 27 F o r suc h a pa ir ( ¯ u 1 , ¯ u 2 ), the following inequalities hold, h ( φ (2 u 1 )) = h ( φ (2 ¯ u 1 )) (194) h ( φ (2 u 2 )) ≤ h ( φ (2 ¯ u 2 )) (195) h ( u ) + 1 − u = h ( f (2 ¯ u 1 , 2 ¯ u 2 )) + 1 − f (2 ¯ u 1 , 2 ¯ u 2 ) (196) 2. If f (2 u 1 , 2 u 2 ) ≤ 1 2 ≤ u ≤ 1 − ( u 1 + u 2 ), then we ha v e b y Lemma 4, h ( u ) + 1 − u ≤ h  1 2  + 1 − 1 2 (197) = 3 2 (198) No w consider the pair ( ¯ u 1 , ¯ u 2 ) = ( 1 4 , 1 4 ), fo r whic h w e ha v e f (2 ¯ u 1 , 2 ¯ u 2 ) = 1 2 . Hence w e ha v e that, h ( φ (2 u 1 )) ≤ h ( φ (2 ¯ u 1 )) = 1 (199) h ( φ (2 u 2 )) ≤ h ( φ (2 ¯ u 2 )) = 1 (200) h ( u ) + 1 − u ≤ h ( f (2 ¯ u 1 , 2 ¯ u 2 )) + 1 − f (2 ¯ u 1 , 2 ¯ u 2 ) = 3 2 (201) W e ha v e th us sho wn that for a n y triple ( u 1 , u 2 , u ), there exists a pair ( ¯ u 1 , ¯ u 2 ), suc h that O 1 ( u 1 , u 2 , u ) ⊆ O 2 ( ¯ u 1 , ¯ u 2 ), whic h in turn implies t ha t O 1 ⊆ O 2 , and consequen tly O 1 ≡ O 2 . Hence, we hav e an outer b ound on the capacit y region as giv en b y O 2 . The outer b ound O 2 is ev aluated o v er the set of pairs ( u 1 , u 2 ) suc h that u 1 , u 2 ∈ [0 , 1 4 ]. F o r a n y suc h a rbitrary pair ( u 1 , u 2 ), a n input distribution whic h achiev es the set of rat e pairs sp ecified by O 2 ( u 1 , u 2 ) is obtained b y selecting |T | = 2, and p 0 = p 1 = 1 2 (202) q 10 = 1 − q 11 = φ (2 u 1 ) (203) q 20 = 1 − q 21 = φ (2 u 2 ) (204) The set of rates a c hiev able b y the distribution sp ecified in ( 2 02)-(204) ar e obtained as, R 1 ≤ H ( X 1 | T ) = h ( φ (2 u 1 )) (205) R 2 ≤ H ( X 2 | T ) = h ( φ (2 u 2 )) (206) R 1 + R 2 ≤ H ( Y ) = h ( f (2 u 1 , 2 u 2 )) + 1 − f (2 u 1 , 2 u 2 ) (207) This sho ws that the capacity region of binary erasure MAC -FB can b e obtained b y a binary and unifo rm selection of the auxiliary random v aria ble T . The capacity region of 28 the binary erasure MA C with a nd without feedbac k and the cut-set b ound are illustrated in Figure 5 . It w as shown in [12 ] that the sum-rate p oin t on the b oundary of the capacit y region lies strictly b elo w the “tota l co op eration” line. This is equiv alent to saying that the cut-set b ound is not tigh t for the sum-rate p oin t. F rom o ur result, it is now clear that the cut-set b ound is not t igh t for asymmetric rate pairs either. In fact, it is not tigh t at a ll b oundary p oints where feedbac k increases capacit y . Moreo v er, our result also sho ws that a simple selection of binary and uniform T is suf- ficien t to ev aluate the b oundary of the capacit y regio n of binary erasure MA C-FB. Simple feedbac k s trategies f or a class of tw o user MA C-FB were dev elop ed in [4]. It was sho wn that for the binary erasure MA C, these feedbac k strategies yield all rate p oin ts for a binary selection of the auxiliary random v a riable T . Th us, our result sho ws that these feedbac k strategies are indeed optimal fo r the binary erasure MAC -FB and yield all rates on the b oundary o f its feedbac k capacit y region. 12 Concl usions In this pap er, we o bta ined a new outer b ound on the capacity region of a MAC -FB by using the idea of dep endence ba lance. W e considered a binary additiv e noisy MA C-FB for whic h it is kno wn that f eedback increases capacit y but the feedbac k capacit y regio n is not known. The b est known o uter b ound on the feedbac k capacity regio n o f this c hannel w as the cut- set b ound. W e used the dep endence ba la nce b ound to improv e up on the cut-set b ound at all p oin ts in the capacity region of this channel where f eedbac k increases capacit y . Our result is somewhat surprising once it is realized that the c hannel w e considered in this pap er is the discrete vers ion of the tw o-user Gaussian MA C-FB considered b y Ozaro w in [2] where the cut-set b ound w as show n to b e tight. Our outer bound is difficult to ev aluate due to an in volv ed auxiliary random v ariable T . F or binary inputs, the cardinalit y b ound on T is | T | ≤ 7 whic h mak es it intractable to ev aluate the outer b ound. W e o v ercome this difficult y b y making use of comp osite functions and their prop erties to obtain a simple c haracterization of our b ound. As a n application of the prop erties o f the comp osite functions dev elop ed in this pap er, w e ar e also able to completely characterize the Cov er-Leung achiev able rat e region for this c hannel. The capacity region of the binary erasure MAC-FB is known and it coincides with the Co v er-Leung ac hiev able rate region. Although the capacit y region is kno wn in principle, it is not known how to compute the en tire region, the difficult y arising a g ain due to the inv olve d auxiliary random v ariable. W e again mak e use of the comp osite functions to giv e an alternate c haracterization of the capacity region of the binary erasure MA C-FB. In addition, w e go on to show that a binar y and uniform auxiliary random v ariable selection is suffi cien t to ev aluate its feedback capacit y region. 29 References [1] A. P . Hekstra and F. M. J. Willems. Dep endence balance bounds for sin gle output t w o-w ay c hannels. IEEE T r ans. on I nformation The ory , 35(1):44–53 , January 1989 . [2] L. Ozaro w. The capacit y of the white Gaussian multiple access channel with feedbac k. IEEE T r ans. on Inform ation The ory , 30(4):623– 629, July 19 84. [3] T. M. Co v er and C. S. K. Leung. An ac hiev a ble ra te region for the m ultiple access c hannel with f eedbac k. IEEE T r ans. on Information The ory , 27(3):292 – 298, May 198 1 . [4] G. Kra mer. F eedbac k strategies for a class of tw o- user m ultiple access c hannels with feedbac k. IEEE T r ans. on Information The ory , 45(6):2054 –2059, Septem b er 1 999. [5] N. G aarder and J. W olf. The capacit y region of a multiple-acces s discrete memoryless c hannel can increase with feedbac k. IEEE T r ans. on Information The ory , 21:1 0 0–102, Jan 1975 . [6] J. P . M. Sch alkwijk and T. Kailath. A co ding sc heme for additive no ise c hannels with feedbac k-Part I: No bandwidth constrain t. IEEE T r ans. on Info rmation The ory , 12:172– 182, April 1966. [7] G. Kramer. Dir e cte d Inform ation for Channels with F e e db ack . Ph.D. dissertation, Swiss F ederal Institute of T ec hnology (ETH), Zuric h, Switzerland, 1 9 98. [8] S. I. Bross and A. Lapidot h. An improv ed a chiev able rate region for the discrete memo- ryless tw o- user m ultiple-access c hannel with noiseless feedbac k. IEEE T r ans. on Infor- mation The ory , 51(3):8 11–833, Marc h 2005. [9] F. M. J. Willems. The f eedback capacity region of a class of discrete memoryless multiple access channe ls. IEEE T r ans. on Information The ory , 28(1):93–95 , January 1982 . [10] G. Kramer. Capacit y results for the discrete me moryless net w ork. IEEE T r ans. on Information The ory , 49(1):4 –21, Jan. 2003. [11] A. J. Vinc k, W. L. M. Ho eks, and K. A. Post. On the capacity of the tw o-user M-ary m ultiple-access c hannel with feedbac k. IEEE T r ans. on Inform ation The ory , 31(4 ):540– 543, July 1985. [12] F. Willems. On multiple access channels with feedbac k. IEEE T r ans. on Information The ory , 30(6):842– 845, Nov ember 1984. [13] T. M. Co v er and J. A. Thomas. Elements of Information Th e ory . New Y or k:Wiley , 1991. [14] S. Boyd and L. V anden b erghe. Convex Optimization . Cam bridge Univers ity Press, 20 0 4. 30 0 0.1 0.2 0.3 0.4 0.5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 R 1 R 2 Cut−set bound MAC with no FB Cover−Leung bound DB with Z=X 1 DB with Z=X 2 Figure 4 . 1: Illustration of our b ounds for the capacit y of binary additive noisy MA C-FB. 0.3 0.35 0.4 0.45 0.5 0.3 0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5 R 1 R 2 Cut−set bound MAC with no FB Cover−Leung bound DB with Z=X 1 DB with Z=X 2 0.45330 Figure 4 . 2: An enlarged illustration of the p ortion of Fig ure 4 . 1 where feedbac k increases capacit y . 31 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 R 1 R 2 Cut−set bound Capacity region with FB Capacity region without FB Figure 5 . 1: Illustration of the capacity region of binar y erasure MA C-FB. 0.5 0.6 0.7 0.8 0.9 1 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 R 1 R 2 Cut−set bound Capacity region with FB Capacity region without FB Figure 5 . 2: An enlarged illustration of the p ortion of Fig ure 5 . 1 where feedbac k increases capacit y . 32