Source and Channel Coding for Correlated Sources Over Multiuser Channels

Source and Channel Coding for Correlate d Sources Ove r Multiuser Channels Deniz G ¨ und ¨ uz, Elza Erkip, Andrea Goldsmith, H. V incent Poor Abstract Source and chann el coding over multiuser chan nels in which receivers have access to c orrelated source side informa tion is consider ed. For se veral m ultiuser chann el models necessary an d sufﬁcient condition s for optimal separatio n of the source and cha nnel codes are obtained . In particular , the multiple access channe l, the compound multiple access channel, the interference channel and the tw o-way channel with co rrelated source s and cor related receiver si de in formation are considered , and the o ptimality of separation is shown to hold for ce rtain source an d side inf ormation stru ctures. In terestingly , the optim al separate sour ce and ch annel cod es identiﬁed fo r these mo dels are n ot necessarily the o ptimal code s for the underly ing so urce c oding o r the chan nel co ding pro blems. In oth er words, while sepa ration of the source and ch annel cod es is optimal, th e n ature of these o ptimal co des is impacted by the joint d esign criterion. I . I N T R O D U C T I O N Shannon’ s source-channel separation theorem states that, in p oint-to-point communication systems, a source can be reliably transmit ted over a channel if and o nly if the min imum sou rce The material in this paper was presented in part at the 2nd UCS D Information Theory and Applications W orkshop (IT A), San Diego, CA, Jan. 2007, at the Data Compression Conference (DCC), Snowbird, UT , March 2007, and at the IEE E International Symposium on Information Theory (ISI T), Nice, France, June 2007. This research was supported in part by t he U.S . National Science Foundation under Grants ANI-03-38807, C CF-04-30885, CCF-06-35177, CC F-07-28208, and CNS -06-25637 and D ARP A ITMANET program under grant 1105741 -1-TFIND and the AR O under MURI award W911NF-05-1-0246. Deniz G ¨ und ¨ u z is with t he Department of Electrical Engineering, Princeton Univ ersity , Princeton, NJ, 08544 and with the Department of Electrical Engineering, S tanford University , S tanford, CA, 94305 (email: dgunduz@princ eton.edu). Elza Erkip is with the Department of Electrical and Computer E ngineering, P olytechnic Institute of New Y ork Univ ersity , Brooklyn, NY 11201 (email: elza@poly .edu). Andrea Goldsmith is with the Department of Electrical Engineering, Stanford Uni versity , Stanford, CA, 94305 (email: andrea@systems.stanford .edu). H. V incent Poor is with t he Departmen t of Electrical Engineering, Princeton Un iv ersity , Princeton, NJ, 08544 (email: poor@princeton.ed u). 2 coding rate is below the channel capacity [1]. This m eans that a simple compariso n of the rates of the opti mal source and channel codes for the underlyin g source and channel distributions, respectiv ely , sufﬁces to conclude whether reliable transmi ssion is possible o r not. Furthermore, the separation t heorem dictates that the source and channel codes can be designed independently without loss of optimality . This t heoretical optimality of modularity has reinforced the notion of network layers, leading to the separate dev elo pment of source and channel coding aspects of a com munication system. The separation theorem hol ds for stati onary and er godic sources and channels u nder the usual information theoretic assum ptions of inﬁni te del ay and compl exity (see [2] for m ore general conditions under w hich separation holds). Howe ver , Shannon’ s source- channel separation theorem does not generalize to multiuser networks. Suboptimalit y of separation for m ultiuser sys tems was ﬁrst shown by Shanno n in [3], where an example of correlated source transmissi on ove r the two-way channel was p rovided. Later , a similar ob serv ation was m ade for transmitting correlated sources ov er multipl e access channels (MA Cs) in [4]. The example provided in [4] revea ls that comparison o f the Slepian-W olf source coding region [5] with the capacity region of the underlying M A C is not sufﬁ cient to decide whether reliabl e transmissio n can be realized. In general communication networks hav e mu ltiple sources a vailable at the network nodes, where the sou rce data m ust be transm itted to its destinati on in a lo ssless or los sy fashion. Some (potentially all) of t he nodes can transmit wh ile some (potentially all) of the nodes can recei ve noisy observa tions of the transmitted signals. The commun ication c hannel is characterized by a probabil ity transit ion matrix from the inputs of the transm itting termin als to the outp uts of the receiving terminals. W e assume that all the transmis sions share a common communicati ons medium; special cases such as orthogon al transm ission can be speciﬁed t hrough the channel transition matrix. The s ources com e from an arbitrary j oint di stribution, that is , t hey might be correlated. For this general model, the prob lem we address is to determine whether the sources can be transmitted lossless ly or within the required ﬁdelity to their destinatio ns for a giv en number of channel uses per source samp le (cupss ), whi ch is deﬁned to be the sourc e-channel rate of the joint source channel code. Equiv alently , we might want to ﬁnd the min imum source- channel rate t hat can b e achiev ed eit her reliably (for lossless reconst ruction) or wit h the required reconstruction ﬁdelity (for loss y reconstruction). The probl em of jo intly optimi zing s ource coding along with th e mult iuser channel cod ing DRAFT 3 in this very general set ting is extremely complicated. If the channels are assumed to be noi se- free ﬁnite capacity li nks, t he problem reduces to a m ultiterminal source coding problem [1]; alternativ ely , i f the sources are in dependent, then we mus t ﬁnd t he capacity region of a general communication network. Furthermore, considering t hat we do not have a separation result for source and channel coding e ven in the case of very sim ple n etworks, t he hope for solving this problem in the general setting is slight . Giv en the dif ﬁculty o f obtaining a general soluti on for arbitrary networks, our goal here is to analyze in detail sim ple, yet fundamental, building bl ocks of a larger net work, su ch as the mult iple access channel, t he broadcast channel , the interference channel and the t wo-wa y channel. Our focus in thi s work is on lo ssless transmiss ion and our goal i s to characterize the set of achiev abl e sou rce-channel rates for these canonical networks. Four fundamental questions that need to b e addressed for each model can be stated as foll o ws: 1) Is it possib le to characterize th e optim al source-channel rate o f th e network (i.e., the mini- mum number of channel uses per so urce samp le (cups s) requi red for lossless t ransmission) in a computable way? 2) Is it possible to achie ve the optimu m source-channel rate by statistically independent source and channel codes? By st atistical independent source and channel codes, we mean that the source and the channel codes are designed solely based on the distri butions of the source and the channel distributions, respective ly . In general, these codes need no t b e th e optimal codes for the underlying sources or the channel. 3) Can we determine the optimal sou rce-channel rate by si mply comparing t he source coding rate region with the capacity region? 4) If the comparison of these canoni cal regions is not su f ﬁcient to obtain the optim al s ource- channel rate, can we identi fy alt ernativ e ﬁnite dimensional source and channel rate regions pertaining to t he source and channel dis tributions, r espectiv ely , whose com parison provides us t he necessary and sufﬁcient conditions for the achiev ability of a source-channel rate? If the answer to q uestion (3) is afﬁrmati ve for a given setup, t his would maintain the opt imality of the layered approach described earlier , and would correspond t o the mul tiuser version of Shan- non’ s source-channel separation theorem. Howe ver , even when this classical layered approach is suboptimal , we can still obtain modularity in the system desig n, if the ans wer t o question DRAFT 4 (2) is afﬁrmati ve, in which case the optimal source-channel rate can b e achiev ed by statistically independent sou rce and channel codes, without taking the join t distribution into account. In t he point-to-po int setting, th e answer to questi on (3) is afﬁr mative , that is, the minim um source-channel rate i s simply t he rati o of the source entropy to the channel capacity; hence these two numbers are all we need t o i dentify the necessary and su f ﬁcient conditions for the achie va bility of a source-channel rate. Therefore, a source code th at meets the entropy bound when us ed with a capacity achieving channel code result s in t he best source-channel rate. In multiuser scenarios, we need to compare more than two numbers. In classical S hannon separation, it i s required that t he intersectio n of the source coding rate region for the give n sources and the capacity region of the underlyi ng multiuser channel is not empty . Thi s would deﬁnitely lead to modular so urce and chann el code design without sacriﬁcing opti mality . Howe ver , we show in this work th at, i n various mult iuser scenarios, even if this is not the case for the canonical source coding rate region and the capacity region, it m ight still be po ssible to i dentify alternative ﬁnite dimensional rate regions for the sources and the channel, respectiv ely , su ch that comparison of t hese rate regions provide the necessary and sufﬁcient condi tions for the achiev abilit y of a source-channel rate. Hence, the answer to q uestion (4) can be afﬁrmati ve ev en if the answer to question (3) is negativ e. Furthermore, we show that in those cases we also hav e an afﬁrmati ve answer to question (2), that is , statisti cally independent source and channel codes are optim al. Follo wing [6], we will u se the fol lowing deﬁnitions t o diff erentiate between t he t wo types of source-channel separation. Informati onal s eparation refers t o classical separation in th e Shannon sense, i n which concatenating opti mal source and channel codes for the underlyin g source and channel dist ributions result in the optim al sou rce-channel codi ng rate. Equiva lently , in informational separation, comp arison of the u nderlying source cod ing rate region and t he channel capacity region is sufﬁcient to ﬁnd the opt imal source-channel rate and the answer to question (3) is af ﬁrmativ e. Operational separation , on the other hand, refers to stati stically ind ependent source and channel codes that are n ot necessarily the optimal codes for the underlying source or the channel. Op timality of operation al separation allows the comparison of more general source and channel rate regions to provide necessary and sufﬁcient conditions for achiev abi lity of a so urce-channel rate, wh ich sug gests an afﬁrmati ve answer to questi on (4). These source and channel rate regions are required to be dependent solely on the source and the channel distributions, respective ly; howe ver , these regions need not be the canonical source coding DRAFT 5 rate region or t he channel capacity region. Hence, th e source and channel codes that achieve diffe rent p oints of these two regions will be statis tically ind ependent, providing an af ﬁrmativ e answer to questi on (2), while individually they may not be the opti mal source or channel codes for the underlying source comp ression and channel coding problems. Note that the class of codes satis fying operatio nal separation is larger than that satisfyi ng informational separation. W e should remark here t hat we are not providing precise mathematical deﬁnitions for operational and informati on s eparation. O ur goal i s to p oint out the limitatio ns of t he classical separation approach based on the direct comparison of so urce coding and channel capacity regions. This paper provides answers to the four fundamental questi ons abo ut source-channel coding posed above for some special mult iuser networks and source structures. In particular , we consider correlated sources av ailable at mu ltiple t ransmitters com municating with receiv ers that hav e correlated side information . Our contributions can be summarized as follows. • In a multi ple access channel we sho w that informati onal separation hol ds i f the so urces are independent given the recei ver side information. Th is is different from the pre vious separation results [7]- [9] in t hat we sho w th e o ptimality of separation for an arbit rary multiple access channel u nder a special source s tructure. W e also prove that the optimality of informational separation continue to hold for in dependent sou rces i n the presence of correlated side information at the receive r , giv en whi ch the sources are correlated. • W e characterize an achiev able sou rce-channel rate for compound m ultiple access channels with side information, which is sho w n to be opti mal for som e special scenarios. In particular , optimalit y hol ds either when each user’ s source is independent from the ot her source and one of the side information sequences, or when there is no multiple access int erference at the receiv ers. For t hese cases we argue that operational s eparation is optimal. W e further show the opti mality of inform ational separation when t he t wo sources are i ndependent given the side i nformation common to both receiv ers. Note that t he compound mul tiple access channel mo del combi nes both the mu ltiple access channels with correlated sources and the broadcast channels with correlated side informati on at the receiv ers. • For an interference channel with correlated side in formation, we ﬁrst deﬁne the str o ng sour ce-channel interfer ence conditions, which provide a generalization of the usual strong interference condi tions [10]. Our results show the opt imality of op erational separation under strong sou rce-channel interference condit ions for certain source structures. DRAFT 6 • W e consider a two-way channel with correlated sources. The achiev able scheme for com- pound M A C can also be us ed as an achie va ble coding scheme in wh ich t he users do not exploit their channel outputs for channel encoding (‘restricted encoders’). W e g eneralize Shannon’ s outer bou nd for t wo-wa y channels to correlated sources. Overall, our results characterize the necessary and s uf ﬁcient conditions for reliable transmi s- sion of correlated s ources over various mul tiuser networks, hence answering question (1) for those scenarios. In these cases, the optimal performance is achieve d by statistically independent source and channel codes (by either informational or operational separation), thus pro mising a lev el of modularit y e ven when sim ply concatenating optimal source and chann el codes is suboptim al. Hence, for t he cases where we provide the optimal source-channel rate, we answer questions (2), (3) and (4) as well. The remainder of the paper i s organized as follows. W e re v ie w the prior work on j oint source- channel coding for multiuser syst ems in Section II, and t he notation s and the technical tools t hat will be used throughout the paper in Section III. In Section IV, we introduce the system model and the deﬁnitions. The next four sections are dedicated to t he analysis of s pecial cases of the general system model. In particular , we cons ider mult iple access channel mo del in Section V, compound mult iple access channel model in Section VI, interference channel m odel in Section VII and ﬁnally the two-way channel model i n Section VIII. Ou r conclusions can b e found in Section IX followed by the App endix. I I . P R I O R W O R K The existin g l iterature provides limited answers to the four questions stated in Section I i n speciﬁc settings. For the MA C with correlated sources, ﬁnite-letter sufﬁcient conditi ons for achie va bility of a source-channel rate are giv en in [4] in an attempt to resolve t he ﬁrst probl em; howe ver , these conditio ns are later shown not to be necessary by Dueck [11]. The correlation pr eserving mappi ng technique of [4] used for achiev ability is later extended to so urce coding with s ide informati on via m ultiple access channels in [12], to broadcast channels with correlated sources in [13], and to interference channels in [14]. In [15 ], [16] a graph theoretic frame work wa s used to achie ve imp rove d source-channel rates for transmit ting correlated sources over mul tiple access and broadcast channels, respectively . A new data processin g inequali ty was proved in [17] DRAFT 7 that is used to derive new necessary conditio ns for reliable transmissi on of correlated sources over MA Cs. V arious special classes of source-channel pairs have been st udied in the li terature in an effort to resolve the third question above, loo king for the most general class of s ources for which the comparison of the un derlying source cod ing rate region and the capacity region is suf ﬁcient to determine the achie va bility of a source-channel rate. Optimalit y of separation in this classical sense is proved for a network o f independent, non-interfering channels in [7]. A special class of the MAC , called the asymmetric MA C, in which o ne of t he s ources i s ava ilable at both encoders, is considered in [8] and t he class ical s ource-channel separation optimality is shown to hold wit h or without causal p erfect feedback at either or both of the transm itters. In [9], it is shown t hat for the class of MA Cs for wh ich the capacity region cannot be enlarged by cons idering correlated channel i nputs, cl assical s eparation is opti mal. Note that all of t hese result s hold for a special class of MA Cs and arbitrary source correlations. There have also been results for joi nt source-channel codes in broadcast channels . Speciﬁcally , in [6], T uncel ﬁnds the optimal source-channel rate for broadcasting a common source to mult iple recei vers having ac cess to dif ferent correlated side information sequences, thus answering the ﬁrst question. Thi s work also shows that the comparison of th e broadcast channel capacity region and the minimum source coding rate region is not sufﬁcient to decide whether reliable transmiss ion is possible. Therefore, the classical informat ional source-channel separation, as stated in the third question, does not ho ld in this setup. T uncel also answers t he second and fourth questi ons, and su ggests that we can achieve the op timal source-channel rate by so urce and channel codes that are st atistically independent, and that, for the achiev ability of a source-channel rate b , th e intersection of two regions, one sol ely depending on the so urce distributions, and a second one solely depend ing on the channel dis tributions, is necessary and sufﬁcient. The codes proposed in [6] consist of a source encoder that does n ot use the correlated s ide information, and a joint source-channel decoder; hence th ey are not stand-alone s ource and channel codes 1 . Thus the techniques i n [6] require the design of new codes appropriat e for joint decoding with the side 1 Here we note that the joint source-channel decoder proposed by Tu ncel in [6] can also be implemented by separate source and channel decoders in w hich the channel decoder is a list decoder [19 ] that outputs a li st of possible channel inputs. Howe ver , by st and-alone source and channel codes, we mean unique decoders that produce a single code word output, as i t is understood in the classical source-channel separation theorem of Shannon. DRAFT 8 information; howe ver , it is shown in [18] that the same performance can b e achieved by using separate source and channel codes with a speciﬁc message passing m echanism between the source/channel encoders/decoders. Therefore we can use existing near-optimal codes to achieve the t heoretical bound. Broadcast channel in the presence of receiv er message side information, i.e., messages at the transmit ter known partially or totally at one of the receivers, is also stud ied from the perspectiv e of achiev able rate regions in [20] - [23]. The problem of broadcasting wi th receiver side information is also encountered in the t wo-wa y relay channel problem studied in [24], [25]. I I I . P R E L I M I N A R I E S A. Notation In the rest of the paper we adopt th e foll owing n otational con ventions. Random va riables will be denoted by capit al lett ers while their realizations will be denoted by the respective lower case letters. The alphabet of a scalar rando m var iable X will be denoted by t he corresponding calligraphic l etter X , and t he alphabet of the n -length vectors over the n -fold Cartesian product by X n . The cardinalit y of set X will be denoted b y |X | . The random vector ( X 1 , . . . , X n ) will be denoted by X n while the vector ( X i , X i +1 , . . . , X n ) by X n i , and their realizations , respectiv ely , by ( x 1 , . . . , x n ) or x n and ( x i , x i +1 , . . . , x n ) or x n i . B. T ypes and T ypical Sequ ences Here, w e brieﬂy revie w the noti ons of types and strong typi cality that will be used in the paper . Given a distribution p X , the type P x n of an n -tu ple x n is the empirical dis tribution P x n = 1 n N ( a | x n ) where N ( a | x n ) is t he number of occurances of the l etter a i n x n . The set of all n -tuples x n with type Q i s called the type class Q and denoted by T n Q . The set of δ -strongly t ypical n -tuples according to P X is denoted by T n [ X ] δ and is deﬁned by T n [ X ] δ =  x ∈ X n :     1 n N ( a | x n ) − P X ( a )     ≤ δ ∀ a ∈ X and N ( a | x n ) = 0 wheneve r P X ( x ) = 0  . The deﬁnitions of type and strong t ypicality can be extended to joint and condit ional d istri- butions in a simi lar mann er [1]. The following results concerning typical sets will be used in DRAFT 9 Channel P S f r a g r e p l a c e m e n t s S m 1 S m 2 X n 1 X n 2 Y n 1 Y n 2 W m 1 W m 2 ( ˆ S m 1 , 1 , ˆ S m 1 , 2 ) ( ˆ S m 2 , 1 , ˆ S m 2 , 2 ) p ( y 1 , y 2 | x 1 , x 2 ) T ransm itter 1 T ransm itter 2 Recei ver 1 Recei ver 2 Fig. 1. The general system model for transmitting correlated sources over multi user channels with correlated side information. In the MA C scenario, we hav e only one recei ver Rx 1 ; in the compound MAC scenario, we hav e two r ecei vers which want to recei ve both sources, while in the interference channel scenario, we have two receiv ers, each of which wants to receiv e only i ts o wn source. The compound MAC model reduces t o the “restricted” two-way channel model when W m i = S m i for i = 1 , 2 . the sequ el. W e have     1 n log | T n [ X ] δ | − H ( X )     ≤ δ |X | (1) for suf ﬁciently l ar ge n . Gi ven a join t distribution P X Y , if ( x i , y i ) is drawn independent and identically distri b uted (i.i .d.) with P X P Y for i = 1 , . . . , n , where P X and P Y are the m ar ginals, then Pr { ( x n , y n ) ∈ T n [ X Y ] δ } ≤ 2 − n ( I ( X ; Y ) − 3 δ ) . (2) Finally , for a joint distribution P X Y Z , if ( x i , y i , z i ) is drawn i.i .d. with P X P Y P Z for i = 1 , . . . , n , where P X , P Y and P Z are t he marginals, then Pr { ( x n , y n , z n ) ∈ T n [ X Y Z ] δ } ≤ 2 − n ( I ( X ; Y ,Z )+ I ( Y ; X,Z )+ I ( Z ; Y ,X ) − 4 δ ) . (3) I V . S Y S T E M M O D E L W e introduce the most general s ystem model here. Throughout the paper we consider various special cases, where t he restrictions are stated explicitly for each case. W e consider a network of two transm itters Tx 1 and Tx 2 , and two receiv ers Rx 1 and Rx 2 . For i = 1 , 2 , the transmitter Tx i observes the output of a discrete memoryl ess (DM) source S i , while the receive r Rx i observes DM side information W i . W e assume that the source and the DRAFT 10 side informati on sequences, { S 1 ,k , S 2 ,k , W 1 ,k , W 2 ,k } ∞ k =1 are i.i.d. and are drawn according to a joint probabil ity mass functi on (p.m. f.) p ( s 1 , s 2 , w 1 , w 2 ) over a ﬁnite alph abet S 1 × S 2 × W 1 × W 2 . The transmi tters and the recei vers all know t his joi nt p.m.f., but hav e no di rect access to each other’ s informatio n source or the si de information. The transmitter Tx i encodes its source vector S m i = ( S i, 1 , . . . , S i,m ) into a channel codeword X n i = ( X i, 1 , . . . , X i,n ) us ing the encodi ng function f ( m,n ) i : S m i → X n i , (4) for i = 1 , 2 . These code words are transmitted over a DM channel to th e receiver s, each o f whi ch observes the ou tput vector Y n i = ( Y i, 1 , . . . , Y i,n ) . The in put and o utput alphabets X i and Y i are all ﬁnite. The DM channel is characterized by the condi tional d istribution P Y 1 ,Y 2 | X 1 ,X 2 ( y 1 , y 2 | x 1 , x 2 ) . Each recei ver is interested in one or both of the sources depending on the scenario. Let receiver Rx i form th e est imates of the source vectors S m 1 and S m 2 , denoted by ˆ S m i, 1 and ˆ S m i, 2 , based on its recei ved s ignal Y n i and the side informatio n vector W m i = ( W i, 1 , . . . , W i,m ) us ing the decodi ng function g ( m,n ) i : Y n i × W m i → S m 1 × S m 2 . (5) Due to the reliable t ransmission requirement, the reconstruction alphabets are the same as the source alphabets . In the MA C s cenario, there is only one recei ver Rx 1 , which wants to receiv e both of the sources S 1 and S 2 . In the compound MAC s cenario, both recei vers want to receiv e both sources, while in the int erference channel scenario, each recei ver wants to receive only its own transm itter’ s source. The two-way channel s cenario cannot be o btained as a special case o f the above general m odel, as the receiv ed channel output at each user can be used to generate channel inp uts. On th e other hand, a “restricted” two-way channel model, in which the past channel output s are only us ed for decoding, is a sp ecial case of the above compound channel model with W m i = S m i for i = 1 , 2 . Based on the decodi ng requirements, the error probabil ity of the system, P ( m,n ) e will be deﬁned separately for each mo del. Next, we deﬁne the so urce-channel rate of the s ystem. Deﬁnition 4.1: W e s ay that source-channel rate b is achievable i f, for ever y ǫ > 0 , there exist positive integers m and n with n/m = b for which we hav e encoders f ( m,n ) 1 and f ( m,n ) 2 , and decoders g ( m,n ) 1 and g ( m,n ) 2 with decoder o utputs ( ˆ S m i, 1 , ˆ S m i, 2 ) = g i ( Y n i , W m i ) , such that P ( m,n ) e < ǫ . DRAFT 11 V . M U LT I P L E A C C E S S C H A N N E L W e ﬁrst consider t he m ultiple access channel, in which we are interested i n the reconstruction at recei ver Rx 1 only . For encoders f ( m,n ) i and a decoder g ( m,n ) 1 , the probability of error for th e MA C is deﬁned as follows: P ( m,n ) e , P r { ( S m 1 , S m 2 ) 6 = ( ˆ S m 1 , 1 , ˆ S m 1 , 2 ) } = X ( s m 1 ,s m 2 ) ∈S m 1 ×S m 2 p ( s m 1 , s m 2 ) P { ( ˆ s m 1 , 1 , ˆ s m 1 , 2 ) 6 = ( s m 1 , s m 2 ) | ( S m 1 , S m 2 ) = ( s m 1 , s m 2 ) } . Note that this model is m ore general than that of [4] as it considers the av ailabil ity of correlated side i nformation at th e receiv er [29]. W e ﬁrst generalize the achie va bility scheme of [4] to ou r model by using the correlation preserving mapping technique of [4], and limi ting the source- channel rate b to 1 . Ext ension to other rates i s pos sible as in Theorem 4 of [4]. Theor em 5.1: Consider arbitrarily correlated sources S 1 and S 2 over th e DM MAC wit h recei ver side informatio n W 1 . Source-channel rate b = 1 is achiev abl e if H ( S 1 | S 2 , W 1 ) < I ( X 1 ; Y 1 | X 2 , S 2 , W 1 , Q ) , H ( S 2 | S 1 , W 1 ) < I ( X 2 ; Y 1 | X 1 , S 1 , W 1 , Q ) , H ( S 1 , S 2 | U, W 1 ) < I ( X 1 , X 2 ; Y 1 | U, W 1 , Q ) , and H ( S 1 , S 2 | W 1 ) < I ( X 1 , X 2 ; Y 1 | W 1 ) , for som e joint d istribution p ( q , s 1 , s 2 , w 1 , x 1 , x 2 , y 1 ) = p ( q ) p ( s 1 , s 2 , w 1 ) p ( x 1 | q , s 1 ) p ( x 2 | q , s 2 ) p ( y 1 | x 1 , x 2 ) and U = f ( S 1 ) = g ( S 2 ) is the comm on part of S 1 and S 2 in the sense of G ` acs and K ¨ orner [26]. W e can bound the cardinality of Q by min {|X 1 | · |X 2 | , |Y |} . W e do not give a proo f here as it closely resembles th e on e in [4]. Note that correlation am ong the sources and the side informati on both condenses the left hand side of the above in equalities, DRAFT 12 and enlar ges their right hand sid e, com pared to transmitting in dependent sources. Whi le the reduction i n entropies on the left hand side is due to Slepian-W olf sou rce coding, the increase in the rig ht hand side i s m ainly due to the poss ibility of generating correlated channel codew ords at the t ransmitters. Applying distributed source coding followed by MA C channel coding, whil e reducing the redundancy , would also lead to the loss of possible correlation among th e channel code words. Howe ver , when S 1 − W 1 − S 2 form a Markov chain, that is, the two sources are independent given t he sid e informati on at the receiver , the receiver already has access to the correlated part of the sources and it is not clear whether addit ional channel correlation would help. The following theorem s uggests t hat channel correlatio n preservation is not n ecessary in this case and so urce-channel separation i n the informational sense is optim al. Theor em 5.2: Consider transmissi on of arbitrarily correlated sources S 1 and S 2 over t he DM MA C wi th receiver side information W 1 , for whi ch the Markov relati on S 1 − W 1 − S 2 holds. Informational separation is opti mal for this setup , and the source-channel rate b is achiev able if H ( S 1 | W 1 ) < b · I ( X 1 ; Y 1 | X 2 , Q ) , (6a) H ( S 2 | W 1 ) < b · I ( X 2 ; Y 1 | X 1 , Q ) , (6b) and H ( S 1 | W 1 ) + H ( S 2 | W 1 ) < b · I ( X 1 , X 2 ; Y 1 | Q ) , (6c) for som e joint d istribution p ( q , x 1 , x 2 , y 1 ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) p ( y 1 | x 1 , x 2 ) , (7) with |Q| ≤ 4 . Con versely , if the source-channel rate b is achieva ble, t hen the inequalit ies in (6) hold with < replaced by ≤ for s ome joint distribution of the form given in (7). Pr oof: W e start with the proof of the direct part. W e use Slepian-W olf source coding followed by multiple access channel coding as the achiev abilit y scheme; howe ver , the error probability analysis needs to be outlined carefully since for the rates wi thin the rate region characterized by the ri ght-hand side o f (6) we can achiev e arbitrarily small averag e err or pr obabi lity rather than the maximum err or p r obabil it y [1]. W e bri eﬂy out line the code generation and encodi ng/decoding steps. DRAFT 13 Consider a rate pair ( R 1 , R 2 ) sati sfying H ( S 1 | W 1 ) < R 1 < b · I ( X 1 ; Y 1 | X 2 , Q ) , (8a) H ( S 2 | W 1 ) < R 2 < b · I ( X 2 ; Y 1 | X 1 , Q ) , (8b) and H ( S 1 | W 1 ) + H ( S 2 | W 1 ) < R 1 + R 2 < b · I ( X 1 , X 2 ; Y 1 | Q ) . (8c) Code generation: At transmitter k , k = 1 , 2 , independently ass ign ev ery s m i ∈ S m i to one of the 2 mR k bins wi th uni form dist ribution. Denote the bin index of s m k by i k ( s m k ) ∈ { 1 , . . . , 2 mR k } . This const itutes the Slepian-W olf source code. Fix p ( q ) , p ( x 1 | q ) and p ( x 2 | q ) such th at the conditio ns in (6) are satisﬁed. Generate q n by choosing q i independently from p ( q ) for i = 1 , . . . , n . For each source bin index i k = 1 , . . . , 2 mR k of transmi tter k , k = 1 , 2 , generate a channel codeword x n k ( i k ) by cho osing x k i ( i k ) independ ently from p ( x k | q i ) . This constitutes the MA C code. Encoders: W e use t he abov e separate source and the chann el codes for encoding. The source encoder k ﬁnds the b in index of s m k using the Slepian-W olf sou rce code, and forwards it to the channel encoder . The channel encoder transmits t he codeword x n k corresponding to the source bin index usi ng the MAC code. Decoder: W e use separate so urce and channel decoders. Upon recei ving y n 1 , the channel decoder tries to ﬁnd t he in dices ( i ′ 1 , i ′ 2 ) such that the correspondi ng channel codew o rds satisfy ( q n , x n 1 ( i ′ 1 ) , x n 2 ( i ′ 2 )) ∈ T n [ QX 1 X 2 Y ] δ . If one such p air is found, call it ( i ′ 1 , i ′ 2 ) . If no or m ore than one su ch pair i s found, declare an error . Then these in dices are provided to t he source decoder . Source decoder tries to ﬁnd ˆ s m 1 ,k such that i k ( ˆ s m k ) = i ′ k and ( ˆ s m k , W m 1 ) ∈ T m [ S k W 1 ] δ . If one su ch pair is found, it i s d eclared as th e output. Otherwise, an error i s declared. Pr obabi lity of err or a nalysis: For brevity of t he expressions, we deﬁne s = ( s m 1 , s m 2 ) , S = ( S m 1 , S m 2 ) and ˆ s = ( ˆ s m 1 , 1 , ˆ s m 1 , 2 ) . The indices corresponding to the sources are denoted by i = ( i 1 ( s m 1 ) , i 2 ( s m 2 )) , and the indices estimated at the channel decoder are denoted by i ′ = ( i ′ 1 , i ′ 2 ) . DRAFT 14 The average probability of error can be written as follows: P ( m,n ) e , X s P { ˆ s 6 = s | S = s } p ( s ) = X s [ P { ˆ s 6 = s | i = i ′ , S = s } p ( i = i ′ | S = s ) + P { ˆ s 6 = s | i 6 = i ′ , S = s } p ( i 6 = i ′ | S = s )] p ( s ) ≤ X s [ P { ˆ s 6 = s | i = i ′ , S = s } + p ( i 6 = i ′ | S = s )] p ( s ) = X s P { ˆ s 6 = s | i = i ′ , S = s } p ( s ) + X s p ( i 6 = i ′ | S = s ) p ( s ) (9) Now , i n (9) the ﬁrst s ummation is the average error probability g iv en the fact that the recei ver knows the indices correctly . This can be made arbitrarily small with increasing m , which follows from t he Slepian-W olf theorem. The s econd term in (9) is t he a vera ge error prob ability for the indices aver aged over all source pairs. Thi s can als o be written as X s p ( i 6 = i ′ | S = s ) p ( s ) = X i p ( i 6 = i ′ , I = i ) = X i p ( i 6 = i ′ | I = i ) p ( I = i ) = 1 2 m ( R 1 + R 2 ) X i p ( i 6 = i ′ | I = i ) (10) where (10) follows from the uniform assignment of the bin indices in the creation of t he source code. N ote that (10) is the av erage error probability expression for the MA C code, and we know that it can also be made arbitraril y small with increasing m and n under the condit ions of the theorem [1]. W e note here that for b = 1 the direct part can also be obtained from Theorem 5.1. For this , we ignore the comm on p art of t he sources and choose th e channel i nputs independent of the source dis tributions, that is, we choose a joint d istribution of the form p ( q , s 1 , s 2 , w 1 , x 1 , x 2 , y 1 ) = p ( q ) p ( s 1 , s 2 , w 1 ) p ( x 1 | q ) p ( x 2 | q ) p ( y 1 | x 1 , x 2 ) . From th e conditional independence of t he sources giv en the recei ver side information , both the left and the ri ght hand s ides of th e condi tions in Th eorem 5 .1 can b e si mpliﬁed t o the su f ﬁciency conditions of Theorem 5.2. W e next prove the con verse. W e assume P ( m,n ) e → 0 for a sequence of encoders f ( m,n ) i ( i = 1 , 2 ) and decoders g ( m,n ) as n, m → ∞ with a ﬁxed rate b = n/m . W e wi ll use Fano’ s DRAFT 15 inequality , which states H ( S m 1 , S m 2 | ˆ S m 1 , 1 , ˆ S m 1 , 2 ) ≤ 1 + mP ( m,n ) e log |S 1 × S 2 | , , mδ ( P ( m,n ) e ) , (11) where δ ( x ) is a non-negative function that approaches zero as x → 0 . W e also o btain H ( S m 1 , S m 2 | ˆ S m 1 , 1 , ˆ S m 1 , 2 ) ≥ H ( S m 1 | ˆ S m 1 , 1 , ˆ S m 1 , 2 ) , (12) ≥ H ( S m 1 | Y n 1 , W m 1 ) , (13) where the ﬁrst inequality fol lows from the chain rule of entropy and the nonnegativity of the entropy funct ion for discrete sources, and th e second inequality follows from th e data processing inequality . Then we ha ve, for i = 1 , 2 , H ( S m i | Y n 1 , W m 1 ) ≤ mδ ( P ( m,n ) e ) . (14) W e hav e 1 n I ( X n 1 ; Y n 1 | X n 2 , W m 1 ) ≥ 1 n I ( S m 1 ; Y n 1 | W m 1 , X n 2 ) , (15) = 1 n [ H ( S m 1 | W m 1 , X n 2 ) − H ( S m 1 | Y n 1 , W m 1 , X n 2 )] , (16) = 1 n [ H ( S m 1 | W m 1 ) − H ( S m 1 | Y n 1 , W m 1 , X n 2 )] , (17) ≥ 1 n [ H ( S m 1 | W m 1 ) − H ( S m 1 | Y n 1 , W m 1 )] , (18) ≥ 1 b h H ( S 1 | W 1 ) − δ ( P ( m,n ) e ) i , (19) where ( 15 ) fol lows from the Markov relatio n S m 1 − X n 1 − Y n 1 giv en ( X n 2 , W m 1 ) ; ( 17 ) from the Markov relatio n X n 2 − W m 1 − S m 1 ; ( 18 ) from the fact that condit ioning reduces entropy; and ( 19 ) from t he memoryless source assumptio n and from (11 ) which uses Fano’ s inequalit y . DRAFT 16 On the other hand, we also ha ve I ( X n 1 ; Y n 1 | X n 2 , W m 1 ) = H ( Y n 1 | X n 2 , W m 1 ) − H ( Y n 1 | X n 1 , X n 2 , W m 1 ) , (20) = H ( Y n 1 | X n 2 , W m 1 ) − n X i =1 H ( Y 1 ,i | Y i − 1 1 , X n 1 , X n 2 , W m 1 ) , (21) = H ( Y n 1 | X n 2 , W m 1 ) − n X i =1 H ( Y 1 ,i | X 1 i , X 2 i , W m 1 ) , (22) ≤ n X i =1 H ( Y 1 ,i | X 2 i , W m 1 ) − n X i =1 H ( Y 1 ,i | X 1 i , X 2 i , W m 1 ) , (23) = n X i =1 I ( X 1 i ; Y 1 ,i | X 2 i , W m 1 ) , (24) where (21) follows from th e chain rule; (22) from the m emoryless channel assumpt ion; and (23) from t he chain rul e and t he fact that conditionin g reduces entropy . For t he joi nt mutual info rmation we can write the following set of inequalities: 1 n I ( X n 1 , X n 2 ; Y n 1 | W m 1 ) ≥ 1 n I ( S m 1 , S m 2 ; Y n 1 | W m 1 ) , (25) = 1 n [ H ( S m 1 , S m 2 | W m 1 ) − H ( S m 1 , S m 2 | Y n 1 , W m 1 )] , (26) = 1 n [ H ( S m 1 | W m 1 ) + H ( S m 2 | W m 1 ) − H ( S m 1 , S m 2 | Y n 1 , W m 1 )] , (27) ≥ 1 n [ H ( S m 1 | W m 1 ) + H ( S m 2 | W m 1 ) − H ( S m 1 , S m 2 | ˆ S m 1 , ˆ S m 2 ) , (28) ≥ 1 b " H ( S 1 | W 1 ) + H ( S 2 | W 1 ) − δ ( P ( m,n ) e ) # , (29) where ( 25 ) fol lows from the M arkov relation ( S m 1 , S m 2 ) − ( X n 1 , X n 2 ) − Y n 1 giv en W m 1 ; ( 27 ) from the Markov relation S m 2 − W m 1 − S m 1 ; ( 28 ) from the fact that ( S m 1 , S m 2 ) − ( Y n 1 , W m 1 ) − ( ˆ S m 1 , ˆ S m 2 ) form a Markov chain; and ( 29 ) from the memoryless sou rce assump tion and from (11) whi ch uses Fano’ s inequalit y . By following similar arguments as in (20)-(24) above, we can also show that I ( X n 1 , X n 2 ; Y n 1 | W m 1 ) ≤ n X i =1 I ( X 1 i , X 2 i ; Y 1 ,i | W m 1 ) . (30) Now , we introduce a ti me-sharing random variable ¯ Q in dependent of all other random vari- DRAFT 17 ables. W e ha ve ¯ Q = i with probability 1 /n , i ∈ { 1 , 2 , . . . , n } . Then we can write 1 n I ( X n 1 ; Y n 1 | X n 2 , W m 1 ) ≤ 1 n n X i =1 I ( X 1 i ; Y 1 ,i | X 2 i , W m 1 ) , (31) = 1 n n X i =1 I ( X 1 ¯ q ; Y ¯ q | X 2 ¯ q , W m 1 , ¯ Q = i ) , (32) = I ( X 1 ¯ Q ; Y ¯ Q | X 2 ¯ Q , W m 1 , ¯ Q ) , (33) = I ( X 1 ; Y | X 2 , Q ) , (34) where X 1 , X 1 ¯ Q , X 2 , X 2 ¯ Q , Y , Y ¯ Q , and Q , ( W m 1 , ¯ Q ) . Since S m 1 and S m 2 , and therefore X 1 i and X 2 i , are independent giv en W m 1 , for q = ( w m 1 , i ) we have P r { X 1 = x 1 , X 2 = x 2 | Q = q } = P r { X 1 i = x 1 , X 2 i = x 2 | W m 1 = w m 1 , ¯ Q = i } = P r { X 1 i = x 1 | W m 1 = w m 1 , ¯ Q = i } P r { X 2 i = x 2 | W m 1 = w m 1 , ¯ Q = i } = P r { X 1 | Q = q } · P r { X 2 | Q = q } . Hence, the probability di stribution is of the form giv en in Theorem 5.2. On combini ng the inequ alities above we can ob tain H ( S 1 | W 1 ) − δ ( P ( m,n ) e ) ≤ bI ( X 1 ; Y | X 2 , Q ) , (35) H ( S 2 | W 1 ) − δ ( P ( m,n ) e ) ≤ bI ( X 2 ; Y | X 1 , Q ) , (36) and H ( S 1 | W 1 ) + H ( S 2 | W 1 ) − δ ( P ( m,n ) e ) ≤ bI ( X 1 , X 2 ; Y | Q ) . (37) Finally , taking t he limit as m, n → ∞ and l etting P ( m,n ) e → 0 leads t o the condit ions of t he theorem. T o th e best of our knowledge, t his result constitutes the ﬁrst example in which the un derlying source structure leads t o the opti mality of (informational) source-channel separation i ndependent of the channel. W e can also interpret this result as follows: The side information provided to the recei ver satisﬁes a s pecial Markov chain cond ition, which enabl es the opt imality of in formational source-channel separatio n. W e can also observe from Theorem 5.2 t hat the optimal source- channel rate in this setup is determined by identifyi ng the smallest scaling fac tor b of the MA C capacity region such that the point ( H ( S 1 | W 1 ) , H ( S 2 , W 1 )) falls int o th e scaled region. This answers quest ion (3) afﬁrmati vely in this setup. DRAFT 18 A natural question to ask at this point is whether providing some side information to the recei ver can break th e opti mality of source-channel separation in t he case of independent mes- sages. In the next t heorem, we show that this is not the case, and t he op timality of informational separation continu es to hold . Theor em 5.3: Consider ind ependent sources S 1 and S 2 to be transmitted ov er the DM MA C with correlated receive r side information W 1 . If th e joint distribution satisﬁes p ( s 1 , s 2 , w 1 ) = p ( s 1 ) p ( s 2 ) p ( w 1 | s 1 , s 2 ) , then the source-channel rate b is achiev able if H ( S 1 | S 2 , W 1 ) < b · I ( X 1 ; Y 1 | X 2 , Q ) , (38) H ( S 2 | S 1 , W 1 ) < b · I ( X 2 ; Y 1 | X 1 , Q ) , (39) and H ( S 1 , S 2 | W 1 ) < b · I ( X 1 , X 2 ; Y 1 | Q ) , (40) for som e input d istribution p ( q , x 1 , x 2 , y 1 ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) p ( y 1 | x 1 , x 2 ) , (41) with |Q| ≤ 4 . Con versely , if the s ource-channel rate b is achiev able, then (38 )-(40) hold with < replaced by ≤ for some joint dist ribution of the form given in (41). Informational separation is optimal for this setu p. Pr oof: The proo f is giv en in Appendix I. Next, we illust rate the results of this section with som e examples. Consider b inary so urces and si de information, i.e., S 1 = S 2 = W 1 = { 1 , 2 } , with the following joi nt distribution: P S 1 S 2 W 1 { S 1 = 0 , S 2 = 0 , W 1 = 0 } = P S 1 S 2 W 1 { S 1 = 1 , S 2 = 1 , W 1 = 1 } = 1 / 3 and P S 1 S 2 W 1 { S 1 = 0 , S 2 = 1 , W 1 = 0 } = P S 1 S 2 W 1 { S 1 = 0 , S 2 = 1 , W 1 = 1 } = 1 / 6 . As the underlying multipl e access channel, we consider a binary i nput adder channel, in which X 1 = X 2 = { 0 , 1 } , Y = { 0 , 1 , 2 } and Y = X 1 + X 2 . DRAFT 19 P S f r a g r e p l a c e m e n t s H ( S 1 | S 2 ) H ( S 2 | S 1 ) H ( S 1 | W 1 ) H ( S 2 | W 1 ) (0 . 46 , 0 . 46) 0 . 5 0 . 5 1 1 . 5 1 . 58 Fig. 2. Capacity region of the binary adder MAC and the source coding rate regions i n the example. Note that, when the side inform ation W 1 is not a vailable at the recei ver , this model is the same as the example considered in [4], which was used to show the subopti mality of separate source and channel codes over the M A C. When the recei ver does no t ha ve access to side information W 1 , we can identify t he s eparate source and channel coding rate regions using the conditi onal entropies. These regions are shown in Fig. 2. The mini mum source-channel rate is found as b = 1 . 58 / 1 . 5 = 1 . 05 cupss in the case of separate s ource and channel codes. On t he ot her hand, it is easy t o see that uncoded transmissio n is opti mal i n this setup which requires a source-channel rate of b = 1 cupss. Now , if we cons ider the av ail ability of the side information W 1 at the receiv er , we ha ve H ( S 1 | W 1 ) = H ( S 2 | W 1 ) = 0 . 46 . In this case, usin g Theorem 5.2 , the minimum required source-channel rate is found to be b = 0 . 92 cups s, which is lower than the one achieved b y uncoded transmission. Theorem 5.3 states that, if the two sources are i ndependent, informational source-channel separation i s opti mal even if th e receiver has s ide inform ation g iv en which ind ependence of the sources no longer holds. Consider , for example, the same binary adder channel i n our example. W e now consider two independent binary sources with uniform distribution, i .e., P ( S 1 = 0) = P ( S 2 = 0) = 1 / 2 . Assume t hat the side information at the receive r is n ow given by W 1 = X 1 ⊕ X 2 , where ⊕ denotes the binary xor operation. For these sources and t he channel, the DRAFT 20 minimum s ource-channel rate wit hout the side information at the receiv er is found as b = 1 . 3 3 cupss. When W 1 is a va ilable at th e receiver , the m inimum required so urce-channel rate reduces to b = 0 . 67 cupss, wh ich can s till be achieved by separate source and channel coding. Next, we consi der the case when the receiv er s ide information i s also provided to the trans- mitters. From the s ource cod ing perspective, i .e., when the underly ing M A C is com posed o f orthogonal ﬁnite capacity li nks, it is known th at having the si de information at the transmitters would no t help . Howe ver , it is not clear in general, from the source-channel rate perspective, whether p roviding t he receiver si de information to t he transm itters would improve the p erfor - mance. If S 1 − W 1 − S 2 form a Markov chain , i t is easy to see that t he resul ts in Theorem 5.2 continu e to hold ev en when W 1 is provided to t he transm itters. Let ˜ S i = ( S i , W 1 ) b e the ne w so urces for which ˜ S 1 − W 1 − ˜ S 2 holds. Then, we have the same necessary and sufﬁcient conditi ons as before, hence providing t he receiv er side information to the transmitters would not help in this setup. Now , let S 1 and S 2 be two independent bin ary random variables, and W 1 = S 1 ⊕ S 2 . In thi s setup, providing the receiver side information W 1 to the transmitters means that the transmi tters can learn each other’ s source, and hence can ful ly cooperate to transmit both s ources. In this case, source-channel rate b is achiev able if H ( S 1 , S 2 | W 1 ) < bI ( X 1 , X 2 ; Y 1 ) (42 ) for som e input d istribution p ( x 1 , x 2 ) , and if sou rce-channel rate b is achie v able then (42) holds with ≤ for som e p ( x 1 , x 2 ) . On the o ther hand, if W 1 is not a va ilable at the transmi tters, we can ﬁnd from Theorem 5.3 t hat the input di stribution in (42) can only be p ( x 1 ) p ( x 2 ) . Thus, in this setup, providing receiver side informatio n to the transmi tters po tentially leads t o a smaller source-channel rate as thi s additi onal inform ation may enable cooperation ove r the MA C, whi ch is not poss ible wit hout the side informat ion. In our example of independent binary sou rces, t he total transmission rate that can be achieved b y tot al cooperation of the transm itters is 1 . 58 bits per channel use. Hence, the m inimum source-channel rate that can be achiev ed when th e sid e information W 1 is a vailable at both the transm itters and the receiv er is found to be 0 . 63 cupss. This is lower than 0 . 67 cupps that can be achieved when the side information is only av ailable at th e recei ver . DRAFT 21 W e conclude th at, as opposed to t he pure lossless sou rce coding scenario, having side i nforma- tion at the transm itters migh t improve the achie va ble source-channel rate in m ultiuser systems . V I . C O M P O U N D M AC W I T H C O R R E L A T E D S O U R C E S Next, we cons ider a com pound mu ltiple access channel, in which two transmit ters wish to transmit their correlated s ources reliably t o t wo receivers simultaneou sly [29]. The error probability of this sy stem is giv en as follows: P ( m,n ) e , P r    [ k =1 , 2 ( S m 1 , S m 2 ) 6 = ( ˆ S m k , 1 , ˆ S m k , 2 )    = X ( s m 1 ,s m 2 ) ∈S m 1 ×S m 2 p ( s m 1 , s m 2 ) P    [ k =1 , 2 ( ˆ s m k , 1 , ˆ s m k , 2 ) 6 = ( s m 1 , s m 2 )    ( S m 1 , S m 2 ) = ( s m 1 , s m 2 )    . The capacity region of the compoun d MA C i s shown t o be the intersectio n of the t wo MA C capacity regions in [27] in the case of independent sources and no receive r side inform ation. Howe ver , necessary and sufﬁcient conditio ns for lossless transmission in t he case of correlated sources are not known in g eneral. Note that, when there is si de inform ation at the recei vers, ﬁnding the achiev able source-channel rate for the compou nd M A C is not a simpl e extension of the capacity region in the case o f ind ependent sources. Due to different side information at the receivers, each transm itter should send a different part of i ts source to different recei vers. Hence, in this case we can consider the com pound MA C b oth as a combination of two MAC s, and as a com bination of two broadcast channels. W e remark here that ev en in the case o f single source broadcastin g with receiv er side i nformation, information al separation is not optimal, but the op timal source-channel rate can be achiev ed by operational separation as is s hown in [6]. W e ﬁrst state an achiev ability result for rate b = 1 , which extends the achieva bility scheme proposed in [4] to the com pound MA C with correlated side information. The extension to other rates is po ssible by consid ering blocks of sources and chann els as superlett ers similar to Theorem 4 in [4]. Theor em 6.1: Consider lossless transmissi on of arbitrarily correlated sources ( S 1 , S 2 ) over a DM compound MA C with side information ( W 1 , W 2 ) at the recei vers a s in Fig. 1. Source -channel DRAFT 22 rate 1 is achiev able if, for k = 1 , 2 , H ( S 1 | S 2 , W k ) < I ( X 1 ; Y k | X 2 , S 2 , W k , Q ) , H ( S 2 | S 1 , W k ) < I ( X 2 ; Y k | X 1 , S 1 , W k , Q ) , H ( S 1 , S 2 | U, W k ) < I ( X 1 , X 2 ; Y k | U, W k , Q ) , and H ( S 1 , S 2 | W k ) < I ( X 1 , X 2 ; Y k | W k ) , for som e joint d istribution of the form p ( q , s 1 , s 2 , w 1 , w 2 , x 1 , x 2 , y 1 , y 2 ) = p ( q ) p ( s 1 , s 2 , w 1 , w 2 ) p ( x 1 | q , s 1 ) p ( x 2 | q , s 2 ) p ( y 1 , y 2 | x 1 , x 2 ) and U = f ( S 1 ) = g ( S 2 ) is the common part of S 1 and S 2 in t he sense of G ` acs and K ¨ orner . Pr oof: The proof follows by using the correlation preserving mapp ing scheme of [4], and is thus omitted for the sake o f brevity . In the next theorem, we provide sufﬁcient cond itions for the achiev abi lity of a source-channel rate b . The achiev abil ity scheme is based on operational s eparation where the source and th e channel cod ebooks are generated independentl y of each other . In parti cular , the typical source outputs are m atched to the channel inpu ts without any explicit binning at the encoders. At the recei ver , a join t so urce-channel decoder i s used, which can be cons idered as a concatenati on of a list decoder as the channel decoder , and a source decoder t hat searches among the li st for the s ource codew o rd that is also joint ly typical with the side informatio n. Howe ver , there are no explicit source and channel codes that can be in dependently used either for compressi ng the sources or for in dependent data transm ission ov er the underlying compo und MA C. An alt ernativ e coding scheme composed of explicit sou rce and channel coders that interact with each other is proposed in [18 ]. Howe ver , the channel code in th is latter scheme is not the channel code for the un derlying multius er channel either . Theor em 6.2: Consider los sless transmissio n of arbit rarily correlated s ources S 1 and S 2 over a DM compound MA C with side information W 1 and W 2 at th e receiv ers. Source-channel rate DRAFT 23 b is achiev able i f, for k = 1 , 2 , H ( S 1 | S 2 , W k ) < bI ( X 1 ; Y k | X 2 , Q ) , (43) H ( S 2 | S 1 , W k ) < bI ( X 2 ; Y k | X 1 , Q ) , (44) and H ( S 1 , S 2 | W k ) < bI ( X 1 , X 2 ; Y k | Q ) , (45) for som e |Q| ≤ 4 and input d istribution of the form p ( q , x 1 , x 2 ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) . Remark 6.1: The achiev abi lity part of Theorem 6.2 can be o btained from the achiev ability of Theorem 6.1. Here, we constrain the channel input dis tributions to be independent of the source dist ributions as opposed to th e conditi onal distribution used in Theorem 6.1. W e provide the proof o f the achiev abilit y of Theorem 6.2 b elow to ill ustrate t he nature of the o perational separation scheme that is used. Pr oof: Fix δ k > 0 and γ k > 0 for k = 1 , 2 , and P X 1 and P X 2 . For b = n/m and k = 1 , 2 , at transmitter k , we generate M k = 2 m [ H ( S k )+ ǫ/ 2] i.i.d. length- m source code words and i.i.d. length- n channel code words usin g probability di stributions P S k and P X k , respecti vely . These code words are indexed and re vealed to the receivers as well, and are denoted b y s m k ( i ) and x n k ( i ) for 1 ≤ i ≤ M k . Encoder: Each source outcome is directly mapp ed to a channel codew ord as follows: Giv en a source out come S m k at transm itter m , we ﬁnd the sm allest i k such that S m k = s m k ( i k ) , and transmit the code word x n k ( i k ) . An error occurs i f no such i k is found at eit her of the t ransmitters k = 1 , 2 . Decoder: At recei ver k , we ﬁnd the unique pai r ( i ∗ 1 , i ∗ 2 ) th at simult aneously s atisﬁes ( x n 1 ( i ∗ 1 ) , x n 2 ( i ∗ 2 ) , Y n k ) ∈ T ( n ) [ X 1 X 2 Y ] δ k , and ( s m 1 ( i ∗ 1 ) , s m 2 ( i ∗ 2 ) , W m k ) ∈ T ( m ) [ S 1 S 2 W k ] γ k , where T ( n ) [ X ] δ is the set of weakly δ -typical sequences. An error i s declared if the ( i ∗ 1 , i ∗ 2 ) pair is not uniq uely determined. DRAFT 24 Pr obabi lity o f err or : W e deﬁne the following e vents: E k 1 = { S m k 6 = s m k ( i ) , ∀ i } E k 2 = { ( s m 1 ( i 1 ) , s m 2 ( i 2 ) , W m k ) / ∈ T ( m ) [ S 1 S 2 W k ] γ k } E k 3 = { ( x n 1 ( i 1 ) , x n 2 ( i 2 ) , Y n k ) / ∈ T ( n ) [ X 1 X 2 Y ] δ k } and E k 4 ( j 1 , j 2 ) = { ( s m 1 ( j 1 ) , s m 2 ( j 2 ) , W m k ) ∈ T ( m ) [ S 1 S 2 W k ] γ k and ( x n 1 ( j 1 ) , x n 2 ( j 2 ) , Y n k ) ∈ T ( n ) [ X 1 X 2 Y ] δ k } Here, E 1 denotes the error eve nt in which either of the encoders fails t o ﬁnd a un ique source codew ord in i ts cod ebook t hat corresponds to its current so urce ou tcome. When such a codew ord can be found, E k 2 denotes the error eve nt in which the sources S m 1 and S m 2 and the side i nformation W k at receiver k are not jo intly typical. On th e other h and, E k 3 denotes the error event i n which channel codewords th at match the current s ource realization s are not jointly typical with the channel output at recei ver k . Finally E k 4 ( j 1 , j 2 ) is the event that the source code words corresponding to the indices j 1 and j 2 are jointly t ypical with th e sid e inform ation W k and s imultaneousl y that th e channel codew ords corresponding to the indices j 1 and j 2 are jointly typi cal with th e channel ou tput Y k . Deﬁne P ( m,n ) k , Pr { ( S m 1 , S m 2 ) 6 = ( ˆ S m k , 1 , ˆ S m k , 2 ) } . Then P ( m,n ) e ≤ P k =1 , 2 P ( m,n ) k . Again, from the union bou nd, we ha ve P ( m,n ) k ≤ Pr { E k 1 } + Pr { E k 2 } + Pr { E k 3 } + X j 1 6 = i 1 , j 2 = i 2 E k 4 ( j 1 , j 2 ) + X j 1 = i 1 , j 2 6 = i 2 E k 4 ( j 1 , j 2 ) + X j 1 6 = i 1 , j 2 6 = i 2 E k 4 ( j 1 , j 2 ) , (46) where i 1 and i 2 are t he correct indices. W e have E k 4 ( j 1 , j 2 ) = Pr n ( s m 1 ( j 1 ) , s m 2 ( j 2 ) , W m k ) ∈ T ( m ) [ S 1 ,S 2 ,W k ] γ k o Pr n ( x n 1 ( j 1 ) , x n 2 ( j 2 ) , Y n k ) ∈ T ( n ) [ X 1 ,X 2 ,Y k ] δ k o . (47) In [6] it is shown that, for any λ > 0 and s uf ﬁciently large m , Pr { E k 1 } = (1 − Pr { S m k = s m k (1) } ) M k ≤ exp − 2 − n [ H ( S k )+6 λ ] M k = exp − 2 n [ ǫ 2 − 6 λ ] . (48) DRAFT 25 W e choose λ < ǫ 12 , and obtain Pr { E 1 } → 0 as m → ∞ . Similarly , we can also prove that Pr( E i ( k )) → 0 for i = 2 , 3 and k = 1 , 2 as m, n → ∞ using st andard techniques. W e can also obtain X j 1 6 = i 1 , j 2 = i 2 Pr n ( s m 1 ( j 1 ) , s m 2 ( j 2 ) , W m k ) ∈ T ( m ) [ S 1 ,S 2 ,W k ] γ k o Pr n ( x n 1 ( j 1 ) , x n 2 ( j 2 ) , Y n k ) ∈ T ( n ) [ X 1 ,X 2 ,Y k ] δ k o ≤ 2 m [ H ( S 1 )+ ǫ 2 ] − m [ I ( S 1 ; S 2 ,W k ) − λ ] − n [ I ( X 1 ; Y k | X 2 ) − λ ] (49) = 2 − m [ H ( S 1 | S 2 ,W k ) − bI ( X 1 ; Y k | X 2 ) − ( b +1) λ − ǫ 2 ] = 2 − m [ ǫ 2 − ( b +1) λ ] (50) where in (49) we used (1) and (2); and (50 ) holds if th e conditions in the th eorem hold . A si milar bound can be found for the second summ ation in (46). For the third one, we have the fol lowing bound. X j 1 6 = i 1 , j 2 6 = i 2 Pr n ( s m 1 ( j 1 ) , s m 2 ( j 2 ) , W m k ) ∈ T ( m ) [ S 1 ,S 2 ,W k ] γ k o Pr n ( x n 1 ( j 1 ) , x n 2 ( j 2 ) , Y n k ) ∈ T ( n ) [ X 1 ,X 2 ,Y ] δ k o ≤ 2 m [ H ( S 1 )+ ǫ/ 2]+ m [ H ( S 2 )+ ǫ/ 2] 2 − m [ I ( S 1 ; S 2 ,W k )+ I ( S 2 ; S 1 ,W k ) − I ( S 1 ; S 2 | W k )] − λ ] 2 − n [ I ( X 1 ,X 2 ; Y k ) − λ ] (51) ≤ 2 − m [ H ( S 1 | S 2 ,W k )+ H ( S 2 | S 1 ,W k ) − bI ( X 1 ,X 2 ; Y k ) − ( b +1) λ − ǫ ] = 2 − m [ ǫ − ( b +1) λ ] , (52) where (51) follows from (1) and (3); and (52) holds if the condit ions in the theorem hol d. Choosing λ < min n ǫ 12 , ǫ 2( b +1) o , we can make sure that all terms of the sum mation in (46) also vanish as m, n → ∞ . Any rate pair in th e con vex hul l can be achiev ed by time sharing, hence the time-sharing random v ariable Q . The cardinality bound on Q follows from the classical ar guments. W e next prove that the conditio ns in Theorem 6.2 are als o necessary to achieve a source- channel rate of b for s ome special settings, hence, answering questi on (2) af ﬁrmati vely for these cases. W e ﬁrst consi der t he case in which S 1 is independent of ( S 2 , W 1 ) and S 2 is independent of ( S 1 , W 2 ) . This mi ght model a scenario in which Rx 1 ( Rx 2 ) and Tx 2 ( Tx 1 ) are located clos e to each ot her , t hus h a ving correlated observations, while the two transmit ters are far away from each other (see Fig. 3). Theor em 6.3: Consider los sless transmissio n of arbit rarily correlated s ources S 1 and S 2 over a DM compound M A C wi th side information W 1 and W 2 , where S 1 is independent of ( S 2 , W 1 ) DRAFT 26 P S f r a g r e p l a c e m e n t s H ( S 1 | S 2 ) H ( S 2 | S 1 ) H ( S 1 | W 1 ) H ( S 2 | W 1 ) ( 0 . 4 6 , 0 . 4 6 ) 0 . 5 1 1 . 5 1 . 5 8 S m 1 S m 2 X n 1 X n 2 Y n 1 Y n 2 W m 1 W m 2 ( ˆ S m 1 , 1 , ˆ S m 1 , 2 ) ( ˆ S m 2 , 1 , ˆ S m 2 , 2 ) Tx 1 Tx 1 T x 2 Rx 1 Rx 2 p ( s 1 , w 2 ) p ( s 2 , w 1 ) p ( y 1 , y 2 | x 1 , x 2 ) Fig. 3. Compo und multiple access channel in which the transmitter 1 (2) and receiv er 2 (1) are located close to each other, and hence have correlated observations, independent of the other pair , i. e., S 1 is independent of ( S 2 , W 1 ) and S 2 is independent of ( S 1 , W 2 ) . and S 2 is independent of ( S 1 , W 2 ) . Separation (in the o perational sens e) i s opt imal for this setup, and the source-channel rate b is achiev able if, for ( k , m ) ∈ { (1 , 2) , ( 2 , 1) } , H ( S k ) < bI ( X k ; Y k | X m , Q ) , (53) H ( S m | W k ) < bI ( X m ; Y k | X k , Q ) , (54) and H ( S k ) + H ( S m | W k ) < bI ( X k , X m ; Y k | Q ) , (55) for som e |Q| ≤ 4 and input d istribution of the form p ( q , x 1 , x 2 ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) . (56) Con versely , if source-channel rate b is achiev able, then (53)-(55) hold with < replaced by ≤ for an input p robability distribution of the form given in (56). Pr oof: Achiev abil ity follows from Theorem 6.2, and the con verse proo f is giv en in Appendix II. Next, we consider the case in which t here i s no m ultiple access int erference at th e receiv ers (see Fig. 4). W e let Y k = ( Y 1 ,k , Y 2 ,k ) k = 1 , 2 , where the memoryless channel is characterized by p ( y 1 , 1 , y 2 , 1 , y 1 , 2 , y 2 , 2 | x 1 , x 2 ) = p ( y 1 , 1 , y 1 , 2 | x 1 ) p ( y 2 , 1 , y 2 , 2 | x 2 ) . (57) On the other hand, we allow arbitrary correlation among the sources and the side information . Howe ver , since there i s n o multi ple access interference, using the so urce correlation to create DRAFT 27 P S f r a g r e p l a c e m e n t s H ( S 1 | S 2 ) H ( S 2 | S 1 ) H ( S 1 | W 1 ) H ( S 2 | W 1 ) ( 0 . 4 6 , 0 . 4 6 ) 0 . 5 1 1 . 5 1 . 5 8 S m 1 S m 2 X n 1 X n 2 W m 1 W m 2 ( ˆ S m 1 , 1 , ˆ S m 1 , 2 ) ( ˆ S m 2 , 1 , ˆ S m 2 , 2 ) Y m 1 , 1 Y m 1 , 2 Y m 2 , 1 Y m 2 , 2 Tx 1 Tx 1 T x 2 Rx 1 Rx 2 p ( y 1 , 1 , y 1 , 2 | x 1 ) p ( y 2 , 1 , y 2 , 2 | x 2 ) Fig. 4. Compo und multiple access channel with correlated sources and correlated side information with no multi ple access interference. correlated channel codew ords does not enlarge the rate region of the channel. W e also remark that this mod el is not equiv alent to two independent broadcast channels wit h sid e information. The two encoders interact with each other throug h the correlation among t heir sources. Theor em 6.4: Consider los sless transmissio n of arbit rarily correlated s ources S 1 and S 2 over a DM compound MAC w ith n o multi ple access in terference characterized by (57) and receive r side inform ation W 1 and W 2 (see Fig. 4). Separation (in th e operational sens e) is optimal for this setu p, and the source-channel rate b is achiev able if, for ( k , m ) = { (1 , 2) , (2 , 1) } H ( S k | S m , W k ) < bI ( X k ; Y k ,k ) , (58) H ( S m | S k , W k ) < bI ( X m ; Y m,k ) , (59) and H ( S k , S m | W k ) < b [ I ( X k ; Y k ,k ) + I ( X m ; Y m,k )] , (60) for an input d istribution of the form p ( q , x 1 , x 2 ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) . (61) Con versely , if the s ource-channel rate b is achiev able, then (53 )-(55) hold with < replaced by ≤ for an inpu t probabilit y distribution of the form give n in (56). Pr oof: The achiev abili ty follows from Theorem 6.2 by letting Q be constant and taking into consideration the characteristics of th e channel, where ( X 1 , Y 1 , 1 , Y 1 , 2 ) is independent of DRAFT 28 ( X 2 , Y 2 , 1 , Y 2 , 2 ) . The con verse can be p rove n similarly to Theorem 6 .3, and will be om itted for the sake of brevity . Note that the mo del consid ered in Theorem 6.4 is a generalization of the model in [30] (which is a special case o f the more general network stu died in [7]) t o more than one receiv er . Theorem 6.4 considers correlated receiver sid e i nformation which can be incorporated into the model of [30] by considering an additional transmitter sending this side inform ation over an inﬁnite capacity link. In thi s case, using [30], we observe that informati onal source-channel separation is optimal. Howe ver , Theorem 6.4 ar gues that thi s i s no longer true wh en the num ber of sink nodes is greater than one even when there is no recei ver si de inform ation. The mod el i n Theorem 6.4 is also consi dered in [31] in the special case of no si de information at the recei vers. In the achie v ability scheme of [31], transmitters ﬁ rst ra ndomly bin their correlated sources, and t hen m atch the bi ns to channel codew ords. Theorem 6.4 shows that we can achiev e the same opti mal performance without explicit binning eve n i n the case of correlated receiv er side informati on. In bo th T heorem 6. 3 and Theorem 6.4, we provide the optimal source-channel matching conditions for lossless transmi ssion. Wh ile general m atching conditions are not known for compound MA Cs, the reason we are able t o resolve the pro blem in these two cases is the lack of multi ple access interference from users with correlated sources. In the ﬁrst setup the two sources are independent, hence it is not p ossible to generate correlated chann el inp uts, while in the second setup, there is no multi ple access interference, and thu s there is no need to generate correlated channel inputs. W e note here th at th e opti mal source-channel rate in both cases is achiev ed by operatio nal s eparation ans wering b oth questi on (2) and question (4) af ﬁrmativ el y . The s upoptimalit y of i nformational s eparation in these m odels follows from [6], since the broadcast channel mod el s tudied in [6] is a special case of the com pound MA C mod el we consider . W e refer to the example p rovided in [31] for the suboptim ality of i nformational separation for the setu p of T heorem 6.4 e ven without side information at the receiv es. Finally , we consider t he special case in which t he two receiver s share com mon sid e infor- mation, i.e., W 1 = W 2 = W , in which case S 1 − W − S 2 form a M arko v chain. For example this mod els the scenario in which th e two receiv ers are close to each other , hence t hey h a ve the same side inform ation. Th e following theorem prove s the optimalit y of inform ational separation under th ese conditions. DRAFT 29 Theor em 6.5: Consider lossless transmission o f correlated s ources S 1 and S 2 over a DM compound MAC with com mon receiver side i nformation W 1 = W 2 = W satisfyin g S 1 − W − S 2 . Separation (in the i nformational sense) is opti mal in this setup, and the s ource-channel rate b is achie va ble if, for k = 1 and 2 , H ( S 1 | W ) < b · I ( X 1 ; Y k | X 2 , Q ) , (62) H ( S 2 | W ) < b · I ( X 2 ; Y k | X 1 , Q ) , and H ( S 1 | W ) + H ( S 2 | W ) < b · I ( X 1 , X 2 ; Y k | Q ) , for som e joint d istribution p ( q , x 1 , x 2 , y ) = p ( q ) p ( x 1 | q ) p ( x 2 | q ) p ( y | x 1 , x 2 ) , wi th |Q| ≤ 4 . Con versely , if the s ource-channel rate b is achiev able, then (62 )-(63) hold with < replaced by ≤ for an inpu t probabilit y distribution of the form give n above. Pr oof: The achiev ability follo ws from information al source-channel separation, i.e, Slepian- W olf compression conditioned on the recei ver side information foll o wed by an optimal compound MA C codi ng. The proof of the con verse follows si milarly to the proof of Theorem 5.2, and is omitted for bre vity . V I I . I N T E R F E R E N C E C H A N N E L W I T H C O R R E L A T E D S O U R C E S In this section, we consid er t he interference channel (IC) wit h correlated sources and side information. In the IC each transmitter wi shes t o com municate onl y wit h its corresponding recei ver , while the two simultaneous transm issions i nterfere wit h each other . Eve n when the sources and the side information are all independent, the capacity region of the IC is i n general not known. The best achiev able scheme is given in [32]. The capacity region can b e characterized in the strong interference case [36], [10], where i t coincides with the capacity re gion of t he compound m ultiple access channel, i.e., it is opti mal for the receiver s to decode bo th m essages. The interference channel has gained recent interest due to its practical v alue in cellular and cognitive radio systems. See [33] - [35] and references therein for recent results relating to the capacity region of various int erference channel scenarios. DRAFT 30 For encoders f ( m,n ) i and decoders g ( m,n ) i , the probability of error for the interference channel is given as P ( m,n ) e , P r    [ k =1 , 2 S m k 6 = ˆ S m k ,k    = X ( s m 1 ,s m 2 ) ∈S m 1 ×S m 2 p ( s m 1 , s m 2 ) P    [ k =1 , 2 ˆ s m k ,k 6 = s m k    ( S m 1 , S m 2 ) = ( s m 1 , s m 2 )    . In the case of correlated sources and recei ver sid e information, sufﬁ cient conditions for the compound MAC m odel giv en in Theorem 6 .1 and Theorem 6.2 serve as sufﬁc ient condition s for the IC as w ell, s ince we can constrain both receivers to obtain loss less reconstructio n of both sources. Our goal here is to characterize the conditions u nder wh ich we can provide a con verse and achie ve either informati onal or operational separation similar to the resul ts of Section VI. In order t o extend the necessary con ditions of Theorem 6.3 and Theorem 6.5 to ICs, we will deﬁne the ‘s trong source-channel in terference’ condit ions. Note that t he int erference channel version of Theorem 6.4 i s trivial s ince the two transmiss ions d o not interfere with each o ther . The regular strong interference condi tions giv en in [36] correspond to t he case in which, for al l input distributions at transmit ter Tx 1 , the rate of information ﬂow to recei ver Rx 2 is higher than the information ﬂow to the intended recei ver Rx 1 . A simi lar cond ition h olds for transm itter Tx 2 as well. Hence there is no rate loss if bot h receiver s decode the messages of bo th transm itters. Consequently , under strong interference conditions, the capacity region of the IC i s equiv alent to the capacity region of the compound MA C. Howe ver , in the j oint source-channel coding scenario, the receivers have access to correlated si de inform ation. Thus whi le calculating the total rate of informatio n ﬂow to a particul ar receiv er , we should not only con sider t he in formation ﬂow through the channel, but als o the mutual in formation that already exists bet ween t he source and the receiver sid e informati on. W e ﬁrst focus on the scenario of Theorem 6.3 in which the source S 1 is independent of ( S 2 , W 1 ) and S 2 is independent of ( S 2 , W 1 ) . Deﬁnition 7.1: For the int erference channel in which S 1 is independent of ( S 2 , W 1 ) and S 2 is independent of ( S 2 , W 1 ) , we say t hat the strong so ur ce-channel in terfer ence condit ions are satisﬁed for a s ource-channel rate b if, b · I ( X 1 ; Y 1 | X 2 ) ≤ b · I ( X 1 ; Y 2 | X 2 ) + I ( S 1 ; W 2 ) , (63) DRAFT 31 and b · I ( X 2 ; Y 2 | X 1 ) ≤ b · I ( X 2 ; Y 1 | X 1 ) + I ( S 2 ; W 1 ) , (64) for all distributions of the form p ( w 1 , w 2 , s 1 , s 2 , x 1 , x 2 ) = p ( w 1 , w 2 , s 1 , s 2 ) p ( x 1 | s 1 ) p ( x 2 | s 2 ) . For an IC satisfying these conditions , we next prove the following theorem. Theor em 7.1: Consider lossless transmiss ion of S 1 and S 2 over a DM IC with side information W 1 and W 2 , where S 1 is i ndependent of ( S 2 , W 1 ) and S 2 is i ndependent of ( S 2 , W 1 ) . Ass uming that t he stron g source-channel int erference conditio ns of Deﬁnition 7 .1 are s atisﬁed for b , separation (in the i nformational sense) is optimal. The s ource-channel rate b is achiev able if, the conditions (43 )-(45) in Theorem 6.2 hol d. Con versely , if rate b is achie vable, then the conditions in Theorem 6.2 hold with < replaced by ≤ . Before we proceed wit h the proof of t he theorem, we ﬁrst prove the following lem ma. Lemma 7.2: If ( S 1 , W 2 ) is independent of ( S 2 , W 1 ) and th e st rong source-channel int erference conditions (63)-(64) hold, th en we have I ( X n 2 ; Y n 2 | X n 1 ) ≤ I ( X n 2 ; Y n 1 | X n 1 ) + I ( S m 2 ; W m 1 ) , (65 ) and I ( X n 1 ; Y n 1 | X n 2 ) ≤ I ( X n 1 ; Y n 2 | X n 2 ) + I ( S m 1 ; W m 2 ) , (66 ) for all m and n satisfying n/m = b . Pr oof: T o prove the l emma, we follow the techniques in [10]. Condit ion (64) im plies I ( X 2 ; Y 2 | X 1 , U ) − I ( X 2 ; Y 1 | X 1 , U ) ≤ 1 b I ( S 2 ; W 1 ) (67) for all U satisfying U − ( X 1 , X 2 ) − ( Y 1 , Y 2 ) . Then as in [10], we can obtain I ( X n 2 ; Y n 2 | X n 1 ) − I ( X n 2 ; Y n 1 | X n 1 ) = I ( X 2 n ; Y 2 n | X n 1 , Y n − 1 2 ) − I ( X 2 n ; Y 1 n | X n 1 , Y n − 1 2 ) + I ( X n − 1 2 ; Y n − 1 2 | X n 1 , Y 1 n ) − I ( X n − 1 2 ; Y n − 1 1 | X n 1 , Y 1 n ) = I ( X 2 n ; Y 2 n | X 1 n ) − I ( X 2 n ; Y 1 n | X 1 n ) + I ( X n − 1 2 ; Y n − 1 2 | X n − 1 1 ) − I ( X n − 1 2 ; Y n − 1 1 | X n − 1 1 ) = n X i =1 [ I ( X 2 i ; Y 2 i | X 1 i ) − I ( X 2 i ; Y 1 i | X 1 i )] . DRAFT 32 Using the hypothesis (64) of the theorem, we obtain I ( X n 2 ; Y n 2 | X n 1 ) − I ( X n 2 ; Y n 1 | X n 1 ) ≤ n b I ( S 2 ; W 1 ) = I ( S m 2 ; W m 1 ) . Eqn. (66) follows sim ilarly . Pr oof: (of Theor em 7.1) Achiev ability fol lows by having each recei ver decode bot h S 1 and S 2 , and then us ing Theorem 6.1. W e next prove the con verse. From (95)-(98), we have 1 n I ( X n 1 ; Y n 1 | X n 2 ) ≥ 1 b h H ( S 1 ) − δ ( P ( m,n ) e ) i . (68) W e can als o obtain 1 n I ( X n 1 ; Y n 2 | X n 2 ) ≥ 1 n [ I ( X n 1 ; Y n 1 | X n 2 ) − I ( S m 1 ; W m 2 )] , (69) = 1 b [ H ( S 1 ) − δ ( P ( m,n ) e )] − 1 n I ( S m 1 ; W m 2 ) , (70) = 1 b [ H ( S 1 | W 2 ) − δ ( P ( m,n ) e )] , (71) in which (69) fol lows from (66 ), and (70) from (68). Finally for the joint mutual in formation, we have 1 n I ( X n 1 , X n 2 ; Y n 1 ) = 1 n [ I ( X n 1 ; Y n 1 ) + I ( X n 2 ; Y n 1 | X n 1 )] , ≥ 1 n [ I ( S m 1 ; Y n 1 ) + I ( X n 2 ; Y n 2 | X n 1 ) − I ( S m 2 ; W m 1 )] , (72) ≥ 1 n [ I ( S m 1 ; Y n 1 ) + I ( S m 2 ; Y n 2 | X n 1 ) − I ( S m 2 ; W m 1 )] , (73) = 1 n [ H ( S m 1 ) − H ( S m 1 | Y n 1 ) + H ( S m 2 | X n 1 ) − H ( S m 2 | Y n 2 , X n 1 ) + H ( S m 2 | W m 1 ) − H ( S m 2 )] , ≥ 1 n [ H ( S m 1 ) − H ( S m 1 | Y n 1 ) − H ( S m 2 | Y n 2 ) + H ( S m 2 | W m 1 )] , (74) = 1 n [ H ( S m 1 ) − H ( S m 1 | Y n 1 , W m 1 ) − H ( S m 2 | Y n 2 , W m 2 ) + H ( S m 2 | W m 1 )] , (75) ≥ 1 b [ H ( S 1 ) + H ( S 2 | W 1 ) − 2 δ ( P ( m,n ) e )] , (76) for any ǫ > 0 and large enough m and n , where ( 72 ) follows from the data processing inequality and (65); ( 73 ) follows from the data processing inequality since S m 2 − X n 2 − Y n 2 form a Markov chain giv en X n 1 ; ( 74 ) follows from the independence of X n 1 and S m 2 and the fact DRAFT 33 P S f r a g r e p l a c e m e n t s H ( S 1 | S 2 ) H ( S 2 | S 1 ) H ( S 1 | W 1 ) H ( S 2 | W 1 ) ( 0 . 4 6 , 0 . 4 6 ) 0 . 5 1 1 . 5 1 . 5 8 S m 1 S m 2 X n 1 X n 2 Y n 1 Y n 2 ˆ S m 1 ˆ S m 2 ˆ S m 2 p ( y 1 , y 2 | x 1 , x 2 ) User 1 User 2 T wo-w ay Channel Fig. 5. The two-way channel model with correlated sources. that conditi oning reduces entropy; ( 75 ) follows from the fact that S 1 is independent of ( S 2 , W 1 ) and S 2 is independent of ( S 2 , W 1 ) ; and ( 76 ) follo ws from Fano’ s inequality . The rest of the proof clos ely resembles t hat of T heorem 6.3. Next, we consider the IC version of the case in Theorem 6.5, in which the two receiv ers hav e access to the s ame side inform ation W and with this side information the sources are independent. Whil e we still have correlation between the sources and th e common receive r side information, th e amo unt of mutual information arising from this correlation is equiv alent at both recei vers since W 1 = W 2 . This suggest s that t he usual strong interference channel conditions sufﬁ ce to o btain the con verse result. W e have the following theorem for t his case. Theor em 7.3: Consider lossless transmiss ion of correlated sources S 1 and S 2 over t he strong IC with common receiv er side inform ation W 1 = W 2 = W satisfying S 1 − W − S 2 . Separation (in the informational sense) is optimal in this setup, and the source-channel rate b is achie va ble if and only if the cond itions in T heorem 6.5 hold. Pr oof: The proof follows from arguments sim ilar to those in the proof of Theorem 6.5 and results in [28], where we incorporate the strong interference con ditions. V I I I . T WO - W A Y C H A N N E L W I T H C O R R E L A T E D S O U R C E S In t his section, w e consider the two-way channel s cenario with correlated source sequences (see Fig. 5). The two-way channel model was intro duced by Shannon [3] who g a ve inner and outer bounds on the capacity region. Shannon sh o wed that his inner bound is in deed th e capacity region of the “restricted” two-way channel, in which th e channel inputs of t he users depend only on the messages (not on the previous channel o utputs). Sev eral improved o uter bounds are given in [37]-[39] using the “dependence-balance bounds” proposed by Hekstra and W ill ems. DRAFT 34 In [3] Shanno n also considered the case of correlated sources, and showed by an example th at by exploiting the correlation structu re of the sources we m ight achiev e rate pairs give n by the outer bound. Here we cons ider arbitrarily correlated sources and provide an achiev abil ity resu lt using th e coding scheme for the compoun d MA C model i n Section VI. It is possible to extend the results to the scenario where each user also h as si de i nformation correlated with the so urces. In the general two-way channel m odel, t he encoders obs erve the past channel o utputs and hence they can use these observations for encodi ng future channel in put symbol s. The encoding function at user i at time instant j i s given by f i,j : S m i × Y j − 1 i → X i , (77) for i = 1 , 2 . The probabi lity of error for the two-way channel is given as P ( m,n ) e , P r    [ k =1 , 2 S m k 6 = ˆ S m k    = X ( s m 1 ,s m 2 ) ∈S m 1 ×S m 2 p ( s m 1 , s m 2 ) P    [ k =1 , 2 ˆ s m k 6 = s m k    ( S m 1 , S m 2 ) = ( s m 1 , s m 2 )    . Note that, if we only consider restricted encoders at the users, than the system model is e quiv alent to t he compoun d MA C model of Fig. 1 with W m 1 = S m 1 and W m 2 = S m 2 . From Theorem 6. 1 we obtain the following coroll ary . Cor ollary 8.1: In l ossless t ransmission of arbitrarily correlated sources ( S 1 , S 2 ) over a DM two-way channel, the source-channel rate b = 1 is achiev able if H ( S 1 | S 2 ) < I ( X 1 ; Y 2 | X 2 , S 2 , Q ) and H ( S 2 | S 1 ) < I ( X 2 ; Y 1 | X 1 , S 1 , Q ) , for som e joint d istribution of the form p ( q , s 1 , s 2 , x 1 , x 2 , y 1 , y 2 ) = p ( q ) p ( s 1 , s 2 ) p ( x 1 | q , s 1 ) p ( x 2 | q , s 2 ) p ( y 1 , y 2 | x 1 , x 2 ) . Note that here we u se the source correlation rather than the correlation that can b e created through the inh erent feedback av ai lable in th e t wo-wa y channel. Thi s correlation amon g the channel codewords pot entially helps us achiev e source-channel rates t hat cannot be achieve d by independent inputs. Shannon’ s outer bound can also be extended to the case of correlated sources to obtain a lower bo und on the achiev able source-channel rate as follows. DRAFT 35 Pr oposi tion 8.2: In l ossless transm ission of arbit rarily correlated sources ( S 1 , S 2 ) over a DM two-way channel, if the source-channel rate b is achiev able, then H ( S 1 | S 2 ) < bI ( X 1 ; Y 2 | X 2 ) and H ( S 2 | S 1 ) < bI ( X 2 ; Y 1 | X 1 ) , for som e joint d istribution of the form p ( s 1 , s 2 , x 1 , x 2 , y 1 , y 2 ) = p ( s 1 , s 2 ) p ( x 1 , x 2 ) p ( y 1 , y 2 | x 1 , x 2 ) . Pr oof: W e have H ( S m 1 | S m 2 ) = I ( S m 1 ; Y n 2 | S m 2 ) + H ( S m 1 | S m 2 , Y n 2 ) (78) ≤ I ( S m 1 ; Y n 2 | S m 2 ) + mδ ( P ( m,n ) e ) (79) = H ( Y n 2 | S m 2 ) − H ( Y n 2 | S m 1 , S m 2 ) + mδ ( P ( m,n ) e ) (80) = n X i =1 H ( Y 2 i | S m 2 , Y i − 1 2 ) − H ( Y 2 i | S m 1 , S m 2 , Y i − 1 2 ) + mδ ( P ( m,n ) e ) (81) ≤ n X i =1 H ( Y 2 i | S m 2 , Y i − 1 2 , X i 2 ) − H ( Y 2 i | S m 1 , S m 2 , Y i − 1 2 , Y i − 1 1 , X 2 i ) + mδ ( P ( m,n ) e ) (82) ≤ n X i =1 H ( Y 2 i | X 2 i ) − H ( Y 2 i | S m 1 , S m 2 , Y i − 1 2 , Y i − 1 1 , X 1 i , X 2 i ) + mδ ( P ( m,n ) e ) (83) ≤ n X i =1 H ( Y 2 i | X 2 i ) − H ( Y 2 i | X 1 i , X 2 i ) + mδ ( P ( m,n ) e ) (84) ≤ n X i =1 I ( X 1 i ; Y 2 i | X 2 i ) + mδ ( P ( m,n ) e ) (85) where (79) foll ows from Fano’ s inequality; (82) follows since X k 2 is a deterministic functio n of ( S m 2 , Y i − 1 2 ) and the fact that condition ing reduces entropy; (83) fol lows sim ilarly as X k 1 is a deterministic function of ( S m 1 , Y i − 1 1 ) and the fact that condition ing reduces entropy; and (84) follows sin ce Y 2 i − ( X 1 i , X 2 i ) − ( S m 1 , S m 2 , Y i − 1 2 , Y i − 1 1 ) form a Markov chain. Similarly , we can show th at H ( S m 2 | S m 1 ) ≤ n X i =1 I ( X 2 i ; Y 1 i | X 1 i ) + mδ ( P ( m,n ) e ) . (86) From con vexity ar guments and letting m, n → ∞ , we o btain H ( S 1 | S 2 ) ≤ bI ( X 1 ; Y 2 | X 2 ) , (87) H ( S 2 | S 1 ) ≤ bI ( X 2 ; Y 1 | X 1 ) , (88) DRAFT 36 for som e joint d istribution p ( x 1 , x 2 ) . Remark 8.1: Not e that the lo wer bound of Proposition 8.2 allows all possible joint distributions for the channel inp uts. Th is lets us express t he lower bound in a separable form , since the so urce correlation becomes useless t o i ntroduce any additional structure to the transm itted channel code words. In general, not all joint channel input distributions can be achie ved at the two us ers, and tighter bounds can be obtained by lim iting the set of pos sible j oint distributions as in [37]- [39]. Howe ver , if the existing so urce correlation allows the users to generate the optimal joint channel input dis tribution, then the achie v able region g iv en in Corollary 8.1 might meet th e upper bound without the need t o exploit the feedback to generate further correlation. This has been ill ustrated by an example in [3]. Shannon considered correlated bin ary sources S 1 and S 2 such that P S 1 S 2 ( S 1 = 0 , S 2 = 1) = P S 1 S 2 ( S 1 = 1 , S 2 = 0) = 0 . 275 and P S 1 S 2 ( S 1 = 1 , S 2 = 1) = 0 . 45 , and a binary mul tiplier two-way channel, in which X 1 = X 2 = Y 1 = Y 2 = { 0 , 1 } and Y 1 = Y 2 = X 1 · X 2 . Using Propos ition 8.2, we can set a lower b ound of b = 1 on the achiev able source-channel rate. On the other hand, t he source-channel rate of 1 can b e achieved simply by uncoded t ransmission. Hence, in this example, the correlated source structure enabl es the t ransmitter to achie ve the optimal joint d istribution for the channel inputs withou t exploiting the inherent feedback in the two-way channel. Note that the Shannon o uter bound i s not achieva ble in t he case of i ndependent sources in a binary m ultiplier two-way channel [37], and the achiev able rates can be improved by using channel in puts dependent on the pre vious channel o utputs. DRAFT 37 I X . C O N C L U S I O N S W e have considered source and c hannel coding over mu ltiuser channels with correlated recei ver side i nformation. Du e to the lack of a general source-channel s eparation theorem for multi user channels, optimal performance in general requires j oint sou rce-channel coding. Giv en t he dif- ﬁculty of ﬁnding the optimal source-channel rate in a general sett ing, we hav e analyzed some fundamental building-blocks of the general settin g in terms of separation optim ality . Speciﬁcally , we ha ve characterized the necessary and sufﬁcient conditions for l ossless t ransmission over var ious fundamental mult iuser channels, such as multiple access, compound multiple access, interference and two-w ay channels for certain source-channel distributions and s tructures. In particular , we hav e considered transmi tting correlated sources over t he MA C with recei ver side information given which t he s ources are independent, and transmitting independent sources over the MA C with recei ver side information gi ven which the sources are correlated. For the compound MA C, we hav e provided an achie v ability result, whi ch has been shown to be tight i) when each source is independent of the o ther source and one of the side i nformation sequences, i i) when the so urces and t he side information are arbitrarily correlated but there is no m ultiple access interference at the receiv ers, iii) when the sources are correlated and the receive rs have access to the sam e side information given whi ch the two sources are in dependent. W e have then showed that for cases (i) and (iii ), the condi tions provided for the compou nd MA C are also necessary for interference channels under some strong source-channel condit ions. W e have also provided a lower bound on the achiev able source-channel rate for th e two-way channel. For the cases analyzed in this paper , we hav e proven the optimality of desig ning source and channel codes that are statistically independent of each other , hence resulting in a modular system design without los ing the end-to-end op timality . W e have s hown that, in some scenarios, thi s modularity can be diffe rent from the classical Shannon t ype separation, called the ‘informational separation’, in which comparison of the source coding rate region and the channel capacity region provides the necessary and su f ﬁcient conditions for the achiev abilit y of a source-channel rate. In other words, i nformational separation requires the separate codes used at the source and the channel coders t o be t he optimal source and t he channel cod es, respectiv ely , for the underlying model. Howev er , foll owing [6], we have shown here for a number of m ultiuser systems t hat a more general notion of ‘operational s eparation’ can hold even in cases for which DRAFT 38 informational separation fails to achiev e th e optim al s ource-channel rate. O perational separation requires statisticall y independent source and channel codes whi ch are not necessarily t he optimal codes for the und erlying sources or the channel. In t he case of operatio nal separation, comparison of two rate regions (not necessarily th e compression rate and the capacity regions) that depend only on the source and channel distributions, respectively , p rovides t he necessary and sufﬁcient conditions for l ossless transmiss ion of the sources. These results help us to o btain insights in to source and channel codi ng for larger multiuser networks, and potenti ally would lead to improved design princip les for p ractical imp lementations. A P P E N D I X I P RO O F O F T H E O R E M 5 . 3 Pr oof: The achie va bility again follows from separate source and channel coding. W e ﬁrst use Slepian-W olf compression of the sources condi tioned o n the receiv er si de inform ation, then transmit the compressed messages using an optimal mu ltiple access channel code. An alternative approach for the achiev abili ty is possible by cons idering W 1 as the o utput of a parallel channel from S 1 , S 2 to t he recei ver . Not e that this parallel channel is used m times for n uses of th e m ain channel. The achie va ble rates are then obtained following the ar guments for the st andard MA C: mH ( S 1 ) < I ( S m 1 , X n 1 ; Y n 1 , W m 1 | X n 2 , S m 2 , Q ) (89) = I ( S m 1 ; W m 1 | S m 2 ) + I ( X n 1 ; Y n 1 | X n 2 , Q ) (90) = mI ( S 1 ; W 1 | S 2 ) + nI ( X 1 ; Y 1 | X 2 , Q ) , (91) and using the fact that p ( s 1 , s 2 , w 1 ) = p ( s 1 ) p ( s 2 ) p ( w 1 | s 1 , s 2 ) we obtain (38) (similarly for (39) and (40)). N ote that, thi s approach provides achiev able source-channel rates for g eneral joint distributions of S 1 , S 2 and W 1 . DRAFT 39 For t he con verse, we use Fano’ s inequali ty given in (11 ) and (14). W e ha ve 1 n I ( X n 1 ; Y n 1 | X n 2 ) ≥ 1 n I ( S m 1 ; Y n 1 | X n 2 ) , (92) = 1 n I ( S m 1 , W m 1 ; Y n 1 | X n 2 ) , (93) ≥ 1 n I ( S m 1 ; Y n 1 | X n 2 , W m 1 ) , ≥ 1 n [ H ( S m 1 | S m 2 , W m 1 ) − mδ ( P ( m,n ) e )] , (94) ≥ 1 b [ H ( S 1 | S 2 , W 1 ) − δ ( P ( m,n ) e )] , where ( 92 ) follows from th e M arko v relation S m 1 − X n 1 − Y n 1 giv en X n 2 ; ( 93 ) from t he Marko v relation W m 1 − ( X n 2 , S m 1 ) − Y n 1 ; and ( 94 ) from Fano’ s i nequality (14). W e also have 1 n n X i =1 I ( X 1 i ; Y 1 ,i | X 2 i ) ≥ 1 n I ( X n 1 , X n 2 ; Y n 1 ) ≥ 1 b [ H ( S 1 | S 2 , W 1 ) − δ ( P ( m,n ) e )] . Similarly , we have 1 n n X i =1 I ( X 2 i ; Y 1 ,i | X 1 i ) ≥ 1 b [ H ( S 2 | S 1 , W 1 ) − δ ( P ( m,n ) e )] , and 1 n n X i =1 I ( X 1 i , X 2 i ; Y 1 ,i ) ≥ 1 b [ H ( S 1 , S 2 | W 1 ) − δ ( P ( m,n ) e )] . As usual, we let P ( m,n ) e → 0 , and i ntroduce the time sharing random v ariable Q uniformly distributed over { 1 , 2 , . . . , n } and independent of all the other random var iables. Then we deﬁne X 1 , X 1 Q , X 2 , X 2 Q and Y 1 , Y 1 Q . Note t hat P r { X 1 = x 1 , X 2 = x 2 | Q = q } = P r { X 1 | Q = q } · P r { X 2 | Q = q } si nce the two sources, and hence the channel codew ords, are independent of each other conditioned on Q . Thus , we obtain (38)-(40) for a joint distribution of the form (41). DRAFT 40 A P P E N D I X I I P RO O F O F T H E O R E M 6 . 3 W e hav e 1 n I ( X n 1 ; Y n 1 | X n 2 ) ≥ 1 n I ( S m 1 ; Y n 1 | X n 2 ) , (95) = 1 n [ H ( S m 1 | X n 2 ) − H ( S m 1 | Y n 1 , X n 2 )] , (96) ≥ 1 n [ H ( S m 1 ) − H ( S m 1 | Y n 1 )] , (97) ≥ 1 b h H ( S 1 ) − δ ( P ( m,n ) e ) i , (98) for any ǫ > 0 and suf ﬁcientl y large m and n , where ( 9 5 ) foll ows from the condit ional data processing inequali ty since S m 1 − X n 1 − Y n 1 forms a Markov chain gi ven X n 2 ; ( 97 ) from the independence of S m 1 and X n 2 and th e fact that condit ioning reduces entropy; and ( 98 ) from the memoryless source assumption, and from Fano’ s inequalit y . For t he joi nt mutual info rmation, we can write the following set of inequalities: 1 n I ( X n 1 , X n 2 ; Y n 1 ) ≥ 1 n I ( S m 1 , S m 2 ; Y n 1 ) , (99) = 1 n I ( S m 1 , S m 2 , W m 1 ; Y n 1 ) , (100) ≥ 1 n I ( S m 1 , S m 2 ; Y n 1 | W m 1 ) , (101) = 1 n [ H ( S m 1 , S m 2 | W m 1 ) − H ( S m 1 , S m 2 | Y n 1 , W m 1 )] , = 1 n [ H ( S m 1 ) + H ( S m 2 | W m 1 ) − H ( S m 1 , S m 2 | Y n 1 , W m 1 )] , (102) ≥ 1 b " H ( S 1 ) + H ( S 2 | W 1 ) − δ ( P ( m,n ) e ) # , (103) for any ǫ > 0 and sufﬁ ciently large m and n , where ( 99 ) follows from the data processing inequality since ( S m 1 , S m 2 ) − ( X n 1 , X n 2 ) − Y n 1 form a Markov chain; ( 100 ) from the Marko v relation W m 1 − ( S m 1 , S m 2 ) − Y n 1 ; ( 101 ) from the chain rul e and the no n-negati vity of the m utual information; ( 102 ) from the i ndependence of S m 1 and ( S m 2 , W m 1 ) ; and ( 1 03 ) from the m emoryless source assum ption and Fano’ s inequality . It is also possi ble to sh ow that n X i =1 I ( X 1 i ; Y 1 i | X 2 i ) ≥ I ( X n 1 ; Y n 1 | X n 2 ) , (104) DRAFT 41 and si milarly for o ther mut ual information t erms. Then, usi ng the above set of inequali ties and letting P ( m,n ) e → 0 , we obtain 1 b H ( S 1 ) ≤ 1 n n X i =1 I ( X 1 i ; Y 1 i | X 2 i ) , 1 b H ( S 2 | W 1 ) ≤ 1 n n X i =1 I ( X 2 i ; Y 1 i | X 1 i ) , and 1 b ( H ( S 1 ) + H ( S 2 | W 1 )) ≤ 1 n n X i =1 I ( X 1 i , X 2 i ; Y 1 i ) , for any product distri bution on X 1 × X 2 . W e can write si milar expressions for the second recei ver as well. Then the necessi ty of the conditions of Theorem 6.2 can be ar gued si mply by inserting the t ime-sharing random variable Q . R E F E R E N C E S [1] I. Csisz ´ ar and J. K ¨ orner , Information Theory: Coding T heor ems for Discr ete Memoryless Systems, New Y ork: Academic, 1981. [2] S. V emb u, S. V erd ´ u and Y . Steinberg, “The source-channel separation theorem revisited, ” IE EE T ran s. Information Theory , vol. 41, no. 1, pp. 44-54, Jan. 1995. [3] C. E. S hannon, “T wo-way communication channels, ” in Pr oc. 4th Berkele y Symp. Math. Satist. P r obability , vol. 1, 1961, pp. 611-644 . [4] T . M. Cove r , A. E l Gamal and M. Salehi, “Multiple access channels with arbitr arily correlated sources, ” IEEE Tr ans. Information Theory , vol. 26, no. 6, pp. 648-657, Nov . 1980. [5] D. Slepian and J. W olf, “Noiseless coding of correlated i nformation sources, ” IEEE Tr ans. Information Theory , vol. 19, no. 4, pp. 471-480, Jul. 1973. [6] E. Tuncel, “Sl epian-W olf coding ov er broadcast channels, ” IEEE T rans. Information T heory , vol. 52, no. 4, pp. 1469-14 82, April 2006. [7] T . S. Han, “Slepian-W olf-Cove r theorem for a network of channels, ” Information and Contr ol , vol. 47, no. 1, pp. 67-83, 1980. [8] K. de Br uyn, V . V . Pr elov and E. van der Meulen, “Reliable transmission of two correlated sources ov er an asymmetric multiple-access channel, ” IEEE T ran s. Information Theory , vol. 33, no. 5, pp. 716-718, Sep. 1987. [9] M. Effros, M. M ´ edard, T . Ho, S. Ray , D. Karger and R. Koetter , “Linear network codes: A Uniﬁed F rame work for Source Channel, and Network Coding, ” in DIMACS Series in Discr ete Mathematics and Theor etical Computer Science , vo l. 66, American Mathematical Society , 2004. [10] M. H. M. Costa and A. A. El Gamal, “The capacity region of the discrete memoryless interference channel with strong interference, ” IE EE T rans. Information Theory , vol. 33, no. 5, pp. 710-711, Sep. 1987. [11] G. Dueck, “ A note on the multiple access channel with correlated sources, ” IEE E T ran s. Information Theory , vol. 27, no. 2, pp. 232-235, Mar . 1981. DRAFT 42 [12] R. Ahlswede and T . S. Han, “On source coding with side information via a multiple access channel and related problems in multiuser information theory , ” IEEE T ran s. Information Theory , vol. 29, no.3, pp.396412, May 1983. [13] T . S. Han and M. H . M. Costa, “Br oadcast channels wit h arbitrarily correlated sources, ” IEE E Tr ans. Information Theory , vol. 33, no. 5, pp. 641-650, Sep. 1987. [14] M. Salehi and E. Kurtas, “Interference channels with correlated sources, ” Proc. IEEE International Symposium on Information Theory , San Antonio, T X, Jan. 1993. [15] S. Choi and S. S. Pradhan, “ A graph-based framew ork for transmission of correlated sources ov er broadcast channels, ” IEEE T rans. Information Theory , 2008. [16] S. S. Pradhan, S. Choi and K. Ramchandran, “ A graph-based framewo rk for transmission of correlated sources o ver multiple access channels, ” IEEE T ran s. Information Theory , vol. 53, no. 12, pp. 4583-4604, Dec. 2007. [17] W . K ang and S. Ulukus, “ A new data processing inequality and it s applications in distributed source and channel coding, ” submitted to IEEE T ran s. Information Theory . [18] D. G ¨ und ¨ uz and E. E rkip, “Reliable cooperativ e source transmission with side information, ” P r oc. IEEE Information Theory W orkshop (ITW) , Bergen, Norway , July 2007. [19] P . E lias, “List decoding for noisy channels, ” T echnical report, Research Laboratory of Electronics, MIT , 1957. [20] Y . W u, “Broadcasting when receiv ers kno w some messages a priori, ” P r oc. IEEE International Symp. on Information Theory (ISIT) , Nice, France, Jun. 2007. [21] F . Xue and S. Sandhu, “PHY -layer network coding for broadcast channel with si de i nformation, ” Pr oc. IEEE Information Theory W orkshop (ITW) , Lake T ahoe, CA, Sep. 2007. [22] G. Kramer and S. S hamai, “Capacity for classes of broadcast channels with receiv er side information, ” Pro c. IEEE Information Theory W orkshop (ITW) , Lake T ahoe, CA, Sep. 2007. [23] W . Kang, G. Kramer, “Broadcast channel with degrade d source random variables and recei ve r side information, ” Pr oc. IEEE International Symp. on Information T heory (ISIT) , T oronto, C anada, July 2008. [24] T . J. Oechtering, C. Schnurr , I . Bjelako vic, H. Boche, “Broadcast capacity region of two-phase bidirectional relaying, ” IEEE T rans. Information Theory , vol. 54, no. 1, pp. 454-458, Jan. 2008. [25] D. G ¨ und ¨ uz , E . Tuncel and J. Nayak, “Rate regions for the separated two-way relay channel, ” 46th Allerton Confer ence on Communication, Contr ol, and Computing , Monticello, IL, Sep. 2008. [26] P . G ´ acs and J. K ¨ orner , “Common information is far less than mutual information, ” Pro blems of C ontr ol and Information Theory , vol. 2, no. 2, pp. 149-162, 1973. [27] R. Ahlswede, “The capacity region of a channel wit h two senders and two receiv ers, ” Annals of Pr obability , vol. 2, no. 5, pp. 805-814 , 1974. [28] D. G ¨ und ¨ u z and E. Erkip, “Lossless tr ansmission of correlated sources over a multiple access chann el with side information, ” Data Compr ession Confer ence (DCC) , Snowbird, UT , March 2007. [29] D. G ¨ und ¨ uz and E. Erkip, “Interference channel and compoun d MA C with correlated sources and receiv er side information, ” IEEE Int’l Symposium on Information Theory ( ISIT) , Nice, France, June 2007. [30] J. Barros and S . D. Servetto, “Network information ﬂow with correlated sources, ” IEEE T rans. Information Theory , vol. 52, no. 1, pp. 155-170, Jan. 2006. [31] T . P . Coleman, E. Martinian and E. Ordentlich, “Joint source-channel decoding for t ransmitting correlated sources ov er broadcast networks, ” submitted to IEEE T rans. Information Theory . DRAFT 43 [32] T . S. Han and K. Kobay ashi, “ A new achie v able rate region for the interference channel, ” IEEE Tr ans. Information Theory , vol. 27, no.1, pp. 49-60, Jan. 1981. [33] X. Shang, G. Kramer, and B. Chen, “ A new outer bound and the noisy-interference sum-rate capacity for Gaussian interference channels, ” submitted to the IEEE T rans. Information Theory . [34] A. S . Motahari and A . K. Khandani, “Capacity bounds for the Gaussian interference channel, ” submitted to IEE E Tr ans. Information Theory . [35] V . S. Annapureddy and V . V eerav alli, “Sum capacity of the Gaussian i nterference channel i n the low interference regime, ” submitted to IEEE T ran s. Inform. Theory . [36] H. Sato, “The capacity of the Gaussian i nterference channel under strong interference, ” IEEE T rans. Information Theory , vol. 27, no. 6, pp. 786-788, Nov . 1981. [37] Z. Zhang, T . Berger , and J. P . M. Schalkwijk, “Ne w outer bounds to capacity regions of two-way channels, ” IEEE Tr ans. Information Theory , vol. 32, no. 3,pp. 383386, May 1986. [38] A. P . Hekstra and F . M. J. W illems, “Depende nce balance bounds for single output two-way channels, ” IE EE T rans. Information Theory , vol. 35, no. 1, pp. 4453, Jan. 1989. [39] R. T andon and S. Ul ukus, “On dependence balance bound s for two-way channels, ” Pr oc. 41st Asilomar Confer ence on Signals, Systems and Computers , Paciﬁc Grov e, CA, Nov . 2007. DRAFT

Source and Channel Coding for Correlated Sources Over Multiuser Channels

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment