Joint Source-Channel Coding Revisited: Information-Spectrum Approach

Typically, the traditional separation theorem, which we call the separation theorem in the narrow sense, states that if the infimum R f (V) of all achievable fixed-length coding rates for the source V is smaller than the capacity C(W) for the channel W, then the source V is reliably transmissible by two step encoding over the channel W; whereas if R f (V) is larger than C(W) then the reliable transmission is impossible. While the former statement is always true for any general source V and any general channel W, the latter statement is not always true. Then, a very natural question may be raised for what class of sources and channels and in what sense the separation theorem holds in general.

Shannon [1] has first shown that the separation theorem holds for the class of stationary memoryless sources and channels. Since then, this theorem has received extensive attention by a number of researchers who have attempted to prove versions that apply to more and more general classes of sources and channels. Among others, for example, Dobrushin [4], Pinsker [5], and Hu [6] have studied the separation theorem problem in the framework of information-stable sources and channels.

Recently, on the other hand, Vembu, Verdú and Steinberg [9] have put forth this problem in a much more general information-spectrum context with general source V and general channel W. From the viewpoint of information spectra, they have generalized the notion of separation theorem and shown that, usually in many cases even with R f (V) > C(W), it is possible to reliably transmit the output of the source V over the channel W. Furthermore, in terms of information spectra, they have established a sufficient condition for the transmissibility as well as a necessary condition. It should be noticed here that, in this general joint source-channel coding situation, what indeed matters is not the validity problem of the traditional type of separation theorems but the derivation problem of necessary and/or sufficient conditions for the transmissibility from the information-spectrum point of view.

However, while their sufficient condition looks simple and significantly tight, their necessary condition does not look quite close to tight.

The present paper was mainly motivated by the reasonable question why the forms of these two conditions look rather very different from one another. First, in Section 3, the basic tools to answer this question are established, i.e., two fundamental lemmas: a generalization of Feinstein’s lemma [2] and a generalization of Verdú-Han’s lemma [8], which provide with the very basis for the key results to be stated in the subsequent sections. These lemmas are of dualistic information-spectrum forms, which is in nice accordance with the general joint source-channel coding framework. In Section 4, given a general source V and a general channel W, we establish, in terms of information-spectra, a sufficient condition (Direct theorem) for the transmissibility as well as a necessary condition (Converse theorem). The forms of these two conditions are very close from each other, and “fairly” coincides with one another, provided that we dare disregard some relevant asymptotically vanishing term.

Next, we equivalently rewrite these conditions in the forms useful to see relations to the separation theorem. As a consequence, it turns out that a separation-theorem-like equivalent of our sufficient condition just coincides with the sufficient condition given by Vembu, Verdú and Steinberg [9], whereas a separation-theorem-like equivalent of our necessary condition is shown to be strictly stronger than or equivalent to the necessary condition given by them. Here it is pleasing to observe that a nice duality is found between our necessary and sufficient conditions, whereas we cannot fully enjoy such a duality between the necessary condition and the sufficient condition by Vembu, Verdú and Steinberg [9].

On the other hand, in Section 5, we demonstrate a sufficient condition as well as a necessary condition for the ε-transmissibility, which is the generalization of the sufficient condition as well as the necessary condition as was shown in Section 4. Finally, in Section 6, we restrict the class of sources and channels to those that satisfy the strong converse property (or, more generally, the semi-strong converse property) to show tha

…(Full text truncated)…