On the statistical physics of directed polymers in a random medium and their relation to tree codes
📝 Abstract
Using well-known results from statistical physics, concerning the almost-sure behavior of the free energy of directed polymers in a random medium, we prove that random tree codes achieve the distortion-rate function almost surely under a certain symmetry condition.
💡 Analysis
Using well-known results from statistical physics, concerning the almost-sure behavior of the free energy of directed polymers in a random medium, we prove that random tree codes achieve the distortion-rate function almost surely under a certain symmetry condition.
📄 Content
Tree source coding with a fidelity criterion has been studied since the late sixties and the early seventies of the previous century, see, e.g., [1,Subsection 6.2.4], [6], [10], [12], [14], [15]. The first results, that were obtained by Jelinek and Anderson [15], were for tree coding of binary sources with the Hamming distortion measure, and by Dick, Berger and Jelinek [10] for Gaussian sources and the squared error distortion measure. Davis and Hellman [6] proved a tree coding theorem for a general memoryless source and a general fidelity criterion. In particular, they pointed out that in an earlier paper by Jelinek [14], the proof of the coding theorem was valid only for symmetric sources, and so, by modifying the branching process associated with the tree code, they were able to relax the symmetry condition of the tree coding theorem. In this context, it should be pointed out that Gallager [12] also made a symmetry assumption in the same spirit.
The main message in this short paper is, first of all, in the observation that the tree source coding problem is very intimately related to an important model in statistical physics of disordered systems, namely, the directed polymer in a random medium (DPRM), cf. e.g., [2], [3], [4], [5], [7], [8], [11], [18], [19] and references therein. Loosely speaking, in the DPRM, each configuration of the underlying physical system corresponds to a walk along consecutive bonds of a certain lattice, or a tree, where each such bond is assigned with an independent random variable (energy), and where the total energy (which is analogous to the distortion of the tree code) of this walk is the sum of energies along the bonds visited. For a given realization of these random energy variables, the probability of each walk is given by the Boltzmann distribution, namely, it is proportional to an exponential function of the negative total energy. The main challenge, as usual in equilibrium statistical physics, is to characterize the asymptotic normalized free energy of a typical realization of the system. For the case where the walks are defined on a tree (from the root to one of the leaves), this problem has a closed-form solution.
This relationship between tree codes and the DPRM is interesting on its own right. It turns out to be so strong, that the various analysis techinques 1 and the results concerning the DPRM can readily be harnessed to the ensemble peformance analysis of tree codes. In particular, the distortion achieved by the best codeword in the tree codebook is identified with the free energy 1 These techniques are different from those of the papers mentioned in the first paragraph.
of the DPRM when the system is frozen (taken to zero temperature). This observation, does not merely provide an alternative proof of the tree coding theorem, but moreover, it enables to show that, at least under a certain symmetry assumption concerning the source and the distortion function 2 , the distortion-rate function is achieved eventually almost surely (with respect to the randomness of the code) for every individual source sequence. This is different from (and stronger than) the previous findings, mentioned in the first paragraph above, which were coding theorems concerning the average distortion.
The outline of this work is as follows: In Section 2, we establish our notation conventions and give a brief background in statistical mechanics in general and on the DPRM in particular. In Section 3, we show how the solution to the DPRM model can be used to prove that the tree code ensemble achieves distortion-rate function almost surely for every input. Finally, in Section 4, we provide a short summary of this paper.
Throughout this paper, scalar random variables (RV’s) will be denoted by capital letters, like X and Y , their sample values will be denoted by the respective lower case letters, and their alphabets will be denoted by the respective calligraphic letters. A similar convention will apply to random vectors and their sample values, which will be denoted with the same symbols in the boldface font. Thus, for example, X will denote a random n-vector (X 1 , . . . , X n ), and x = (x 1 , …, x n ) is a specific vector value in X n , the n-th Cartesian power of X . Sources and other probability measures that underly sequence generation will be denoted generically by the letters P and Q, and specific letter probabilities will be denoted by the corresponding lower case letters, e.g., p(x), q(y), etc. The expectation operator will be denoted by E{•}. Information theoretic quantities like entropies and mutual informations will be denoted following the usual conventions of the Information Theory literature.
Consider a physical system with n particles, which can be in a variety of microscopic states (‘microstates’), defined by combinations of physical quantities associated with these particles, e.g., positions, momenta, angular momenta, spins, etc., of all n particles. For each such mic
This content is AI-processed based on ArXiv data.