Using Relative Entropy to Find Optimal Approximations: an Application to Simple Fluids

but why this particular functional and not another? And also, why limit oneself to an approximation by a single member of the trial family? Why not consider a linear combination of the trial distributions, some kind of average over the trial family? But then, how should we choose the optimal weight assigned to each p(x)? We propose to tackle these questions using the method of Maximum relative Entropy (which we abbreviate as ME) and information geometry [1]. The ME method, which is developed in [2]- [8], has historical roots in the earlier method of maximum entropy that was pioneered by E. T. Jaynes and is commonly known as MaxEnt [9]. The ME method is designed for updating probabilities from arbitrary priors for information in the form of arbitrary constraints and it includes Bayes’ rule and the older MaxEnt as special cases [7], [8]. The purpose of this paper is to develop a ME based method to generate optimal approximations (brief accounts of some of the results discussed below have previously been presented in [10] and [11]). The general formalism, which is the main result of this paper, is developed in section 2. In section 2.1 we justify the use of relative entropy as the unique and natural criterion to select the preferred approximation, which is labeled by some parameters. The optimal approximation is obtained in section 2.2 by marginalizing over the variational parameters. A suitably weighed average over the whole family of trial distributions with the optimal weight provides an optimal approximation.

In the second part of the paper we demonstrate the proposed formalism by applying it to simple classical fluids, a well studied field in the past [12]- [15]. To approximate the behavior of simple fluids we chose trial distributions that describe hard spheres [12]- [15] (section 3). The ME formalism is first used (section 4.1) to select the preferred value of the hard-sphere diameter. This is equivalent to applying the Bogoliubov variational principle and reproduces the results obtained by Mansoori et al. [16] whose variational principle was justified by a very different argument.

An advantage of the variational or the ME methods over the perturbative approaches such as Barker and Henderson (BH) [12] and of Weeks, Chandler and Anderson (WCA) [13] is that there is no need for ad hoc criteria dictating how to separate the intermolecular potential into a strong short range repulsion and a weak long range attraction. On the other hand, a disadvantage of the standard variational approach is that it fails to take the softness of the repulsive core into account. At high temperatures this leads to results that are inferior to the perturbative approaches.

In the standard variational a single preferred value of the hard-sphere diameter is selected. But, as discussed in [7] and [17], in the ME method non-preferred values are not completely ruled out. This allows us (in section 4.2) to marginalize over hard sphere diameter to obtain an optimal hardsphere approximation with a suitable weighting. That leads to significant improvements over the standard variational method.

In section 5 we test our method by comparing its predictions for a Lennard-Jones model for argon with molecular dynamics simulation data ( [18], [19]). We find that the ME predictions for thermodynamic variables and for the radial distribution function are considerable improvements over the standard Bogoliubov variational result, and are comparable to the perturbative results [12] [13]. (For a recent discussion of some of the strengths and limitations of the perturbative approach see [20].). Despite the shortcomings of perturbation theory it remains very popular because it provides quantitative insights at a much lower computational cost than dynamical simulations. For recent applications to the glass transition and other more complex systems see [21] - [25]. Although in this work the ME is not applied to such complex problems one may fully expect that the information theory based ME method will yield insights not only the thermodynamic behavior of complex systems but also about the approximation methods needed to analyze them. Finally, our conclusions and some remarks on further improvements are presented in section 6.

Consider a system with microstates labeled by q (for example, the location in phase space or perhaps the values of spin variables). Let the probability that the microstate lies within a particular range dq be given by the int

…(Full text truncated)…