Is the Information Entropy the Same as the Statistical Mechanical Entropy?

It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the expression for the entropy as a function of the probability. A simple derivation of the general formula is given.

💡 Research Summary

The paper addresses a subtle but important distinction between the Shannon information entropy, widely used in information theory, and the thermodynamic entropy that appears in statistical mechanics. While the familiar Shannon formula

\