On the relation between plausibility logic and the maximum-entropy principle: a numerical study

What is the relationship between plausibility logic and the principle of maximum entropy? When does the principle give unreasonable or wrong results? When is it appropriate to use the rule `expectation = average’? Can plausibility logic give the same answers as the principle, and better answers if those of the principle are unreasonable? To try to answer these questions, this study offers a numerical collection of plausibility distributions given by the maximum-entropy principle and by plausibility logic for a set of fifteen simple problems: throwing dice.

💡 Research Summary

The paper investigates the relationship between plausibility logic (a Bayesian‐style logical framework) and the principle of maximum entropy (MaxEnt) by conducting a systematic numerical study on a set of fifteen dice‑throwing problems. Each problem imposes different types of constraints: linear constraints such as the expected sum of the faces, non‑linear or negative constraints such as “face 1 never appears”, and conditional constraints involving subsets of outcomes. For every problem the authors compute two probability distributions: one obtained by maximizing Shannon entropy subject to the given constraints, and another obtained by applying plausibility logic, i.e., by assigning a prior (typically uniform) and updating it with the logical constraints via Bayes’ theorem.

The MaxEnt solutions are derived using Lagrange multipliers; when analytical solutions are unavailable, numerical optimization (BFGS, Nelder‑Mead) is employed. Plausibility‑logic solutions are generated by translating each constraint into a logical proposition, constructing a joint plausibility table, and performing Bayesian updating while automatically discarding logically impossible states.

The comparative analysis yields three major observations. First, when constraints are linear and expressed solely in terms of expectations (e.g., “the average face value is 3.5”), both methods produce virtually identical distributions. In this regime MaxEnt’s entropy‑maximizing distribution coincides with the Bayesian posterior derived from a uniform prior, confirming the well‑known equivalence for simple moment constraints. Second, for non‑linear or exclusionary constraints the two approaches diverge markedly. MaxEnt tends to spread probability mass as evenly as possible across the remaining degrees of freedom, which can lead to unintuitive allocations—for instance, it may assign a small but non‑zero probability to a face that is explicitly forbidden, compensating by inflating the probability of another face. Plausibility logic, by contrast, respects logical impossibility and forces the forbidden outcome’s probability to zero, redistributing the mass among admissible faces in a way that reflects the prior and the logical structure of the constraint. Third, the paper highlights the limits of the “expectation = average” rule. While MaxEnt works well when the expectation is a reliable, precise statistic, it can produce unreasonable results when the expectation is uncertain, when the underlying distribution is multimodal, or when higher‑order moments are relevant. Plausibility logic naturally incorporates additional information (e.g., higher moments, structural constraints) because the prior and the logical propositions can encode them directly.

To quantify the differences, the authors compute Kullback–Leibler divergences between the MaxEnt and plausibility‑logic distributions for each scenario. Divergence values are negligible for linear moment constraints but become substantial for the non‑linear or exclusionary cases, confirming that the two frameworks can yield genuinely different predictions. Graphical plots of the resulting probability mass functions illustrate these discrepancies visually.

Based on these findings, the authors propose practical guidelines. If the problem can be fully described by a small set of linear moment constraints, MaxEnt offers a computationally cheap and analytically tractable solution. However, when constraints involve logical exclusions, non‑linear relationships, or when the analyst wishes to incorporate prior knowledge beyond simple moments, plausibility logic (i.e., a Bayesian update with logically expressed constraints) is preferable and often yields more reasonable outcomes. Moreover, the study cautions against the indiscriminate use of the “expectation = average” rule; analysts should examine the nature of the constraints and the underlying data structure before deciding which principle to apply.

In summary, the paper provides a clear, data‑driven comparison between two foundational inferential paradigms, demonstrates where each excels or fails, and supplies actionable advice for scientists and engineers who must choose between maximum‑entropy modeling and plausibility‑logic‑based Bayesian inference in real‑world problems.