Measurement Invariance, Entropy, and Probability

We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use measurement scale as a type of information constraint. We argue that a very common measurement scale is linear at small magnitudes grading into logarithmic at large magnitudes, leading to observations that often follow Student’s probability distribution which has a Gaussian shape for small fluctuations from the mean and a power law shape for large fluctuations from the mean. An inverse scaling often arises in which measures naturally grade from logarithmic to linear as one moves from small to large magnitudes, leading to observations that often follow a gamma probability distribution. A gamma distribution has a power law shape for small magnitudes and an exponential shape for large magnitudes. The two measurement scales are natural inverses connected by the Laplace integral transform. This inversion connects the two major scaling patterns commonly found in nature. We also show that superstatistics is a special case of an integral transform, and thus can be understood as a particular way in which to change the scale of measurement. Incorporating information about measurement scale into maximum entropy provides a general approach to the relations between measurement, information and probability.

💡 Research Summary

The paper proposes a unified framework that links the way we measure a quantity to the probability distribution that most plausibly describes the observed data. Traditional maximum‑entropy (MaxEnt) methods generate a distribution by imposing constraints on a few statistical moments (e.g., mean, variance). However, they ignore the fact that the measurement process itself often applies a non‑linear transformation to the raw variable. The authors formalize this idea by introducing “measurement scale” as an additional information constraint, grounded in the principle of measurement invariance: the functional form of the transformation that leaves the essential statistical structure unchanged.

Two canonical measurement‑scale families are examined. The first, “linear‑to‑log” scaling, behaves linearly for small magnitudes (so the measured value is essentially the raw value) and gradually transitions to a logarithmic response for large magnitudes (as many sensors, human perception, or economic reporting do). When this scaling is encoded as a constraint in the MaxEnt variational problem, the resulting optimal density is the Student‑t distribution. The Student‑t has a Gaussian core—reflecting the linear region—and power‑law tails—reflecting the logarithmic region. Consequently, it naturally captures data that exhibit frequent small fluctuations but occasional extreme outliers, a pattern observed in physics (velocity distributions), finance (asset returns), and many other domains.

The second family, “log‑to‑linear” scaling, is the inverse: measurements are logarithmic for small values (e.g., when dealing with quantities that span many orders of magnitude) and become linear for large values. Imposing this scaling as a MaxEnt constraint yields the gamma distribution. The gamma density displays a power‑law rise at low values (consistent with the log region) and an exponential decay at high values (consistent with the linear region). This shape fits phenomena such as reaction times, pollutant concentrations, and biological expression levels.

A central theoretical contribution is the demonstration that the two families are mathematically linked by the Laplace integral transform. The Student‑t and gamma densities form a transform pair: applying the Laplace transform to one produces the other. This relationship shows that changing the measurement scale is equivalent to applying a specific integral transform to the underlying probability law. In other words, the “scale change” is not an ad‑hoc re‑parameterization but a well‑defined operation that preserves the information content while reshaping the distribution.

The authors further argue that superstatistics—a framework in which a system experiences slowly varying intensive parameters (e.g., temperature, volatility) and thus exhibits a mixture of local equilibrium distributions—is a special case of this transform‑based view. Superstatistical mixtures can be written as Laplace transforms of conditional distributions, meaning that superstatistics is essentially a particular way of changing measurement scale across hierarchical levels of a complex system.

Empirical illustrations are provided across several fields. In turbulence and kinetic theory, particle speed data follow a Student‑t shape, confirming the linear‑to‑log scaling hypothesis. In ecological and genomic data, count distributions are better described by gamma laws, supporting the log‑to‑linear scaling. In each case, incorporating the appropriate measurement‑scale constraint into MaxEnt yields a markedly better fit than traditional moment‑based MaxEnt models.

The paper concludes that measurement scale is a fundamental source of information that should be treated on equal footing with moment constraints in entropy‑based inference. By doing so, one obtains a principled explanation for why the Student‑t and gamma families appear so ubiquitously in nature, and it provides a coherent mathematical bridge—via the Laplace transform—between seemingly disparate scaling patterns. This perspective invites researchers to explicitly model the measurement process when designing experiments, analyzing data, or selecting statistical models, thereby improving both interpretability and predictive performance across the physical, biological, and social sciences.