Test Martingales, Bayes Factors and $p$-Values

A nonnegative martingale with initial value equal to one measures evidence against a probabilistic hypothesis. The inverse of its value at some stopping time can be interpreted as a Bayes factor. If we exaggerate the evidence by considering the largest value attained so far by such a martingale, the exaggeration will be limited, and there are systematic ways to eliminate it. The inverse of the exaggerated value at some stopping time can be interpreted as a $p$-value. We give a simple characterization of all increasing functions that eliminate the exaggeration.

💡 Research Summary

The paper introduces a unified framework that connects non‑negative martingales, Bayes factors, and p‑values, offering a principled way to quantify statistical evidence while controlling for the inevitable “exaggeration” that arises when one looks at the maximal value a martingale attains. The authors start by defining a non‑negative martingale (M_t) with initial value (M_0=1). Because a martingale has constant expectation, the process can be interpreted as a fair betting game: as the value of (M_t) grows, the underlying probabilistic hypothesis becomes less plausible. The key observation is that the reciprocal of the martingale at a stopping time (\tau), namely (1/M_\tau), coincides with a Bayes factor for the hypothesis under consideration. In Bayesian terms, this factor is the ratio of posterior to prior odds, and the martingale construction automatically supplies it without the need to specify a prior distribution explicitly.

However, the raw martingale can temporarily achieve very large values, and if one simply takes the inverse of the largest value observed up to a stopping time, the resulting “p‑value” can be far smaller than warranted – a phenomenon the authors call evidence exaggeration. To remedy this, they introduce an “exaggeration‑eliminating function” (f). The function must be monotone increasing and satisfy (f(x)\le x) for all (x\ge 1). The corrected evidence at a stopping time (\tau^) is then (1/f(M_{\tau^}^)), where (M_{\tau^}^) denotes the supremum of the martingale up to (\tau^). This quantity can be interpreted as a valid p‑value.

The central theoretical contribution is a complete characterization of all admissible functions (f). The authors prove that any such function can be written as \