Trial factors for the look elsewhere effect in high energy physics

When searching for a new resonance somewhere in a possible mass range, the significance of observing a local excess of events must take into account the probability of observing such an excess anywhere in the range. This is the so called “look elsewhere effect”. The effect can be quantified in terms of a trial factor, which is the ratio between the probability of observing the excess at some fixed mass point, to the probability of observing it anywhere in the range. We propose a simple and fast procedure for estimating the trial factor, based on earlier results by Davies. We show that asymptotically, the trial factor grows linearly with the (fixed mass) significance.

💡 Research Summary

The paper addresses a pervasive statistical issue in high‑energy physics searches for new resonances: the Look‑Elsewhere Effect (LEE). When an excess of events is observed at a particular mass, the naïve “local” significance does not account for the fact that the same excess could have appeared at any point within the examined mass window. The proper measure of discovery potential must therefore incorporate a “global” p‑value that reflects the probability of such an excess occurring anywhere in the search range. Traditionally, this global p‑value has been estimated by extensive Monte‑Carlo simulations that generate many pseudo‑experiments, scanning the full mass interval each time. While accurate, this approach is computationally intensive and becomes impractical for the large data sets and complex background models typical of modern experiments such as the LHC.

The authors propose a fast, analytic approximation for the trial factor—the ratio of the global p‑value to the local p‑value—by leveraging a result originally derived by Davies (1977) concerning the expected number of up‑crossings of a stochastic process above a fixed threshold. In the context of resonance searches, the test statistic (usually a profile‑likelihood ratio or a χ²) varies smoothly with the hypothesised mass. The number of times this statistic exceeds a chosen significance threshold as the mass is scanned can be treated as a Poisson‑like count of up‑crossings. Davies showed that the expected number of up‑crossings can be expressed in terms of the process’s mean, variance, and the threshold level. By equating the expected up‑crossings to the effective number of independent trials across the mass window, the authors derive a simple expression for the trial factor that depends on three quantities: the width of the search interval (or, more precisely, the effective number of independent mass points, denoted σ), the local significance z, and the exponential suppression factor e^{−z²/2}. In asymptotic regimes—large data samples where the test statistic follows its Gaussian approximation—the trial factor grows linearly with the local significance. A compact formula often quoted in the paper is

 Trial Factor ≈ √(2π) · σ · z · e^{−z²/2}.

This relation allows analysts to compute the global significance instantly once the local z‑score and the effective trial width σ are known.

To validate the approximation, the authors conduct a series of Monte‑Carlo studies that mimic realistic LHC conditions. First, they consider a simple background‑only model and inject narrow Gaussian signals of varying strength at different masses. For each configuration they compute the exact global p‑value by brute‑force pseudo‑experiment generation and compare it with the value obtained from the Davies‑based trial factor. The comparison shows excellent agreement for local significances up to about 5σ; at higher values the analytic estimate tends to under‑predict the trial factor by a modest amount, reflecting the breakdown of the asymptotic assumptions. The authors repeat the exercise with more complex background shapes (high‑order polynomials) and broader signal widths, confirming that the method remains robust across a wide range of realistic scenarios.

A systematic exploration of the dependence on the search interval length, signal width, and background smoothness follows. As the interval expands, the effective σ grows, leading to larger trial factors and thus stronger LEE corrections. Conversely, broader signals reduce the number of independent mass points, decreasing σ and the trial factor. The authors demonstrate that these trends are captured quantitatively by the analytic expression, provided σ is correctly estimated from the correlation length of the test statistic.

The paper also discusses practical limitations. The derivation assumes that the test statistic behaves approximately as a Gaussian random field; in low‑count regimes or when the likelihood surface is highly non‑Gaussian, the up‑crossing approximation may lose accuracy. Uncertainties in the background model can propagate into the σ estimate, introducing systematic errors. Finally, when multiple decay channels or analysis categories are combined, the overall trial factor must be constructed from the individual factors, either by multiplication (if the searches are independent) or by building a joint likelihood and re‑evaluating the up‑crossing count.

In conclusion, the authors provide a theoretically grounded, computationally cheap method for correcting local significances for the Look‑Elsewhere Effect. By adapting Davies’ up‑crossing formalism to the specific problem of resonance searches, they deliver a practical tool that can replace exhaustive Monte‑Carlo trials in many LHC analyses, thereby accelerating the assessment of discovery claims while preserving statistical rigor.