A Coverage Study of the CMSSM Based on ATLAS Sensitivity Using Fast Neural Networks Techniques

We assess the coverage properties of confidence and credible intervals on the CMSSM parameter space inferred from a Bayesian posterior and the profile likelihood based on an ATLAS sensitivity study. In order to make those calculations feasible, we introduce a new method based on neural networks to approximate the mapping between CMSSM parameters and weak-scale particle masses. Our method reduces the computational effort needed to sample the CMSSM parameter space by a factor of ~ 10^4 with respect to conventional techniques. We find that both the Bayesian posterior and the profile likelihood intervals can significantly over-cover and identify the origin of this effect to physical boundaries in the parameter space. Finally, we point out that the effects intrinsic to the statistical procedure are conflated with simplifications to the likelihood functions from the experiments themselves.

💡 Research Summary

This paper investigates the coverage properties of confidence intervals derived from the profile likelihood and credible intervals obtained from the Bayesian posterior for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when interpreted through an ATLAS supersymmetry sensitivity study. A major obstacle in such an analysis is the computational cost associated with repeatedly translating high‑scale CMSSM parameters (m₀, m₁/₂, A₀, tan β, sign μ) into low‑energy particle masses using full renormalization‑group calculations (e.g., SoftSUSY). To overcome this, the authors train a multilayer perceptron neural network on a dataset of one million CMSSM points generated with SoftSUSY. The network learns the non‑linear mapping from the five input parameters to a set of weak‑scale masses and mixing matrices. Validation shows mean absolute errors well below 1 GeV, far smaller than the experimental mass resolutions, allowing the surrogate model to replace the full spectrum calculator without compromising physical fidelity.

Using this surrogate, the authors achieve a speed‑up of roughly four orders of magnitude (≈10 ms per point versus ≈100 s with the full calculation). This makes it feasible to perform large‑scale statistical studies that would otherwise be prohibitive. For the statistical analysis, two parallel approaches are employed. In the Bayesian case, MultiNest is used to sample the posterior distribution, and 68 % and 95 % credible regions are defined from the resulting density. In the frequentist case, the profile likelihood is maximized over nuisance parameters (A₀, tan β, sign μ) to obtain confidence regions at the same nominal coverage levels. Both analyses rely on the ATLAS likelihood function supplied for the 0‑lepton + jets + missing E_T search; this likelihood incorporates simplified treatments of systematic uncertainties and background correlations.

To assess coverage, the authors generate 10⁴ pseudo‑experiments. For each experiment a “true” CMSSM point is drawn, the corresponding spectrum is obtained via the neural network, and synthetic data are sampled from the ATLAS likelihood. The Bayesian posterior and the profile likelihood are then recomputed for each pseudo‑dataset, and the fraction of experiments in which the true parameters fall inside the nominal 68 % or 95 % intervals is recorded.

The results reveal systematic over‑coverage for both statistical paradigms. The effect is most pronounced near the physical boundaries of the CMSSM parameter space, such as the requirement of positive scalar masses, successful electroweak symmetry breaking, and a neutralino Lightest Supersymmetric Particle. Because the allowed region is truncated by these constraints, the sampling density becomes skewed toward the boundary, causing the Bayesian posterior to accumulate probability mass at the edge and the profile likelihood to produce broader intervals when the maximum is forced against the boundary. Consequently, the empirical coverage far exceeds the nominal 68 % and 95 % expectations.

A further complication arises from the simplified ATLAS likelihood. The authors point out that the experimental likelihood used in the study omits many detailed systematic correlations present in the real analysis. When combined with the boundary‑induced distortions, this simplification amplifies the apparent over‑coverage, making it difficult to disentangle methodological artifacts from genuine physical effects.

The paper concludes that (i) neural‑network surrogates are a powerful tool for enabling exhaustive statistical studies of high‑dimensional BSM models, (ii) physical parameter boundaries can dramatically affect the frequentist and Bayesian coverage properties, and (iii) accurate inference requires both a faithful representation of the experimental likelihood and careful handling of model‑specific constraints. The authors suggest extending the approach to more general SUSY frameworks (e.g., NMSSM, pMSSM) and applying it to actual LHC data to further quantify the interplay between boundary effects, likelihood approximations, and statistical inference.