Statistical coverage for supersymmetric parameter estimation: a case study with direct detection of dark matter

Models of weak-scale supersymmetry offer viable dark matter (DM) candidates. Their parameter spaces are however rather large and complex, such that pinning down the actual parameter values from experimental data can depend strongly on the employed statistical framework and scanning algorithm. In frequentist parameter estimation, a central requirement for properly constructed confidence intervals is that they cover true parameter values, preferably at exactly the stated confidence level when experiments are repeated infinitely many times. Since most widely-used scanning techniques are optimised for Bayesian statistics, one needs to assess their abilities in providing correct confidence intervals in terms of the statistical coverage. Here we investigate this for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only constrained by data from direct searches for dark matter. We construct confidence intervals from one-dimensional profile likelihoods and study the coverage by generating several pseudo-experiments for a few benchmark sets of pseudo-true parameters. We use nested sampling to scan the parameter space and evaluate the coverage for the benchmarks when either flat or logarithmic priors are imposed on gaugino and scalar mass parameters. The sampling algorithm has been used in the configuration usually adopted for exploration of the Bayesian posterior. We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified. We show how most of the variation can be explained as the impact of explicit priors as well as sampling effects, where the latter are indirectly imposed by physicality conditions. For comparison, we also evaluate the coverage for Bayesian credible intervals, and observe significant under-coverage in those cases.

💡 Research Summary

This paper investigates the statistical coverage of frequentist confidence intervals in the context of supersymmetric parameter estimation, focusing on the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only direct dark‑matter detection data are available. In frequentist inference, a confidence interval is required to contain the true parameter value with a probability equal to the nominal confidence level (e.g., 68 % or 95 %) in the limit of infinitely many repetitions of the experiment. In practice, however, the high‑dimensional CMSSM parameter space is usually explored with algorithms that are optimised for Bayesian inference, such as nested sampling (MultiNest). The authors ask whether such tools, when used in their standard configuration, can also deliver reliable frequentist intervals.

To answer this, they adopt two prior choices for the two most important mass parameters, the universal gaugino mass (m_{1/2}) and the universal scalar mass (m_0): (i) a flat prior, which treats all values within the allowed range equally, and (ii) a logarithmic prior, which gives more weight to low‑mass regions. Both priors are combined with the usual physical constraints (e.g., electroweak symmetry breaking, absence of tachyons, relic‑density bounds).

Three benchmark points are selected, each representing a distinct region of the CMSSM parameter space (different neutralino masses and scattering cross‑sections). For each benchmark the authors generate thousands of pseudo‑experiments that mimic the statistical fluctuations expected in a realistic direct‑detection experiment (Poisson‑distributed recoil counts, energy‑spectrum smearing, etc.). Each pseudo‑dataset is then analysed with MultiNest, producing a set of samples from the posterior distribution. From these samples the one‑dimensional profile likelihood for each parameter is constructed, and the standard 68 % and 95 % confidence intervals are extracted by locating the likelihood values that correspond to the appropriate chi‑square thresholds.

Coverage is evaluated by counting how often the true benchmark value lies inside the interval obtained from a pseudo‑experiment, and dividing by the total number of pseudo‑experiments. This procedure is repeated for both prior choices and for both confidence levels. In addition, the authors compute Bayesian credible intervals (68 % and 95 %) from the same posterior samples and assess their coverage for comparison.

The results show a pronounced dependence on the prior. With the logarithmic prior, the 68 % intervals often under‑cover dramatically: in some benchmark cases the empirical coverage drops to 40 % or lower, far below the nominal 68 %. This under‑coverage is traced to the fact that the log prior concentrates sampling in the low‑mass region, but the physical boundary conditions (e.g., the requirement of a neutralino LSP) truncate the viable parameter space, leaving large gaps that the sampler fails to explore adequately. Consequently, the profile likelihood is poorly resolved near the true point, and the derived interval is too narrow.

Conversely, the flat prior tends to produce over‑coverage for the 95 % intervals, with empirical coverages approaching 100 % for some benchmarks. The uniform weighting spreads samples over the whole allowed region, but because the likelihood is sharply peaked in a small sub‑volume, the profile likelihood is noisy elsewhere, leading to overly conservative intervals that are much larger than necessary.

The Bayesian credible intervals fare even worse from a frequentist standpoint. Across all benchmarks, the 68 % credible intervals cover the true parameters in less than 30 % of the pseudo‑experiments, indicating severe under‑coverage. This reflects the intrinsic difference between Bayesian probability statements (which condition on the observed data and the prior) and frequentist coverage (which conditions on repeated sampling).

The authors argue that the observed discrepancies are not a failure of the statistical concepts themselves but rather a mismatch between the sampling algorithm’s design goals (efficient Bayesian posterior exploration) and the requirements of frequentist interval construction (uniform coverage irrespective of prior). The nested‑sampling implementation used (the standard MultiNest settings) implicitly incorporates the prior into the sampling density, and physicality cuts further bias the effective sampling distribution. As a result, the profile likelihood obtained from the sampled points does not faithfully represent the true likelihood surface, especially in regions where the prior suppresses sampling.

To mitigate these issues, the paper suggests several possible remedies: (1) employing sampling strategies that are prior‑independent, such as uniform‑weight Markov Chain Monte Carlo or importance‑sampling schemes that deliberately oversample low‑likelihood regions; (2) re‑weighting the nested‑sampling output to remove prior effects before constructing the profile likelihood; (3) increasing the number of live points and decreasing the sampling tolerance in MultiNest to improve resolution of the likelihood surface, albeit at a substantial computational cost; and (4) cross‑validating frequentist intervals with dedicated coverage studies, as performed here, before drawing scientific conclusions.

In summary, this work demonstrates that, for supersymmetric models constrained only by direct‑detection data, the standard Bayesian‑oriented nested‑sampling pipelines can produce both under‑ and over‑coverage in frequentist confidence intervals, with the magnitude of the effect strongly dependent on the choice of prior. Bayesian credible intervals, while useful for posterior inference, do not guarantee frequentist coverage and can be dramatically misleading if interpreted as confidence intervals. The study underscores the necessity of explicit coverage validation when applying sophisticated sampling tools to frequentist problems, and it highlights the importance of developing or adapting algorithms that can simultaneously satisfy Bayesian efficiency and frequentist reliability in high‑dimensional beyond‑Standard‑Model analyses.