A Bayesian Assessment of P-Values for Significance Estimation of Power Spectra and an Alternative Procedure, with Application to Solar Neutrino Data

The usual procedure for estimating the significance of a peak in a power spectrum is to calculate the probability of obtaining that value or a larger value by chance (known as the “p-value”), on the assumption that the time series contains only noise - typically that the measurements are derived from random samplings of a Gaussian distribution. We really need to know the probability that the time series is - or is not - compatible with the null hypothesis that the measurements are derived from noise. This probability can be calculated by Bayesian analysis, but this requires one to specify and evaluate a second hypothesis, that the time series does contain a contribution other than noise. We approach the problem of identifying this function in two ways. We first propose three simple conditions that it seems reasonable to impose on this function, and show that these conditions may be satisfied by a simple function with one free parameter. We then define two different ways of combining information derived from two independent power estimates. We find that this consistency condition may be satisfied, to good approximation, by a special case of the previously proposed likelihood function. We find that the resulting significance estimates are considerably more conservative than those usually associated with the p-values. As two examples, we apply the new procedure to two recent analyses of solar neutrino data: (a) power spectrum analysis of Super-Kamiokande data, and (b) the combined analysis of radiochemical neutrino data and irradiance data.

💡 Research Summary

The paper critiques the conventional use of p‑values for assessing the significance of peaks in power‑spectral analyses, pointing out that a p‑value merely quantifies the probability of obtaining a statistic at least as extreme as the observed one under the null hypothesis (pure noise), but it does not directly answer the question of how probable it is that the data are compatible with that null hypothesis. To address this shortcoming, the authors adopt a Bayesian framework that explicitly compares the posterior probabilities of two competing hypotheses: H0 (the measurements are drawn solely from noise, typically Gaussian) and H1 (the measurements contain an additional deterministic or quasi‑deterministic component).

A central difficulty in Bayesian model comparison is the specification of a prior probability density for the alternative hypothesis. The authors propose three reasonable constraints for this prior: (1) it must be defined on the non‑negative power axis and decrease monotonically with increasing power; (2) it must be normalizable (integrates to unity); and (3) it should resemble the exponential distribution that describes the null‑hypothesis power spectrum, but with a tunable parameter that allows it to be more or less “optimistic” about the presence of a signal. They show that a simple one‑parameter family,

  f(S) = α exp(−α S),

satisfies all three constraints. When α = 1 the prior collapses to the null‑hypothesis distribution; values of α < 1 give a heavier tail, reflecting a higher prior belief that large powers may arise from a genuine signal. This functional form is attractive because it introduces only a single free parameter, making the method easy to implement and interpret.

Having defined a prior for H1, the authors turn to the problem of combining information from two independent power estimates, P1 and P2, which might arise from different data sets, different time windows, or different analysis techniques. They present two combination rules. The first follows directly from Bayes’ theorem: the joint posterior odds are the product of the individual posterior odds, assuming independence. The second, more practical rule, adds the log‑likelihoods (or equivalently, multiplies the likelihood ratios) to obtain a composite log‑Bayes factor. Both approaches yield a combined Bayes factor that quantifies how much more the data support H1 over H0. The authors demonstrate that, under the special case α = 0.5, the composite likelihood function derived from the log‑addition rule coincides closely with the likelihood function previously proposed by other workers for combining power spectra, thereby providing a theoretical justification for that ad‑hoc method.

The methodology is applied to two recent solar‑neutrino investigations. In the first case, a power‑spectrum analysis of Super‑Kamiokande (SK) solar‑neutrino flux measurements identified a peak at a frequency that, under the standard χ² exponential noise model, yielded a p‑value of roughly 0.01, suggesting a statistically significant periodicity. Using the Bayesian approach with α = 0.5, the posterior probability that the SK data contain a genuine periodic component at that frequency drops to about 0.2, indicating a far less compelling case. In the second case, the authors combine radiochemical neutrino data (from the Homestake and GALLEX/GNO experiments) with solar‑irradiance measurements, each providing an independent power estimate for the same frequency band. The conventional analysis, again using p‑values, would claim a combined significance at the 95 % confidence level. By contrast, the Bayesian combination yields a Bayes factor of only ~1.5, corresponding to a combined posterior probability of roughly 0.7 for the presence of a common signal—substantially more conservative than the p‑value result.

Overall, the paper argues that Bayesian significance estimation provides a more honest and conservative assessment of periodicities in noisy astrophysical time series. The single‑parameter prior for H1 is both mathematically tractable and physically plausible, while the log‑likelihood combination rule offers a straightforward way to fuse independent power estimates. The authors suggest several avenues for future work: (i) empirical calibration of the α parameter using simulated data or hierarchical Bayesian models; (ii) extension of the framework to handle non‑Gaussian noise, time‑varying signals, or multiple simultaneous frequencies; and (iii) application to other domains where power‑spectral methods are standard, such as helioseismology, exoplanet transit timing, and gravitational‑wave background searches. By moving beyond p‑values, the Bayesian approach promises more reliable inference about the existence of subtle, physically meaningful periodicities in complex data sets.