Application of quasi-optimal weights to searches of anomalies. Statistical criteria for step-like anomalies in cumulative spectra

The statistical method of quasi-optimal weights can be used to derive criteria for searches of anomalies. As an example we derive a convenient statistical criterion for step-like anomalies in cumulative spectra such as measured in the Troitsk-nu-mass, Mainz and KATRIN experiments. It is almost as powerful as the locally most powerful one near the null hypothesis and appreciably excels the conventional chi^2 and Kolmogorov-Smirnov tests. It is also compared with an ad hoc criterion of {\guillemotleft}pairwise correlations of neighbours{\guillemotright}; the latter is seen to be less powerful if more sensitive to more general anomalies. As a realistic example, the criteria are applied to the Troitsk-nu-mass data.

💡 Research Summary

The paper addresses the problem of detecting step‑like anomalies in cumulative spectra, a situation that arises in precision β‑decay experiments such as Troitsk‑ν‑mass, Mainz, and KATRIN. In these experiments the measured quantity is a cumulative count of events as a function of a control variable (retarding potential or energy). An unexpected excess of electrons near the endpoint manifests as a step in the cumulative spectrum, characterized by a height Δ and a position E_st. Traditional statistical tools—χ² tests and Kolmogorov‑Smirnov (KS) tests—are not tailored to this specific form of deviation; χ² assumes Gaussian errors and KS is designed for continuous one‑dimensional samples, making both relatively insensitive to a localized step, especially when the underlying data follow Poisson statistics.

The authors adopt the quasi‑optimal weight (QOW) framework originally proposed by Tkachov to construct a test that is locally most powerful (LMP) for detecting a step of unknown height but known position. Starting from the Poisson likelihood for each measurement, they derive optimal weights ω_i = ∂ln L/∂Δ, which change sign at the step position. Setting the weighted sum of observed counts to zero yields an estimator for Δ that maximizes the likelihood ratio near the null hypothesis (Δ = 0). This LMP test is theoretically optimal but suffers from two practical drawbacks: (1) it requires an explicit solution of a set of nonlinear equations, and (2) it is highly sensitive to the assumed step position, which in real data can drift over time.

To mitigate these issues, the authors propose a “quasi‑optimal” test that simplifies the weight function while preserving its essential sign‑change property. They replace the exact optimal weights with a piecewise‑linear function w_i that is proportional to (i − m)/M, where m indexes the assumed step position. The resulting test statistic S_qopt = Σ w_i ξ_i (with ξ_i = (N_i − μ_i)/√μ_i) is easy to compute, retains most of the power of the LMP test, and is considerably less affected by mis‑specification of the step location.

As a complementary approach, they introduce a “pairwise neighbours correlation” test. This statistic sums the products of adjacent normalized residuals, S_pair = Σ ξ_i ξ_{i+1}. A consistent deviation of neighbouring points on the same side of the fitted curve yields a positive contribution, making the test sensitive not only to steps but also to more general localized anomalies.

The performance of all five tests (LMP, quasi‑optimal, pairwise, χ², and a modified KS test) is evaluated using Monte‑Carlo simulations that mimic the Troitsk data set (44 energy points with Poisson‑distributed counts). Power functions—probability of rejecting the null hypothesis when a step of height Δ is present—are computed for a range of Δ values. Results show that the LMP test has the highest power, the quasi‑optimal test follows closely, and the pairwise test ranks third. The conventional χ² and modified KS tests are the least powerful across the examined Δ range.

The authors also study robustness to step‑position errors by shifting the true step location by 12 eV and 25 eV relative to the assumed position. The LMP test’s power deteriorates sharply with even modest mis‑alignment, whereas the quasi‑optimal test maintains a relatively stable power curve. For large shifts (≈ 25 eV), the χ² test regains some discriminating ability, while the KS test remains the weakest.

Finally, the methods are applied to real Troitsk‑ν‑mass data from eleven independent runs. For each run the quasi‑optimal and pairwise statistics are computed, and corresponding p‑values (α) are obtained from Monte‑Carlo‑derived null distributions. The quasi‑optimal p‑values range from ~0.2 to 0.6, while the pairwise p‑values are more widely spread (0.1–0.99). None exceed the conventional 0.95 significance threshold, indicating no decisive evidence for a step‑like anomaly in this data set, though the collection of multiple independent tests suggests that a more sophisticated combined analysis could be worthwhile.

In conclusion, the quasi‑optimal test offers a practical compromise: it approaches the theoretical optimality of the LMP test while being computationally simple and robust against uncertainties in step position. The pairwise neighbours correlation test, though less powerful for pure steps, provides a versatile tool for detecting broader classes of localized deviations. Traditional χ² and KS tests are only competitive when the step is substantially displaced from the assumed location. The suite of methods presented can be readily adapted to any experiment involving cumulative spectra, enhancing the ability to identify subtle systematic effects or potential signals of new physics.