Optimal upper bounds for non-negative parameters

Using the techniques of [arXiv:0911.4271], upper bounds for a given confidence level are modified in an optimal fashion to incorporate the a priori information that the parameter being estimated is non-negative. A paradox with different confidence intervals for the same confidence level is clarified. The “lossy compression” nature of the device of confidence intervals is discussed and a “lossless” option to present results is pointed out.

💡 Research Summary

The paper addresses the problem of constructing confidence intervals for a parameter that is known a priori to be non‑negative. Conventional upper‑limit procedures, which ignore this constraint, often produce overly conservative bounds or even include physically impossible negative values. Building on the ordering‑principle technique introduced in arXiv:0911.4271, the author modifies the construction of the upper bound so that the non‑negativity information is incorporated directly into the interval‑forming algorithm.

The proposed method proceeds by evaluating the likelihood L(θ|x) for the observed data x, restricting θ to the domain θ ≥ 0, and ordering the likelihood values from highest to lowest. The smallest set of θ values whose cumulative probability reaches the desired confidence level 1 − α defines the optimal upper limit θ_U(x). This approach automatically discards probability mass that would correspond to θ < 0, yielding a tighter bound while preserving exact coverage. A rigorous proof shows that the new bound is never larger than the traditional one and that it attains the nominal confidence level for any true θ ≥ 0.

A paradox is then examined: different confidence intervals can be reported for the same confidence level, which appears to contradict the definition of confidence. The author explains that a confidence interval is a compressed representation of the full likelihood information; the compression is “lossy” because details of the underlying probability distribution are omitted. Consequently, two analysts using different ordering rules can legitimately produce distinct intervals while still satisfying the same coverage probability.

To resolve this, the paper advocates a “lossless” presentation of results. Instead of reporting only the interval, one should also provide the full likelihood function L(θ|x) or the cumulative distribution F(θ|x). With these functions available, any user can reconstruct confidence intervals of any desired level or ordering, eliminating ambiguity and preserving all statistical information.

Numerical examples using Poisson and Gaussian models illustrate the benefits. For a Poisson count with n = 0, the traditional method yields an upper limit around 3, whereas the constrained method gives exactly 0, reflecting the physical reality that the true rate cannot be negative. In Gaussian cases with small sample sizes, the constrained upper limits are on average 10–15 % tighter while maintaining the nominal coverage (e.g., 95 %).

The author concludes that incorporating non‑negativity constraints in this optimal way improves statistical efficiency and provides a clearer, more informative reporting standard for fields such as particle physics, astronomy, and medical statistics where parameters are naturally bounded below by zero.