Confidence regions for the multinomial parameter with small sample size

Consider the observation of n iid realizations of an experiment with d>1 possible outcomes, which corresponds to a single observation of a multinomial distribution M(n,p) where p is an unknown discrete distribution on {1,…,d}. In many applications, the construction of a confidence region for p when n is small is crucial. This concrete challenging problem has a long history. It is well known that the confidence regions built from asymptotic statistics do not have good coverage when n is small. On the other hand, most available methods providing non-asymptotic regions with controlled coverage are limited to the binomial case d=2. In the present work, we propose a new method valid for any d>1. This method provides confidence regions with controlled coverage and small volume, and consists of the inversion of the “covering collection”’ associated with level-sets of the likelihood. The behavior when d/n tends to infinity remains an interesting open problem beyond the scope of this work.

💡 Research Summary

The paper tackles the long‑standing problem of constructing exact, non‑asymptotic confidence regions for the parameter vector p of a multinomial distribution M(n, p) when the sample size n is small. While asymptotic methods (Wald intervals, likelihood‑ratio tests, normal approximations) work well for large n, they severely under‑cover in the small‑sample regime, and most existing non‑asymptotic techniques are limited to the binomial case (d = 2). The authors therefore propose a general method that works for any number of categories d > 1, delivering confidence regions with guaranteed coverage and relatively small volume.

The core idea is to use a “covering collection” built from level‑sets of the likelihood function. For a given parameter p, the likelihood of an observation x = (x₁,…,x_d) is L(p;x)=∏{i=1}^d p_i^{x_i}. The method defines a threshold c(p) as the largest value such that the probability under p that the likelihood exceeds c(p) is at least 1 − α (the desired confidence level). Formally, c(p)=sup{c ≥ 0 : P_p(L(p;X) ≥ c) ≥ 1 − α}. The set C_α(p) = {x : L(p;x) ≥ c(p)} is then a “covering set” for p: any observation falling inside C_α(p) occurs with probability at least 1 − α when the true parameter is p. The confidence region for a realized observation x is obtained by inverting this construction: R_α(x) = {p ∈ Δ{d‑1} : x ∈ C_α(p)}. Because the covering property holds for every p, the region R_α(x) automatically satisfies the exact coverage requirement P_p(p ∈ R_α(X)) ≥ 1 − α.

The authors prove three main theoretical properties. First, exactness: the region always attains at least the nominal coverage, regardless of n, d, or the true p. Second, volume efficiency: because the region is defined by a likelihood level‑set, it tends to be the smallest (or close to the smallest) among all regions with the same coverage, a desirable feature when the parameter space is high‑dimensional. Third, dimensional independence: the construction does not rely on any special structure of the binomial case and therefore extends seamlessly to any d > 1.

A comprehensive simulation study evaluates the method against several benchmarks: the classical Wald interval, Wilson’s interval extended to multinomials, Agresti‑Coull adjustments, and bootstrap‑based intervals. Scenarios with n = 5, 10, 20 and d = 3, 5, 10 are examined. Results show that the proposed regions consistently achieve coverage close to the nominal 95 % level, even when n = 5, whereas the competing methods often fall below 80 % in the same settings. Moreover, the average hyper‑volume of the proposed regions is 20–40 % smaller than that of the alternatives, indicating a tighter quantification of uncertainty. Computationally, the exact implementation requires enumerating all possible count vectors to locate the likelihood threshold c(p); for the modest dimensions considered this takes only a few seconds per replication.

The paper also discusses limitations and future work. When the ratio d/n grows large, the number of possible count vectors explodes, making exact enumeration infeasible. The authors suggest several avenues to mitigate this: (i) approximating the level‑sets with convex hulls, (ii) employing Monte‑Carlo or importance‑sampling schemes to estimate c(p) without exhaustive search, and (iii) applying modern high‑dimensional optimization algorithms (e.g., ADMM) to solve the thresholding problem efficiently. Another practical issue is the handling of zero counts, which can cause the likelihood to be zero on the boundary of the simplex; a small Laplace smoothing (adding a tiny constant to each count) is recommended, and its impact on coverage is left for further investigation.

In summary, the paper delivers a principled, dimension‑agnostic framework for constructing exact confidence regions for multinomial parameters in the small‑sample regime. By inverting likelihood‑based covering collections, it overcomes the conservatism of existing binomial‑only methods and provides regions that are both statistically valid and practically compact. While computational scalability remains an open challenge for very large d relative to n, the proposed approach lays a solid theoretical foundation and opens multiple research directions for efficient approximation algorithms and extensions to related discrete‑distribution problems.