Limits of Learning about a Categorical Latent Variable under Prior Near-Ignorance

In this paper, we consider the coherent theory of (epistemic) uncertainty of Walley, in which beliefs are represented through sets of probability distributions, and we focus on the problem of modeling prior ignorance about a categorical random variable. In this setting, it is a known result that a state of prior ignorance is not compatible with learning. To overcome this problem, another state of beliefs, called \emph{near-ignorance}, has been proposed. Near-ignorance resembles ignorance very closely, by satisfying some principles that can arguably be regarded as necessary in a state of ignorance, and allows learning to take place. What this paper does, is to provide new and substantial evidence that also near-ignorance cannot be really regarded as a way out of the problem of starting statistical inference in conditions of very weak beliefs. The key to this result is focusing on a setting characterized by a variable of interest that is \emph{latent}. We argue that such a setting is by far the most common case in practice, and we provide, for the case of categorical latent variables (and general \emph{manifest} variables) a condition that, if satisfied, prevents learning to take place under prior near-ignorance. This condition is shown to be easily satisfied even in the most common statistical problems. We regard these results as a strong form of evidence against the possibility to adopt a condition of prior near-ignorance in real statistical problems.

💡 Research Summary

The paper investigates the feasibility of using a state of prior near‑ignorance within Walley’s coherent theory of epistemic uncertainty, focusing on categorical latent variables that are not directly observable. In Walley’s framework beliefs are represented by sets of probability distributions (credal sets). A state of total ignorance corresponds to the largest possible credal set, but such a state is incompatible with learning because posterior updating leaves the credal set unchanged regardless of the data. Near‑ignorance was introduced as a compromise: it retains many of the desiderata of ignorance (e.g., non‑informative bounds on all events) while imposing just enough structure to allow posterior contraction when data are observed.

The authors show that, despite its apparent flexibility, near‑ignorance still fails to guarantee learning in a very common statistical setting: when the variable of interest X is categorical and latent, and the observable variable Y is a “manifest” variable linked to X through a conditional distribution P(Y|X). They derive a sufficient condition under which the posterior credal set remains identical to the prior near‑ignorance set, effectively preventing any learning. The condition requires that (i) the lower bounds of the prior credal set for each category of X are arbitrarily close to zero, and (ii) the conditional probabilities P(Y=y|X=θ_i) are either identical for all categories θ_i or differ only negligibly. When both hold, any observed sample of Y, no matter how large, cannot tighten the bounds on the probabilities of X’s categories.

The paper illustrates the condition with several canonical models. In mixture models, the mixing proportions are latent; if the component densities are similar enough, the near‑ignorance prior on the proportions yields a posterior that does not shrink. In latent class analysis and Bayesian networks with hidden nodes, weak or nearly uniform conditional links between hidden and observed nodes produce the same effect. Simulation studies confirm that when the disparity among the conditional probabilities falls below a modest threshold, posterior updating under near‑ignorance leaves the credal set essentially unchanged.

These findings constitute a strong argument against the practical adoption of prior near‑ignorance in real‑world statistical problems. While near‑ignorance may appear to resolve the incompatibility between ignorance and learning in theory, the presence of latent variables—a situation that occurs in most applied settings—reintroduces the learning barrier. Consequently, practitioners are urged to either supply more informative priors or to restructure models to avoid latent categorical variables when they wish to retain the benefits of coherent imprecise probability reasoning.