Conditional information and definition of neighbor in categorical random fields

We show that the definition of neighbor in Markov random fields as defined by Besag (1974) when the joint distribution of the sites is not positive is not well-defined. In a random field with finite number of sites we study the conditions under which giving the value at extra sites will change the belief of an agent about one site. Also the conditions under which the information from some sites is equivalent to giving the value at all other sites is studied. These concepts provide an alternative to the concept of neighbor for general case where the positivity condition of the joint does not hold.

💡 Research Summary

The paper revisits the notion of “neighbor” in Markov random fields (MRFs) and demonstrates that the classical definition, as introduced by Besag (1974) and later used by Cressie and Subash (1992), is well‑posed only when the joint distribution of all sites is strictly positive. When the joint distribution contains zero‑probability configurations, the functional form of the conditional probability P(X_i | X_{‑i}) is not uniquely determined, and consequently the dependence on a particular site j may be ambiguous. The authors illustrate this ambiguity with a simple construction (Example 2.1) where the conditional probability of X₃ given X₂ and X₁ coincides with the conditional probability given X₂ alone, yet it also coincides with the conditional probability given X₁ alone. This shows that the traditional neighbor definition, which declares site j a neighbor of i if the conditional probability of i depends on the value of j, fails to be well‑defined in the presence of zero‑probability events.

The paper first proves that under the positivity assumption the neighbor concept is indeed unique: Lemma 2.1 shows that if every configuration has positive probability, any two functional representations of the conditional distribution must agree everywhere, forcing the set of influencing sites to be the same.

To handle the general case where positivity does not hold, the authors introduce several information‑theoretic concepts. They define a “uninformative set” for events A and B as any event C such that either P(A | B, C)=P(A | B) or P(B∧C)=0. Lemma 3.1 establishes that the collection of uninformative sets is closed under countable disjoint unions, but a counterexample (Example 3.1) shows that it is not closed under intersections, highlighting that lack of influence is not a simple independence property.

The core of the paper lies in the definitions of “sufficient information set” and “minimal information set.” For a site i and a collection of other sites I, a subset J⊂I is sufficient if the conditional distribution of i given I coincides with that given J on the domain where the conditional is defined. A minimal information set is one that cannot be reduced further without altering the conditional distribution. Proposition 4.1 and Lemma 4.1 explore the relationships among these sets, proving that sufficiency is preserved when moving to larger supersets and that a sufficient set remains sufficient when considered within any intermediate set.

Combining these notions, the authors define an “efficiently sufficient set” for site i as a set that is both minimal and sufficient with respect to the complement of i (i.e., the set of all other sites). If such a set is unique for a given site, it is taken as the neighbor of that site. This generalized neighbor definition coincides with Besag’s original definition when the positivity condition holds, but it remains meaningful when the joint distribution contains zeros.

Example 4.1 demonstrates the new definition in a binary pair (X, Y) where P(X=1, Y=0)=0, violating positivity. Nevertheless, Y is an efficiently sufficient set for X and vice versa, so each variable is considered the neighbor of the other under the new framework.

Overall, the paper proceeds as follows:

1. Identify the limitation of the classical neighbor definition in non‑positive joint distributions.
2. Show that positivity guarantees uniqueness of the neighbor set.
3. Introduce uninformative events and prove basic closure properties.
4. Define sufficient and minimal information sets, and study their algebraic properties.
5. Propose the efficiently sufficient set as a generalized neighbor, and validate it with concrete examples and counterexamples.

The contribution is a rigorous, information‑theoretic reformulation of neighborhood relationships in discrete random fields that does not rely on the positivity of the joint distribution. This framework is particularly relevant for applications in spatial statistics, image analysis, ecological modeling, and any domain where categorical random fields may contain impossible configurations. By providing a well‑defined notion of adjacency even in the presence of zero‑probability events, the paper expands the theoretical toolbox for modeling and inference in complex stochastic systems.