Conditional information and definition of neighbor in categorical random fields

This paper studies the conditional probabilities and the definition of neighbor in categorical random fields. These can be used to describe spatial processes e.g. in plant ecology. We start by the common definition of neighbor in Markov random fields and show that the definition is not well-defined when the joint distribution is not positive. Then we provide a framework to study the conditional probabilities given various amount of "information". For example, the conditional probability of one site given some others. Since the usual definition of neighbor is not well-defined when the "positivity" condition of the joint distribution does not hold, we introduce some new concepts of "uninformative set", "sufficient information set" and "minimal information set".

Suppose we have a finite random field consisting of n sites. The belief of an agent about one site can be summarized by a probability distribution and can be changed to a conditional distribution by relieving new information which can be the value at some other sites. We study when the new information changes the agent’s belief and what is “sufficient” information for the agent in the sense that giving the information would be equivalent to giving the value of all other sites. We answer some interesting questions along the way. For example suppose agent 1 has less information than agent 2 regrading an event A and a new information is released. Now, suppose that agent 1 does not change his belief about A. One might conjecture that since agent 2 has more information, he as well will not change his belief after receiving the new information. We show this conjecture is wrong by counterexamples.

2 Neighbor in categorical random fields Suppose (Ω, Σ, P ) is a probability space and {X i } n i=1 is a stochastic process. Each X i takes values in M i , |M i | = m i < ∞, and P (x i ) > 0, ∀x i ∈ M i . We use the shorthand notation: Besag (1974) and Cressie and Subash (1992), defined the neighbor as follows:

Note that in the above definition, we need to make sure that the conditional probability is defined. The above conditional probability is defined on

We show in the following example this definition is not well-defined in general since the functional form is not unique.

Example 2.1 Let U 1 , • • • , U 4 denote a random sample from the uniform distribution that take only values 0 and 1 each with probability 1/2. Define:

where [ ] denotes the integer part of a real number. By the last equality in above, X 3 if we know the value of X 2 , the value of X 1 will not give us extra information. Hence, P (x 3 |x 2 , x 1 ) = P (x 3 |x 2 ).

But since [X 2 ] = [X 1 ], we also have

wherever the conditional probability is defined. This shows the definition of neighbor is not well-defined in general.

Next we show that the positivity of the joint distribution implies that the definition of neighbor is well-defined. By positivity of the joint distribution, we mean

If the joint distribution is strictly positive then the concept of neighbor is well-defined for this field.

For some functions f, g. By positivity condition, the conditional probability is defined everywhere. Hence,

Suppose h ∈ H -J . Then x h does not appear on the left hand side so g is not dependent on x h . We conclude H -J = ∅. Similarly, J -H = ∅.

In the following, we consider the general case (when the positivity condition does not hold) and define some useful concepts which are well-defined even though the concept of neighbor is not as well-defined as defined by Besag (1974). We start by some useful definitions and lemmas regarding conditional probabilities. Consider the conditional probability P (A|B) where A, B are two events and P (B) > 0. Also consider a third event C. It is interesting to study when C changes (or does not change) our beliefs about probability of A. Formally, we have the following definition.

Let UN(A|B) to be the set of all events C such that P (B, C) = 0 or P (A|B, C) = P (A|B).

One might also conjecture that UN(A|B) is closed under intersection. We show by some counterexamples, this is not true.

Example 3.1 Ω = {1, 2, 3, 4, 5, 6, 7, 8}, A = {1, 2, 3, 4}, B = Ω, C 1 = {2, 4, 6, 8}, C 2 = {1, 3, 5, 8} and consider a uniform probability distribution on Ω.

Then P (A|B) = P (A) = 1/2, P (A|B,

Example 3.2 Consider the joint distribution for (X, Y, Z) given in Table 1, where every row has the same probability of 1/4. Suppose that two agents want to predict the value of X. The first person does not have any information and the second one knows that Z = 0. Now, assume that we provide extra information to both agents. The extra information is the value of Y . For the first agent at the beginning (before the information about Y was given): P (X = 0) = P (X = 1) = 1/2. After he knows the value of Y : P (X = 1|Y = 0) = P (X = 1|Y = 1) = 1/2. Hence, the extra information does not change the belief of the first agent about X. One might conjecture that since the second agent has more

…(Full text truncated)…

This content is AI-processed based on ArXiv data.