We describe a Groebner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generalized permutohedra and describe a conditional probability simplex.
Deep Dive into Relations among conditional probabilities.
We describe a Groebner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generalized permutohedra and describe a conditional probability simplex.
arXiv:0808.1149v1 [math.PR] 8 Aug 2008
Relations among conditional probabilities
Jason Morton
October 28, 2018
Abstract
We describe a Gr¨obner basis of relations among conditional probabilities in a
discrete probability space, with any set of conditioned-upon events. They may be
specialized to the partially-observed random variable case, the purely conditional
case, and other special cases.
We also investigate the connection to generalized
permutohedra and describe a “conditional probability simplex.”
1
Relations among conditional probabilities
In 1974, Julian Besag [4] discussed the “unobvious and highly restrictive consistency
conditions” among conditional probabilities.
In this paper we give an answer in the
discrete case to the question What conditions must a set of conditional probabilities satisfy
in order to be compatible with some joint distribution?
Let Ω= {1, . . . , m} be a finite set of singleton events, and let p = (p1, . . . , pm) be
a probability distribution on them. Let E be a set of observable events which will be
conditioned on, each a set of at least 2 singleton events.
Then for events I ⊂J, J
in E , we can assign conditional probabilities for the chance of I given J, denoted pI|J.
Settling Besag’s question then becomes a matter of determining the relations that must
hold among the quantities pI|J. For example, Besag gives the relation (see also [3]),
P(x)
P(y) =
n
Y
i=1
P(xi|x1, . . . , xi−1, yi+1, . . . , yn)
P(yi|x1, . . . , xi−1, yi+1, . . . , yn).
(1)
Since there are in general infinitely many such relations, we would like to organize them
into an ideal and provide a nice basis for that ideal. A quick review of language of ideals,
varieties, and Gr¨obner bases appears in Geiger et al. [11, p. 1471] and more detail in Cox
et al. [7]. In Theorem 3.2, we generalize relations such as (1) and Bayes’ rule to give a
universal Gr¨obner basis of this ideal, a type of basis with useful algorithmic properties.
The second result generalized in this paper is due to Mat´uˇs [15]. This states that the
space of conditional probability distributions (pi|ij) conditioned on events of size two maps
homeomorphically onto the permutohedron. In Theorem 4.3, we generalize this result to
1
arbitrary sets E of conditioned-upon events. The resulting image is a generalized permu-
tohedron [20, 24]. This is a polytope which provides a canonical, conditional-probability
analog to the probability simplex under the correspondance provided by toric geometry
[23] and the theory of exponential families.
Work on the subject of relations among conditional probabilities has primarily focused
on the case where the events in E correspond to observing the states of a subset of n ran-
dom variables. Arnold et. al. [2] develop the theory for both discrete and continuous
random variables, particularly in the case of two random variables, and cast the com-
patibility of two families of conditional distributions as a solutions to a system of linear
equations. Slavkovic and Sullivant [22] consider the case of compatible full conditionals,
and compute related unimodular ideals.
This paper is organized as follows. In Section 2, we introduce some necessary defini-
tions. In Section 3, we give compatibility conditions in the general case of m events in a
discrete probability space, with any set E of conditioned-upon events. These conditions
come in the form of a universal Gr¨obner basis, which makes them particularly useful
for computations: as a result, they may be specialized to the partially observed random
variable case, the purely conditional case, and other special cases simply by changing E .
In [14, 17], we have seen that permutohedra and generalized permutohedra [20] play a
central role in the geometry of conditional independence; the same is true of conditional
probability. The geometric results of Mat´uˇs [15] map the space of conditional probability
distributions (Definition 2.1) for all possible conditioned events E = {I ⊂[m] : |I| ≥2}
onto the permutohedron Pm−1. See Figure 1 for a diagram of the 3-dimensional permu-
tohedron. In Section 4, we will discuss how to extend this result to general E , in which
case we obtain generalized permutohedra as the image. This will be accomplished using a
3214
•
2314
•
3241•
2341
◦
3124•
2134•
3421•
2431
◦
1324•
1234•
3142
•
2143
•
3412•
2413◦
4321◦
4231
◦
1342•
1243
•
4312•
4213◦
1432•
1423
•
4132•
4123
•
L
L
L
L
L
))))))))
)))))))
PPPPP
O
O
O
O
O
++++++++
::::::::::::
999999999999vvvvvvvvvvvv
xxxxxxxxxxxx
x
x
x
x
x
x
x
x
x
x
x
x
((((((((
999999999999
P
P
P
P
P
P
x
x
x
x
x
x
x
x
x
x
x
x
777777777777
Figure 1: The permutohedron P4.
version of the moment map of toric geometry (Theorem 7.1). In Section 5, we discuss how
to specialize our results to the case of n partially observed random variables, including as
2
an example how to recover the relation (1). Finally, in Section 6 we use this specialization
to explain the relationship of Bayes
…(Full text truncated)…
This content is AI-processed based on ArXiv data.