On Probability Distributions for Trees: Representations, Inference and Learning

We study probability distributions over free algebras of trees. Probability distributions can be seen as particular (formal power) tree series [Berstel et al 82, Esik et al 03], i.e. mappings from trees to a semiring K . A widely studied class of tree series is the class of rational (or recognizable) tree series which can be defined either in an algebraic way or by means of multiplicity tree automata. We argue that the algebraic representation is very convenient to model probability distributions over a free algebra of trees. First, as in the string case, the algebraic representation allows to design learning algorithms for the whole class of probability distributions defined by rational tree series. Note that learning algorithms for rational tree series correspond to learning algorithms for weighted tree automata where both the structure and the weights are learned. Second, the algebraic representation can be easily extended to deal with unranked trees (like XML trees where a symbol may have an unbounded number of children). Both properties are particularly relevant for applications: nondeterministic automata are required for the inference problem to be relevant (recall that Hidden Markov Models are equivalent to nondeterministic string automata); nowadays applications for Web Information Extraction, Web Services and document processing consider unranked trees.

💡 Research Summary

The paper investigates probability distributions defined over the free algebra of trees, interpreting such distributions as particular formal power tree series—functions that map trees to elements of a semiring K. A well‑studied subclass of tree series is the class of rational (or recognizable) tree series, which can be characterized either algebraically or via multiplicity tree automata. The authors argue that the algebraic representation is especially convenient for modeling probability distributions on trees.

First, analogous to the string case, the algebraic view enables the design of learning algorithms that cover the entire class of rational tree series. By formulating the series as solutions to systems of linear equations over K, one can apply a generalized Expectation–Maximization (EM) procedure that simultaneously learns the automaton’s structure (states and transitions) and the associated weights (probabilities). This contrasts with earlier work that typically assumes a fixed automaton topology and only adjusts weights. Consequently, the learning framework directly corresponds to learning weighted tree automata where both topology and parameters are inferred from data.

Second, the algebraic formulation naturally extends to unranked trees—structures where a node may have an arbitrary number of children, such as XML or HTML DOM trees. The authors treat a variable‑arity node as an operator that concatenates an arbitrary list of sub‑trees, and they encode this operator within a finite set of algebraic rules. This yields a rational tree series definition for unranked trees without resorting to ad‑hoc encodings or infinite state spaces. The resulting models retain the expressive power of rational series while supporting efficient inference.

From an inference perspective, nondeterministic tree automata are essential. Nondeterminism allows multiple derivations for the same input tree, mirroring the hidden state dynamics of Hidden Markov Models (HMMs) in the string domain. The paper shows that probabilistic parsing with nondeterministic weighted tree automata yields the most likely tree (maximum a posteriori) and can compute marginal probabilities efficiently, making the approach suitable for tasks where uncertainty over tree structure must be quantified.

The authors discuss several practical applications. In Web Information Extraction, for example, XML or HTML documents can be modeled as unranked trees; a learned probabilistic automaton can capture common structural patterns while tolerating variations across pages. In Web Services composition and document processing, the ability to learn both structure and probabilities enables systems to adapt to evolving schemas and to perform robust pattern matching under noisy or incomplete data.

Overall, the paper establishes that the algebraic representation of rational tree series provides a unified, theoretically sound, and practically viable framework for (1) defining expressive probabilistic models over both ranked and unranked trees, (2) learning these models from data with guarantees comparable to those of string‑based probabilistic models, and (3) performing efficient inference using nondeterministic weighted tree automata. This bridges a gap between formal language theory and real‑world tree‑structured data processing, opening avenues for further research in probabilistic grammar induction, tree‑structured deep learning, and scalable XML/JSON analytics.