Probability Bracket Notation: Markov Sequence Projector of Visible and Hidden Markov Models in Dynamic Bayesian Networks

With the symbolic framework of Probability Bracket Notation (PBN), the Markov Sequence Projector (MSP) is introduced to expand the evolution formula of Homogeneous Markov Chains (HMCs). The well-known weather example, a Visible Markov Model (VMM), illustrates that the full joint probability of a VMM corresponds to a specifically projected Markov state sequence in the expanded evolution formula. In a Hidden Markov Model (HMM), the probability basis (P-basis) of the hidden Markov state sequence and the P-basis of the observation sequence exist in the sequential event space. The full joint probability of an HMM is the product of the (unknown) projected hidden sequence of Markov states and their transformations into the observation P-bases. The Viterbi algorithm is applied to the famous Weather-Stone HMM example to determine the most likely weather-state sequence given the observed stone-state sequence. Our results are verified using the Elvira software package. Using the PBN, we unify the evolution formulas for Markov models like VMMs, HMMs, and factorial HMMs (with discrete time). We briefly investigated the extended HMM, addressing the feedback issue, and the continuous-time VMM and HMM (with discrete or continuous states). All these models are subclasses of Dynamic Bayesian Networks (DBNs) essential for Machine Learning (ML) and Artificial Intelligence (AI).

💡 Research Summary

The paper introduces a novel symbolic framework called Probability Bracket Notation (PBN), which treats events as vectors in a Hilbert‑like space and expresses probabilities as inner products ⟨·|·⟩, analogous to the bra‑ket notation of quantum mechanics. Within this framework the authors define the Markov Sequence Projector (MSP), an operator that projects a state vector through successive applications of the transition matrix. By rewriting the standard homogeneous Markov chain (HMC) evolution pₙ = p₀·Aⁿ as ⟨π|Aⁿ|⟩, the MSP makes the temporal flow of probabilities explicit as a chain of projection operations.

The first application is to Visible Markov Models (VMMs), where observations coincide with the underlying states. The joint probability of a state sequence q₀,…,q_T is expressed as the product of the initial distribution and the transition matrix entries, which in PBN becomes a specific projected state sequence ⟨π|A^{1}|q₁⟩·⟨q₁|A|q₂⟩·…·⟨q_T|A|⟩. The classic weather example (sunny, cloudy, rainy) is used to illustrate this formulation.

The second, more substantial, contribution concerns Hidden Markov Models (HMMs). Here two probability bases are defined: one for the hidden state sequence and another for the observation sequence. The full joint probability factorises into P(q)·P(o|q), where P(q) is the MSP‑projected probability of the hidden chain and P(o|q) is obtained by applying the emission matrix B to each hidden state. In PBN the joint probability reads ⟨π|A|q₁⟩·⟨q₁|B|o₁⟩·⟨q₂|A|q₂⟩·⟨q₂|B|o₂⟩ …, showing that the HMM is simply a cascade of state‑projection and observation‑projection operators.

To validate the theory, the authors apply the Viterbi algorithm to the well‑known Weather‑Stone HMM. Given a sequence of observed stone conditions (dry, wet), the algorithm searches for the most probable hidden weather sequence using the MSP‑based probability calculations. The resulting path is compared with the output of the Elvira Bayesian network software, confirming that the PBN/MSP approach yields identical results while offering a more compact algebraic representation.

The paper then extends the formalism to Factorial HMMs, where multiple independent Markov chains evolve in parallel. Each chain possesses its own MSP, and the overall system transition is represented by the tensor product A₁⊗A₂⊗…⊗A_K. Observations are generated by a combined emission operator that acts on the joint hidden space. PBN unifies these components into a single high‑dimensional inner product, simplifying both model specification and inference.

Brief discussions are provided on two further generalisations. First, an extended HMM with feedback (where current observations influence future transitions) is accommodated by adding a time‑dependent feedback term to the transition operator, which the MSP can still handle. Second, continuous‑time Markov models replace the discrete transition matrix with the matrix exponential exp(Qt); the MSP is then expressed with differential operators, showing that the same bracket notation can be used for both discrete and continuous time dynamics.

Overall, the authors demonstrate that Probability Bracket Notation together with the Markov Sequence Projector yields a unified, algebraically elegant description of a wide class of dynamic Bayesian networks (DBNs), including VMMs, HMMs, Factorial HMMs, and their extensions. This unification clarifies the relationship between state evolution and observation generation, reduces the notational overhead in derivations, and remains compatible with existing Bayesian network tools. The paper suggests future work on non‑homogeneous transitions, continuous state spaces, and hybrid models that combine the PBN framework with deep learning architectures for richer sequential data modelling.