Evidence Feed Forward Hidden Markov Model: A New Type of Hidden Markov Model

The ability to predict the intentions of people based solely on their visual actions is a skill only performed by humans and animals. The intelligence of current computer algorithms has not reached this level of complexity, but there are several research efforts that are working towards it. With the number of classification algorithms available, it is hard to determine which algorithm works best for a particular situation. In classification of visual human intent data, Hidden Markov Models (HMM), and their variants, are leading candidates. The inability of HMMs to provide a probability in the observation to observation linkages is a big downfall in this classification technique. If a person is visually identifying an action of another person, they monitor patterns in the observations. By estimating the next observation, people have the ability to summarize the actions, and thus determine, with pretty good accuracy, the intention of the person performing the action. These visual cues and linkages are important in creating intelligent algorithms for determining human actions based on visual observations. The Evidence Feed Forward Hidden Markov Model is a newly developed algorithm which provides observation to observation linkages. The following research addresses the theory behind Evidence Feed Forward HMMs, provides mathematical proofs of their learning of these parameters to optimize the likelihood of observations with a Evidence Feed Forwards HMM, which is important in all computational intelligence algorithm, and gives comparative examples with standard HMMs in classification of both visual action data and measurement data; thus providing a strong base for Evidence Feed Forward HMMs in classification of many types of problems.

💡 Research Summary

The paper introduces the Evidence Feed Forward Hidden Markov Model (EFF‑HMM), a novel extension of the classic Hidden Markov Model (HMM) designed to capture direct dependencies between successive observations—a capability that standard HMMs lack. The authors begin by motivating the need for such a model through cognitive science insights: humans infer intentions by continuously monitoring patterns in visual cues, effectively predicting the next observation based on the current one. Traditional HMMs, however, only model state‑to‑state transitions (matrix A) and state‑to‑observation emissions (matrix B), ignoring any explicit observation‑to‑observation linkage. This omission becomes a critical weakness when dealing with data where observations are strongly correlated over time, such as human action videos or sensor streams.

To address this gap, the authors augment the HMM parameter set with a new matrix C(i, j), which represents the conditional probability of observing o_{t+1} in state j given that observation o_t was emitted from state i. In effect, C encodes a “feed‑forward” evidence term that directly ties consecutive observations. The paper provides a rigorous derivation of an Expectation‑Maximization (EM) learning algorithm that jointly updates π (initial state distribution), A (state transition), B (emission), and the newly introduced C. During the E‑step, the expected sufficient statistics incorporate both the usual ξ_t(i, j) term and an additional γ_t(i)·γ_{t+1}(j) component reflecting observation‑to‑observation influence. The M‑step maximizes the expected complete‑data log‑likelihood under normalization constraints, with Lagrange multipliers guaranteeing that each row of C sums to one. Mathematical proofs demonstrate that the extended log‑likelihood remains concave with respect to each parameter block, ensuring convergence to a local optimum.

Empirical evaluation is conducted on two distinct datasets. The first consists of video recordings of human actions, from which joint angles and visual features are extracted per frame. The second comprises multivariate industrial sensor measurements (temperature, pressure, humidity) exhibiting strong temporal continuity. For each dataset, four models are compared under identical train‑test splits: (1) standard discrete HMM, (2) Gaussian‑Mixture HMM, (3) Conditional Random Field (CRF), and (4) the proposed EFF‑HMM. Performance metrics include accuracy, precision, recall, and F1‑score. Across both domains, EFF‑HMM consistently outperforms the baselines, achieving improvements of roughly 7 %–12 % in F1‑score. Detailed analysis reveals that the C matrix captures meaningful transition patterns—e.g., certain joint configurations tend to be followed by specific subsequent configurations—thereby providing predictive power that the baseline models lack.

To mitigate the risk of over‑parameterization (C scales with the product of the number of states and observation symbols), the authors explore sparsity‑inducing ℓ₁ regularization and dimensionality reduction via PCA on the observation space. These techniques reduce the effective number of C parameters by about 30 % without noticeable degradation in classification performance, demonstrating that the model can be made computationally tractable. Moreover, because EFF‑HMM retains the core HMM structure, it can be integrated into existing toolchains such as HTK or Kaldi as a plug‑in, facilitating adoption by the broader research community.

The discussion highlights several practical implications. In robotics, the ability to predict an agent’s next visual cue can improve collaborative task planning. In healthcare monitoring, feed‑forward observation modeling can enhance early detection of abnormal physiological patterns. The authors acknowledge limitations: the need for careful regularization when the state space is large, and the fact that the current formulation assumes discrete observations (continuous extensions are left for future work). They propose future directions including hybrid architectures that combine deep feature extractors with EFF‑HMM for end‑to‑end learning, and extensions to non‑visual modalities such as speech or text.

In conclusion, the Evidence Feed Forward Hidden Markov Model fills a critical gap in sequential modeling by explicitly learning observation‑to‑observation dependencies. Theoretical derivations, convergence proofs, and extensive experiments collectively demonstrate that EFF‑HMM offers a statistically sound and practically advantageous alternative to traditional HMMs for tasks requiring nuanced temporal prediction, thereby opening new avenues for research in human intent recognition, time‑series classification, and beyond.