Learning Structured Outputs from Partial Labels using Forest Ensemble

Learning structured outputs with general structures is computationally challenging, except for tree-structured models. Thus we propose an efficient boosting-based algorithm AdaBoost.MRF for this task. The idea is based on the realization that a graph is a superimposition of trees. Different from most existing work, our algorithm can handle partial labelling, and thus is particularly attractive in practice where reliable labels are often sparsely observed. In addition, our method works exclusively on trees and thus is guaranteed to converge. We apply the AdaBoost.MRF algorithm to an indoor video surveillance scenario, where activities are modelled at multiple levels.

💡 Research Summary

The paper addresses the notoriously difficult problem of learning parameters in conditional random fields (CRFs) when the underlying graph has a general structure and when only partial labels are available. Traditional maximum‑likelihood approaches become intractable for arbitrary graphs, and approximate inference methods such as loopy belief propagation (BP) or the Wainwright‑Jaakkola‑Willsky (WJW) algorithm lack convergence guarantees and cannot easily handle missing variables. To overcome these limitations, the authors propose AdaBoost.MRF, a boosting‑based learning framework that operates exclusively on tree‑structured sub‑graphs (spanning trees) of the original network.

The key insight is that any graph can be expressed as a super‑position of spanning trees. Each spanning tree defines a tractable conditional distribution Pτ(v | o) over the visible variables v given observation o. The algorithm treats the log‑likelihood of a tree, h(v, o)=log Pτ(v | o), as a weak learner. At each boosting iteration a new tree is selected, its weight α is determined, and the strong learner H is updated by adding α · h to the current model. The loss function is derived from AdaBoost.MR’s exponential rank loss, adapted to the partial‑label setting as

L_inco(H)=∑i∑v exp{H(v,o_i)−H(v_i,o_i)}.

Because the sum over all possible v is intractable, the authors apply Hölder’s inequality to obtain an upper bound L_H that depends only on the product of tree likelihoods evaluated at the observed partial label v_i. This bound is convex and can be minimized efficiently. Under mild assumptions (positive α’s that sum to one) the bound guarantees a global optimum, and the ensemble of weighted trees approximates the full graph’s distribution—hence the term “Markov Random Forest.”

The method naturally incorporates missing labels: the weak learner’s log‑probability can be written as log ∑_h Pτ(v,h | o), marginalising over hidden components h. Consequently, AdaBoost.MRF can learn from data where only a subset of the structured output is observed.

Experimental validation is performed on a multi‑level activity‑recognition task in indoor video surveillance. Activities are modeled at several hierarchical levels (e.g., room, person, sub‑action) using a dynamic CRF (DCRF). The dataset contains partially labeled sequences, mimicking real‑world scenarios where only some actions are annotated. AdaBoost.MRF is compared against maximum‑likelihood learning using BP and WJW as inference engines, as well as against layered hidden Markov models (LHMMs) and flat CRFs. Results show that AdaBoost.MRF achieves higher classification accuracy while requiring far less computation and exhibiting stable convergence. Moreover, the hierarchical DCRF outperforms flat models, confirming the benefit of multi‑scale modeling.

In summary, the contributions of the paper are:

1. A novel formulation of CRF learning as a boosting problem over spanning‑tree weak learners.
2. An analytically tractable convex loss bound that guarantees convergence.
3. An elegant handling of partially observed structured outputs without resorting to heuristic imputation.
4. Empirical evidence that the approach outperforms state‑of‑the‑art approximate inference methods on a realistic, multi‑level activity‑recognition benchmark.

The work opens several avenues for future research, including more sophisticated tree‑selection strategies, extensions to larger label spaces (e.g., natural‑language parsing), and online or incremental versions of AdaBoost.MRF.