Multiple Object Tracking in Unknown Backgrounds with Labeled Random Finite Sets

This paper proposes an on-line multiple object tracking algorithm that can operate in unknown background. In a majority of multiple object tracking applications, model parameters for background processes such as clutter and detection are unknown and vary with time, hence the ability of the algorithm to adaptively learn the these parameters is essential in practice. In this work, we detail how the Generalized Labeled Multi Bernouli (GLMB) filter a tractable and provably Bayes optimal multi-object tracker can be tailored to learn clutter and detection parameters on the fly while tracking. Provided that these background model parameters do not fluctuate rapidly compared to the data rate, the proposed algorithm can adapt to the unknown background yielding better tracking performance.

💡 Research Summary

The paper addresses a fundamental limitation of most multi‑object tracking (MOT) algorithms: the need for a priori knowledge of background parameters such as clutter rate and detection probability. While the Generalized Labeled Multi‑Bernoulli (GLMB) filter is Bayes‑optimal and provides labeled tracks, it assumes that these parameters are known and stationary. In many practical scenarios—e.g., autonomous driving video streams—the clutter intensity and detection profile change over time, leading to severe performance degradation if fixed values are used.

To overcome this, the authors embed the GLMB filter within a Jump Markov System (JMS) framework. Each object carries a discrete mode (e.g., motion class or sensor class) that evolves according to a finite‑state Markov chain with transition probabilities ϑ⁺(m⁺|m). The state transition density and measurement likelihood become mode‑conditioned, i.e., f⁺(ζ⁺,m⁺|ζ,m) = f⁺(ζ⁺|ζ)·ϑ⁺(m⁺|m) and g(m)(z|ζ) respectively. By augmenting the object state with its mode, the multi‑object system can still be described as a labeled random finite set, allowing the GLMB recursion to be applied directly.

The core contribution is the derivation of a JMS‑GLMB recursion (Proposition 1) that retains the conjugate‑prior property of the GLMB while incorporating mode‑dependent survival, birth, detection, and clutter parameters. The recursion yields a sum over hypothesis sets I, association histories ξ, new label sets I⁺, and association maps θ⁺, each weighted by ω(I,ξ,I⁺,θ⁺). Crucially, the clutter intensity κ(m) and detection probability P_D(m) appear inside the multi‑object likelihood ψ, enabling their online estimation from the data.

A special case—“non‑interacting” JMS—is introduced where (1) all newborn objects share a common mode determined by their label, and (2) existing objects never switch modes. Under this assumption the label space partitions into disjoint subsets B(m) for each mode, dramatically simplifying the recursion. The resulting filter can be implemented efficiently using Murty’s algorithm or Gibbs sampling for hypothesis truncation, achieving linear or near‑linear complexity in the number of measurements.

The authors embed an adaptive learning step for κ and P_D within each update. Assuming that background parameters evolve slower than the measurement update rate, the filter updates their estimates at every frame using the sufficient statistics accumulated from the current set of measurements. This yields a filter that simultaneously tracks objects, maintains unique labels, and refines its own measurement model.

Experimental validation is performed on both synthetic data with time‑varying clutter and on the KITTI video dataset. In the synthetic case, the JMS‑GLMB accurately tracks the true clutter trajectory and outperforms the λ‑CPHD filter in OSPA error. On KITTI, the proposed method produces more stable tracks, fewer ID switches, and higher overall MOTA scores compared with the λ‑CPHD filter, which lacks label continuity and adaptive background learning. The computational load remains compatible with real‑time operation thanks to the efficient truncation scheme.

Limitations are acknowledged: rapid, abrupt changes in clutter or detection profiles may outpace the filter’s adaptation, and the non‑interacting mode assumption may not hold for scenarios where objects change class (e.g., a vehicle turning into a pedestrian‑like silhouette). Future work could integrate change‑point detection for rapid parameter shifts and extend the model to allow mode transitions for existing tracks.

In summary, the paper presents a principled, Bayesian‑optimal multi‑object tracker that learns unknown background characteristics on‑the‑fly, preserving labeled tracks and achieving real‑time performance. This advancement bridges the gap between theoretically optimal MOT and practical “plug‑and‑play” deployment in dynamic, uncertain environments such as autonomous driving, surveillance, and robotics.