MIHT: A Hoeffding Tree for Time Series Classification using Multiple Instance Learning

Due to the prevalence of temporal data and its inherent dependencies in many real-world problems, time series classification is of paramount importance in various domains. However, existing models often struggle with series of variable length or high dimensionality. This paper introduces the MIHT (Multi-instance Hoeffding Tree) algorithm, an efficient model that uses multi-instance learning to classify multivariate and variable-length time series while providing interpretable results. The algorithm uses a novel representation of time series as “bags of subseries,” together with an optimization process based on incremental decision trees that distinguish relevant parts of the series from noise. This methodology extracts the underlying concept of series with multiple variables and variable lengths. The generated decision tree is a compact, white-box representation of the series’ concept, providing interpretability insights into the most relevant variables and segments of the series. Experimental results demonstrate MIHT’s superiority, as it outperforms 11 state-of-the-art time series classification models on 28 public datasets, including high-dimensional ones. MIHT offers enhanced accuracy and interpretability, making it a promising solution for handling complex, dynamic time series data.

💡 Research Summary

The paper introduces MIHT (Multi‑Instance Hoeffding Tree), a novel algorithm designed to tackle multivariate time‑series classification (TSC) problems where series may have variable lengths and high dimensionality. Traditional TSC approaches either assume fixed‑length series or rely on complex generative or deep‑learning models that are computationally heavy and lack interpretability. MIHT addresses these issues by combining a multiple‑instance learning (MIL) representation with an incremental decision‑tree learner based on the Hoeffding bound.

First, each original time series X is transformed into a “bag” of overlapping sub‑series (instances) using a sliding window of length ω and overlap λ. This bag‑of‑sub‑series preserves temporal ordering while allowing the MIL framework to treat each series as a collection of instances. The key assumption is that only a subset of these instances—denoted τ—actually contains the discriminative concept σ needed for classification.

The base learner is an Incremental Decision Tree (IDT), specifically a Hoeffding Tree (HT). The HT starts as a single leaf and updates its statistics online as instances arrive. After every κ instances, the algorithm evaluates potential splits using information gain (or another quality measure) and compares the difference between the two best splits to the Hoeffding bound ε (derived from a user‑specified confidence δ). If the difference exceeds ε, the split is performed, guaranteeing with probability 1‑δ that the chosen attribute is truly superior. This statistical guarantee enables fast, streaming‑compatible tree growth without revisiting past data.

Training proceeds in two phases. In the first phase, all instances from all bags are streamed through the HT, allowing the tree to capture coarse patterns. In the second, an optimization loop iteratively selects the k most representative instances τ for each bag by maximizing the bag‑level likelihood L (Equation 1). These τ are then used to retrain the HT, reinforcing the tree with the most informative sub‑series. The loop repeats until convergence, progressively refining the model’s focus on the true concept σ.

Prediction follows the same bag construction pipeline: a new series is split into overlapping windows, each instance traverses the trained HT, and the leaf’s class distribution yields a posterior probability. The final class is the one with the highest aggregated probability across all instances. Because the model is a single decision tree, the splits directly reveal which variables and time intervals are most influential, providing clear, white‑box interpretability.

The authors evaluate MIHT on 28 public multivariate datasets (including high‑dimensional ones) from the UCR/UEA archives. They compare against 11 state‑of‑the‑art TSC methods such as DrCIF, InceptionTime, ROCKET, and HIVE‑COTE. MIHT consistently outperforms these baselines in overall accuracy and F1‑score, with especially large gains on datasets featuring highly variable lengths or thousands of dimensions. Moreover, MIHT’s training and inference times are competitive, and its memory footprint remains modest due to the incremental nature of the Hoeffding Tree.

Ablation studies examine the impact of window size ω, overlap λ, and the number of selected instances k, showing that reasonable defaults work well across domains but that automatic hyper‑parameter tuning could further improve performance. The authors also discuss limitations: very short series may produce bags with too few instances for reliable Hoeffding bounds, and the method currently relies on linear split tests, which might miss highly non‑linear patterns.

In conclusion, MIHT offers a scalable, accurate, and interpretable solution for multivariate, variable‑length time‑series classification. By uniting MIL’s flexible bag representation with the statistical rigor of Hoeffding Trees, it bridges the gap between high‑performance black‑box models and transparent white‑box approaches. Future work is suggested on adaptive hyper‑parameter selection, incorporation of non‑linear split criteria, and deployment in real‑time streaming environments.