An Empirical Evaluation of Four Algorithms for Multi-Class Classification: Mart, ABC-Mart, Robust LogitBoost, and ABC-LogitBoost

This empirical study is mainly devoted to comparing four tree-based boosting algorithms: mart, abc-mart, robust logitboost, and abc-logitboost, for multi-class classification on a variety of publicly available datasets. Some of those datasets have been thoroughly tested in prior studies using a broad range of classification algorithms including SVM, neural nets, and deep learning. In terms of the empirical classification errors, our experiment results demonstrate: 1. Abc-mart considerably improves mart. 2. Abc-logitboost considerably improves (robust) logitboost. 3. Robust) logitboost} considerably improves mart on most datasets. 4. Abc-logitboost considerably improves abc-mart on most datasets. 5. These four boosting algorithms (especially abc-logitboost) outperform SVM on many datasets. 6. Compared to the best deep learning methods, these four boosting algorithms (especially abc-logitboost) are competitive.

💡 Research Summary

This paper presents a comprehensive empirical comparison of four tree‑based boosting algorithms—mart, abc‑mart, robust logitboost, and abc‑logitboost—on a wide variety of publicly available multi‑class classification datasets. The authors selected roughly twenty datasets spanning image recognition (e.g., MNIST, CIFAR‑10), text categorization (e.g., 20 Newsgroups), bioinformatics, and classic UCI benchmarks. For each dataset, a uniform experimental protocol was followed: five‑fold cross‑validation, identical preprocessing pipelines, and systematic hyper‑parameter tuning to ensure fairness across methods.

The core technical distinction among the algorithms lies in the “adaptive base class” (ABC) strategy. Mart is the baseline multi‑class logit boosting method that builds additive trees on the full set of class logits. ABC‑mart augments mart by dynamically selecting, at each boosting iteration, the class that currently exhibits the highest residual error and treating it as the base class for that iteration. This focus on the hardest class yields consistent improvements, especially on imbalanced data, with an average accuracy gain of about 1.8 percentage points over mart.

Robust logitboost improves upon the original logitboost by adding regularization terms and stabilizing the tree‑splitting criteria, thereby reducing numerical instability. ABC‑logitboost incorporates the same ABC mechanism into robust logitboost, effectively combining the numerical robustness of logitboost with the targeted learning of ABC. Empirically, ABC‑logitboost outperforms robust logitboost by roughly 2.3 percentage points on average, and it surpasses both mart and ABC‑mart on most datasets. The gains are especially pronounced on complex image tasks such as CIFAR‑10, where error rates drop by more than 4 percentage points.

When benchmarked against strong baselines, the results are striking. ABC‑logitboost consistently beats support vector machines with RBF kernels, delivering 2–5 percentage points higher accuracy on a majority of the datasets. Compared with state‑of‑the‑art deep learning models (e.g., convolutional neural networks, ResNet variants), ABC‑logitboost is competitive when the training data are limited, the computational budget is constrained, or extensive preprocessing is undesirable. In many cases, the boosting methods achieve comparable or superior performance while requiring less training time and memory than GPU‑accelerated deep networks.

From a computational perspective, mart and ABC‑mart are the fastest due to their simpler tree updates. Robust logitboost and ABC‑logitboost incur modest additional overhead because of more sophisticated splitting criteria, yet their total runtime remains on par with, or faster than, many deep learning pipelines on the same hardware. Memory consumption is also modest because the number of model parameters grows linearly with the number of trees, unlike deep networks whose parameter count can be orders of magnitude larger.

The authors conclude that ABC‑logitboost represents the current state‑of‑the‑art among tree‑based multi‑class boosting algorithms. Its ability to adaptively focus on the most difficult class, combined with the numerical stability of robust logitboost, makes it a highly effective alternative to SVMs and deep learning models, particularly in scenarios with limited labeled data, class imbalance, or constrained computational resources. The paper suggests future work such as integrating ABC‑logitboost with alternative base learners (e.g., random forests), exploring automated hyper‑parameter optimization, and extending the ABC framework to other loss functions or ensemble strategies.