Risk Bounds for CART Classifiers under a Margin Condition

Risk bounds for Classification and Regression Trees (CART, Breiman et. al. 1984) classifiers are obtained under a margin condition in the binary supervised classification framework. These risk bounds are obtained conditionally on the construction of the maximal deep binary tree and permit to prove that the linear penalty used in the CART pruning algorithm is valid under a margin condition. It is also shown that, conditionally on the construction of the maximal tree, the final selection by test sample does not alter dramatically the estimation accuracy of the Bayes classifier. In the two-class classification framework, the risk bounds that are proved, obtained by using penalized model selection, validate the CART algorithm which is used in many data mining applications such as Biology, Medicine or Image Coding.

💡 Research Summary

The paper investigates the statistical performance of Classification and Regression Trees (CART) in the binary supervised classification setting under a margin condition, a widely used assumption that limits the probability mass near the decision boundary. The authors first formalize the margin condition in the style of Tsybakov, stating that for some constants (C_0>0) and (\kappa>0), the probability that the conditional class probability (\eta(X)=P(Y=1|X)) lies within a distance (t) of 1/2 is bounded by (C_0 t^{\kappa}). This condition captures the intuition that well‑separated classes make the learning problem easier and allows for sharper risk bounds than those derived solely from VC‑dimension arguments.

The analysis proceeds conditionally on the construction of the maximal deep binary tree (T_{\max}) that CART builds by recursively splitting the training data until each leaf is pure or a pre‑specified depth is reached. The complexity of any subtree (T\subseteq T_{\max}) is measured by the number of leaves (|T|). By applying empirical process techniques together with the margin condition, the authors derive a non‑asymptotic upper bound on the excess risk of the empirical risk minimizer on a fixed subtree:

\