Projected Boosting with Fairness Constraints: Quantifying the Cost of Fair Training Distributions

Boosting algorithms enjoy strong theoretical guarantees: when weak learners maintain positive edge, AdaBoost achieves geometric decrease of exponential loss. We study how to incorporate group fairness constraints into boosting while preserving analyzable training dynamics. Our approach, FairBoost, projects the ensemble-induced exponential-weights distribution onto a convex set of distributions satisfying fairness constraints (as a reweighting surrogate), then trains weak learners on this fair distribution. The key theoretical insight is that projecting the training distribution reduces the effective edge of weak learners by a quantity controlled by the KL-divergence of the projection. We prove an exponential-loss bound where the convergence rate depends on weak learner edge minus a “fairness cost” term $δ_t = \sqrt{\mathrm{KL}(w^t | q^t)/2}$. This directly quantifies the accuracy-fairness tradeoff in boosting dynamics. Experiments on standard benchmarks validate the theoretical predictions and demonstrate competitive fairness-accuracy tradeoffs with stable training curves.

💡 Research Summary

The paper introduces FairPROJ, a novel boosting algorithm that incorporates group‑fairness constraints while preserving the classic exponential‑loss convergence guarantees of AdaBoost. Traditional attempts to enforce fairness by directly re‑weighting training examples break AdaBoost’s weight‑update recursion, making it impossible to retain the clean theoretical analysis. FairPROJ solves this by separating the distribution used to compute the boosting coefficient from the distribution used to train the weak learner.

At each boosting round t the algorithm first computes the exponential‑weights distribution qₜ induced by the current ensemble fₜ₋₁, exactly as AdaBoost does. It then projects qₜ onto a convex set C_ε of distributions that satisfy linear moment fairness constraints (e.g., equal opportunity, demographic parity) by solving a KL‑divergence minimization problem. The projection has a closed‑form dual: wₜᵢ ∝ qₜᵢ·exp(−λ*⊤g(i)), where λ* solves a low‑dimensional convex program. The resulting fair distribution wₜ is used to train the weak learner hₜ. Crucially, the boosting coefficient αₜ is still computed from the error of hₜ measured under qₜ, so the exponential‑loss recursion L_exp(fₜ)=L_exp(fₜ₋₁)·E_{i∼qₜ}