Safe Fairness Guarantees Without Demographics in Classification: Spectral Uncertainty Set Perspective

As automated classification systems become increasingly prevalent, concerns have emerged over their potential to reinforce and amplify existing societal biases. In the light of this issue, many methods have been proposed to enhance the fairness guarantees of classifiers. Most of the existing interventions assume access to group information for all instances, a requirement rarely met in practice. Fairness without access to demographic information has often been approached through robust optimization techniques,which target worst-case outcomes over a set of plausible distributions known as the uncertainty set. However, their effectiveness is strongly influenced by the chosen uncertainty set. In fact, existing approaches often overemphasize outliers or overly pessimistic scenarios, compromising both overall performance and fairness. To overcome these limitations, we introduce SPECTRE, a minimax-fair method that adjusts the spectrum of a simple Fourier feature mapping and constrains the extent to which the worst-case distribution can deviate from the empirical distribution. We perform extensive experiments on the American Community Survey datasets involving 20 states. The safeness of SPECTRE comes as it provides the highest average values on fairness guarantees together with the smallest interquartile range in comparison to state-of-the-art approaches, even compared to those with access to demographic group information. In addition, we provide a theoretical analysis that derives computable bounds on the worst-case error for both individual groups and the overall population, as well as characterizes the worst-case distributions responsible for these extremal performances

💡 Research Summary

The paper addresses the pressing problem of ensuring fairness in binary and multi‑class classifiers when demographic (sensitive) attributes are unavailable, a scenario common in real‑world deployments due to legal, ethical, or practical constraints. Existing fairness‑enhancing techniques either rely on proxy groups—assuming that unsupervised clusters correlate with the true sensitive attributes—or adopt a minimax fairness perspective through robust optimization. The latter typically defines an uncertainty set by re‑weighting the empirical training distribution, which often over‑emphasizes outliers and leads to overly pessimistic models that sacrifice overall accuracy while still failing to guarantee robust worst‑group performance.

To overcome these drawbacks, the authors propose SPECTRE (Spectral Uncertainty Set for Robust Estimation), a novel minimax‑fair algorithm that operates without any demographic information. The method builds on the Minimax Risk Classifier (MRC) framework but introduces two key innovations. First, it maps each input vector into a random Fourier feature (RFF) space, effectively representing the data in the frequency domain. By adjusting the spectrum—retaining only the most informative low‑frequency components and discarding high‑frequency noise—the approach mitigates the influence of extreme samples that would otherwise dominate the worst‑case distribution. Second, SPECTRE defines a “spectral uncertainty set” that constrains how far the adversarial (worst‑case) distribution may deviate from the empirical distribution, not in the original data space but in the Fourier‑transformed space. This constraint is expressed via a norm‑based (e.g., ℓ₂ or KL‑divergence) bound on the spectral coefficients, providing a richer and more controllable description of distributional shift than simple re‑weighting.

Theoretical contributions include closed‑form upper bounds on the worst‑case risk for any group and for the overall population, derived under the spectral uncertainty set. The analysis also characterizes the structure of the worst‑case distribution, showing that it concentrates on a limited set of spectral directions, which explains why the method avoids excessive pessimism. Moreover, the bounds are computable, enabling practitioners to assess the safety of the learned classifier without resorting to extensive validation.

Empirically, the authors evaluate SPECTRE on the American Community Survey (ACS) data across 20 U.S. states, predicting outcomes such as income brackets, educational attainment, and employment status. No sensitive attributes (e.g., race, gender, age) are used during training or inference. Baselines include recent distributionally robust optimization (DRO) methods—GDRO, ARL, BPF—as well as methods that have access to demographic labels. Performance is measured by overall accuracy and worst‑group accuracy (the highest error among all latent sub‑populations). Results show that SPECTRE consistently achieves the highest average worst‑group accuracy while also delivering the smallest inter‑quartile range, indicating stable and reliable fairness guarantees. Overall accuracy is comparable to, or slightly better than, the baselines, demonstrating that the spectral regularization does not sacrifice predictive power.

The paper acknowledges limitations: the spectral truncation level and the size of the uncertainty bound are hyper‑parameters that currently require cross‑validation; scaling to very large datasets may demand more efficient optimization schemes; and the method focuses on 0‑1 loss, whereas many practical systems use surrogate losses for computational tractability. Future work is suggested on automated hyper‑parameter selection, extensions to multi‑sensitive‑attribute settings, and integration with deep neural architectures.

In summary, SPECTRE offers a principled, theoretically grounded, and empirically validated solution for achieving safe, robust fairness in classification tasks without any demographic information, advancing the state of the art in both fairness research and robust machine learning.