RISE: Interactive Visual Diagnosis of Fairness in Machine Learning Models

Evaluating fairness under domain shift is challenging because scalar metrics often obscure exactly where and how disparities arise. We introduce \textit{RISE} (Residual Inspection through Sorted Evaluation), an interactive visualization tool that converts sorted residuals into interpretable patterns. By connecting residual curve structures to formal fairness notions, RISE enables localized disparity diagnosis, subgroup comparison across environments, and the detection of hidden fairness issues. Through post-hoc analysis, RISE exposes accuracy-fairness trade-offs that aggregate statistics miss, supporting more informed model selection.

💡 Research Summary

The paper introduces RISE (Residual Inspection through Sorted Evaluation), an interactive visual analytics system designed to diagnose fairness issues in machine learning models, especially under domain shift. Traditional fairness assessment relies on scalar metrics such as demographic parity, equalized odds, or mean difference, which aggregate performance across the entire prediction distribution. While useful, these numbers can hide localized disparities that emerge only for specific subpopulations or in particular regions of the prediction space. RISE addresses this blind spot by visualizing the sorted signed‑error residuals of a model—defined as the difference between predicted probability and true label—on a two‑dimensional plot where the x‑axis represents the normalized rank of the residual and the y‑axis shows its signed magnitude.

The core insight is that the shape of the residual curve encodes both accuracy and fairness information. A curve that stays close to the horizontal axis indicates low overall error, while the alignment of curves for different protected groups (e.g., gender, race) reveals whether errors are distributed evenly. RISE further extracts two “knee” points—one on the left (convex) and one on the right (concave)—using the Kneedle algorithm. These knees mark the transition from low‑error to high‑error regimes. By comparing the positions of the knees across groups, the system can detect whether one group experiences error escalation earlier (different percentile) or more severely (different residual magnitude).

From these visual elements, RISE derives three quantitative indicators:

1. F mean (Median Alignment Fairness) – measures how closely the medians of the groups align with the overall median. It is defined as (F_{\text{mean}} = 1 - \frac{|\tilde m_0 - \tilde m_1|}{2}), where (\tilde m_a) is the deviation of group a’s median from the overall median. Values near 1 indicate balanced error distributions.

2. F shift (Horizontal Knee Disparity) – captures the average relative horizontal displacement of the group knees from the global knees, i.e., the difference in percentile where the error transition occurs. Lower values mean the groups switch error regimes at similar ranks, suggesting less unfairness.

3. F acc (Vertical Knee Disparity) – captures the average relative vertical displacement of the group knees, i.e., the difference in residual magnitude at the transition. Again, lower values indicate more uniform error severity across groups.

These indicators complement traditional metrics (accuracy, demographic parity, mean difference) and are displayed alongside them in a concise table. The interactive interface lets users switch datasets, protected attributes, and models on the fly, instantly updating the residual plot, median rulers, knee markers, and an adaptive segmentation that highlights local disparity segments.

The authors demonstrate RISE on the BDD100K driving dataset, simulating a domain shift between clear and rainy weather. They compare three representative algorithms:

• IRM (Invariant Risk Minimization) – explicitly penalizes error instability across environments, aiming for fairness at the cost of accuracy. RISE shows almost perfectly aligned knees (F shift ≈ 0, F acc ≈ 0) but a residual curve far from the x‑axis, reflecting low overall accuracy (≈ 0.49).

• MBDG (Model‑Based Domain Generalization) – prioritizes accuracy while attempting a middle ground. Its residual curve hugs the x‑axis (high accuracy ≈ 0.96) but exhibits asymmetric knees, leading to modest F shift and F acc values that reveal hidden subgroup bias.

• IGA (Invariant Gradient Alignment) – seeks a balanced trade‑off. Its curve is relatively flat, close to the x‑axis, and the group curves are tightly overlapped. The three RISE indicators are all intermediate, matching the visual impression of a “fair and accurate” model.

These visualizations expose trade‑offs that scalar tables obscure. For instance, MBDG’s high accuracy hides a localized disparity that becomes evident only when examining the knee positions. Conversely, IRM’s fairness is evident from the knee alignment, but the user can instantly see the accuracy penalty.

The paper’s contributions are threefold: (1) a novel visual paradigm that turns sorted residuals into an interpretable diagnostic canvas; (2) three residual‑based fairness metrics that quantify median alignment and knee disparities; (3) an open‑source interactive tool that supports real‑time exploration of model fairness under domain shift.

Limitations include the current focus on binary classification and binary protected attributes; extending to multi‑class or multi‑attribute settings would require more sophisticated knee detection and possibly higher‑dimensional visual encodings. The reliance on Kneedle also means that in very smooth residual curves, knee detection may be less reliable.

Future work aims to broaden RISE to multi‑class problems, large language models (visualizing token‑level confidence gaps), and structured subgroup definitions (e.g., hierarchical text categories). The authors envision RISE evolving into a modality‑agnostic fairness diagnostic interface applicable to vision, language, and speech systems.