One of the key lessons from my recent study on fairness-aware and explainable machine learning was that a model can perform well statistically while still producing substantially different outcomes across demographic groups.
I evaluated machine-learning models across three socially consequential decision contexts: Adult Income, German Credit, and COMPAS recidivism prediction. The comparison showed that predictive performance and fairness were not necessarily aligned.
For example, the Adult Income model demonstrated strong predictive discrimination, but significant disparities were observed across gender, race, and age-related groups. Applying threshold-based mitigation substantially reduced the overall measured demographic disparity, although residual disparities remained.
The German Credit analysis presented a different challenge. The model performed strongly overall, particularly in identifying good-credit cases, yet its ability to identify bad-credit cases varied considerably between demographic groups. This highlighted an important point: aggregate performance metrics can conceal group-specific weaknesses.
The COMPAS analysis provided perhaps the clearest illustration of the performance–fairness trade-off. Threshold optimisation substantially increased recall, but it did not improve overall fairness and resulted in lower accuracy and precision. In other words, simply changing the decision threshold was not sufficient to resolve the underlying disparities.
The explainability analysis using SHAP and LIME added another dimension. The models relied heavily on substantive predictive variables, but demographic and socially relevant attributes also contributed to predictions. In COMPAS, for instance, prior criminal history and the existing risk-score variable were dominant predictors, while age, gender and race-related variables also contributed to model decisions.
This led to an important conclusion:
Fair AI cannot be assessed by accuracy alone, and explainability cannot substitute for fairness evaluation.
A responsible ML workflow should therefore examine at least three complementary questions:
- Does the model predict accurately?
- Does it perform equitably across relevant groups?
- Can we understand what drives its predictions?
For me, this has reinforced the importance of integrating performance evaluation, group-level fairness metrics, bias mitigation, and explainable AI rather than treating them as separate stages.
This is particularly relevant as AI moves into high-impact domains such as employment, financial services, healthcare, criminal justice, and critical infrastructure. The objective should not simply be to build models that predict well, but models whose limitations, disparities, and decision mechanisms can be systematically examined.
Key takeaway: A high-performing model is not necessarily a fair model—and an explainable model is not necessarily an unbiased one.
#AI #MachineLearning #ResponsibleAI #FairnessInAI #ExplainableAI #SHAP #LIME #AWSAI #AWSML #ArtificialIntelligence #DataScience
Loading article