You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
This project investigates how fairness intervention strategies can mitigate conflicts among widely used fairness metrics such as demographic parity, equal opportunity, and predictive parity, even when a machine‑learning model is well‑calibrated. As the paper notes, “unequal base rates among demographic groups drive conflicts between fairness metrics, even with well‑calibrated models”. The study demonstrates that calibration alone does not guarantee equitable outcomes across demographic groups.
Using synthetic data with unequal base rates, the project evaluates three major bias‑mitigation strategies:
Re‑sampling (pre‑processing)
Re‑weighting (in‑processing)
Post‑processing threshold adjustments
The analysis shows that each method improves some fairness metrics while worsening others, illustrating the inherent trade‑offs described in fairness literature. The project contributes a structured comparison of these interventions and highlights the practical challenges of achieving fairness in real‑world AI systems.
1. Analyse fairness conflicts in well‑calibrated models
The project aims to demonstrate that even when predicted probabilities match observed outcomes across groups, fairness metrics can still diverge due to unequal base rates. As the paper states, “improving one fairness metric often leads to deterioration in another.”
2. Evaluate three fairness intervention strategies
The project seeks to systematically test re‑sampling, re‑weighting, and post‑processing threshold adjustments to understand how each affects:
Demographic parity
Equal opportunity
Predictive parity
Model calibration
3. Provide evidence‑based guidance for AI governance
By quantifying trade‑offs, the project aims to support organisations and regulators in selecting fairness definitions aligned with ethical, legal, and operational priorities.
4. Contribute to the scientific community
The project extends existing research by offering a comparative, metric‑driven evaluation of fairness interventions, helping advance practical fairness‑aware modelling approaches.
How We Will Achieve These Goals
1. Generate controlled synthetic data
2. Train a baseline, well‑calibrated model
3. Compute fairness metrics
4. Apply three mitigation strategies
Each strategy is implemented and re‑evaluated:
Re‑sampling
Re‑weighting
Post‑processing
Analyse how each method shifts fairness metrics.
Evaluate impact on calibration and predictive parity.
Summarise results in comparative tables and visualisations.
6. Interpret findings in the context of AI governance
The fundraising project is solely intended to contribute:
Registration: $400
Tickets to Beijing : $2000
Accomodation: $700
Misc: $400
I have solely contributed for this project. I work with team members of various disciplines to develop and build digital products. The project is research article to be presented in ISDSA data science conference.
If the project fails to achieve its goals, several negative outcomes may arise:
1. Misleading assumptions about fairness
Without demonstrating metric conflicts, stakeholders may incorrectly assume that a well‑calibrated model is inherently fair, contradicting the paper’s finding that “calibration… does not ensure fairness across metrics.”
2. Inability to select appropriate fairness interventions
Failure to evaluate mitigation strategies would leave practitioners without guidance on:
When to use re‑sampling vs. re‑weighting
When post‑processing is preferable
How to balance fairness with calibration and accuracy
3. Increased risk of biased AI deployment
Organisations may deploy models that:
Disadvantage minority groups
I have contributed myself so far.