Step 1 of 8Explainer5 min
Same tool, different mistakes
In 2016, journalists at ProPublica studied COMPAS, a tool that scores how likely a defendant is to commit another crime. They followed more than 7,000 people arrested in Broward County, Florida, in 2013 and 2014, for two years. Here is what they found about the tool’s mistakes:
| Mistake | Black defendants | White defendants |
|---|---|---|
| Labeled higher risk, but did not reoffend | 44.9% | 23.5% |
| Labeled lower risk, but did reoffend | 28.0% | 47.7% |
Black defendants who did not reoffend were almost twice as likely to be wrongly labeled higher risk. White defendants who did reoffend were more often wrongly labeled lower risk.
Think about it
Before you read on: where could a gap like this come from? List two possible sources.
The company behind the tool, Northpointe, rejected ProPublica’s conclusion of racial bias. It argued that the scores were equally accurate for both groups once you account for their different rates of reoffending.
Both sides used the same data, and both did careful math. In Step 3 you will see how they could both be right, and why that matters more than either side winning.
A rule that looks neutral but hits one group harder has