New example of possible ethnic profiling when determining recidivism risk. This time in the Netherlands.
#recidivism #prediction #AI #OxRec #ethnicprofiling

@random_walker @mireillemoret @fborgesius @PopTechWorks

Determining recidivism risk is difficult. Really difficult. And controversial. In my research group ( @maaslawtech), COMPAS has regularly come up when discussing the potential and limitations of risk assessment tools. COMPAS has been accused to of ethnic profiling, not performing

better than humans, and not doing better than prediction models with less variables or 'old school' analyses such as logistic regression. Turns out that these things are happening right under my nose. The Dutch Parole Board has been using OxRec since 2019. The issues:

1. It doesn't perform that great. AUC values of around 0.67-0.69 are reported, with false alarm rates going up to sometimes 80%. OK for scientific purposes, but for practical purposes? Imagine 100 cancer suspects going to the doctor, and between 30 and 80 of them getting

chemotherapy when they actually did not need it or did not have cancer in the first place. 2. Possible ethnic profiling. OxRec uses neighborhood as one of the variables for determining recidivism risk. The variable consists of indicators such as welfare recipients, median income,

and single status. It's true that criminals are more likely to be found in poor neighborhoods than in rich neighborhoods, the opposite is not true: the mere fact you live in a certain neighborhood in itself does not make you less or more criminal. Postal code in itself is

meaningless. It's about what it picks up. And it might inadvertently pick up on ethnicity or social class, similarly to job application algorithms that pick up on gender based on applicants' hobbies. We should not use algorithms for important matters like recidivism risk

if we don't know what they measure or have procedures in place that are properly tested. For example, does the information the algorithm produces enter the process at the beginning or at the end (as a final check)? Anchoring effect, confirmation bias, and tunnel vision research

stress the importance of when information enters a process, even with humans (parole officer, judge) in the loop. Curious what parole officers and judges ( @JudgeJoyce_) think about this.

The response of the Dutch Parole Board is in: the risk assessment does not include questions regarding race, ethnicity etc., and its use is warranted since it is based on scientific research. ( https://www.reclassering.nl/actueel/nieuws/risicotaxatie-instrument-reclassering-wetenschappelijk-gevalideerd)

Meanwhile, @fborgesius finding a 2015 report on ethnic profiling in sentencing: https://twitter.com/fborgesius/status/1276480652821639168

Questions will be posed by Dutch MP: https://twitter.com/kathalijne/status/1276437738993209344