Thread by @ptroncosoruiz, The #alevels2020 controversy (and the upcoming one for #gcse2020) is an example [...]

The #alevels2020 controversy (and the upcoming one for #gcse2020) is an example of the limits of statistical analysis and usage. 1

The procedures applied by OFQUAL may be statistically rigorous and the results may follow the intended distribution (a distribution composed of people!), but was it the right thing to do in the first place? Was it ethical to implement these statistical models? 2

In the case of this year's A level exams, grades are the product of a statistical procedure with inherent error. In lay terms, error in predictions here means that the awarded grades can be thought of as one (the middle point, most likely) of a range of possibilities. 3

The awarded grades could have been different, lower or higher. This is simply how statistical error works. 4

But aside from the methods and how good or bad they are, the first question that policy-makers and social science researchers need to ask themselves is whether it is responsible to apply one method or another. 5

Is it ethical to use predictions as a replacement for actual performance at the individual level to make high-stakes decisions?
Is it ethical to use past years' performance to determine the grades of the current cohort? 6

While we're at it, is it actually legal?
GDPR prevents data controllers to make automated decisions and profiling. 7

Even when permitted, data controllers may need the explicit consent of the subjects of the decisions, who also should retain the right to have a simple way to challenge the decisions made by others (the data controllers) about them. 8

It's been said that using Centre Assessment Grades (CAGs) would have led to unprecedented improvement in performance, but hasn't all this year full of unprecedented events? 9

What about the unprecedented emotional turmoil young people have gone through this year? If the concern was to analyse trends over time reliably, then 2020 could have been simply removed from the trend (as an anomaly) or adjustments could be applied for analytical purposes. 10

I don't think I need to remind anyone that trends over time for many other things have been messed up by the year 2020! 11

The appeal system is another problem as it doesn't allow individual appeals and when schools put forward an appeal, they have to pay a fee. This is unfair and it will only contribute to widen the socioeconomic gap. 12

All in all, behind the algorithms, averages, grades, etc., let's not forget there are young people with dreams and hopes who are being told now that their teacher grades (the people they've worked with for a long time and know them best) are "wrong" or "inflated". 13

They are being told their grades need to be changed in the name of comparability. But can this year's results be confidently compared (even as they are now) with previous and subsequent years? No, because they're not actual grades, they're predictions with inherent error! 14

Some compromise or adjustment will need to be made when comparing. In any case, results were going to be "inflated" from the start, simply because appeals are going to be filed to bump up grades. 15

Nobody in their right mind will file an appeal to downgrade their results! Inevitably, a proportion of appeals will be won, which leads to a higher proportion of better grades. 16

To sum up, exceptions are being made everywhere this year, can't we just give young people a break?
The system has, once again, failed the young and vulnerable. 17

@UoMPolicy @MIEBEEblog #alevels2020 #gcse2020

Just to add a bit more to this: https://twitter.com/ptroncosoruiz/status/1294582104504885254?s=19
Those who have been graded "U" have all the right to be absolutely livid about being made to fail without even a chance to sit the test #alevels2020 #examshambles

https://twitter.com/ptroncosoruiz/status/1294582104504885254?s=19

Latest Threads Unrolled: