Read on Twitter
Almost every year we talk about the fundamental problem with standardised testing, and as with everything in the year of covid, this latest debacle simply provides an extreme example that proves our point more succinctly:
The issue is that exam results have two 'audiences' and both want different things. Exam boards, the DofE, universities, employers and the media want them to be an objective measure of students' abilities.
While students themselves, teachers and their schools want the results to be as high as possible because their futures depend on that.
Neither is incorrect, but you can't serve both ends with the same system: they're intrinsically opposed to one another. You can't set an exam that determines a person's future and expect it to reflect their average performance.
It's the very definition of experimenter bias, if what you're expecting to get out of it is an empirical result for the overall abilities of school or 6th form leavers.
Normally this just plays out as media outrage at "exams getting easier", but this year we get to see the two competing aims collide and actually affect results. The algorithm that hurt so many students' grades was designed by the group that wants an objective measure.
So it used a system of likely grade distributions to create what is, on the face of it, a fairly realistic set of overall results. Because the whole point is for it to reflect reality, right? That's why we have standardised testing: to measure that year's cohort.
Except we also use the same results to determine whether those individual students get to attend university, or which university they get to attend, or what letters go on their CV when they enter the most competitive and unfair job market in a century.
The disconnect is obvious. The algorithm might well be fair (though there's plenty of evidence it isn't), but if its only role is to deliver a typical distribution of grades that can serve in place of any other year's results, it's not doing what exams really do.
Because an exam result doesn't reflect a student's average attainment over their academic career. It doesn't reflect the quality of teaching they received or the facilities available at their school.
It reflects their performance on a single day when *everything is on the line*, for better or worse. It might over- or underrepresent their actual ability. Doesn't matter: this is what standardised testing is. It's a high pressure system.
So if what you're looking for from those results is a steady performance measured over time, you're going to be disappointed. It's not empirical. It's inherently skewed to produce better and better results.
The analogy we always use is this: if you think higher results year-on-year can only be because exams are getting easier, you presumably think that athletics events are also reducing the length of their racetracks.
How else do you explain sprint records being improved over and over? Usain Bolt must have run a shorter course than Carl Lewis. What, people have just got *faster* over time? Ridiculous.
Let's extended the analogy further. We couldn't do the Olympics this summer. Maybe we should have handed out medals based on nations' past performances?
It's actually easy: every athlete scheduled to compete will have done enough recent trials of their event(s) that their coaches can give a 'predicted performance'. Then we just take those scores, tweak them based on results of previous Olympics, and hand out medals accordingly.
Would you expect an athlete to accept a gold medal for that? Would you expect an athlete to accept *no* medal for that? "Sorry, the algorithm says you would have finished in last place."
It's self-evidently absurd. No one would stand for it, and it was never even suggested as a possibility. It would be pointless, serving only to provide a placeholder for an abandoned year of top level athletics, to neaten up the record books.
But then, *everyone* wants Olympic athletes to do their best. Exams differ in that everyone except the students and schools want the results to reflect only an average performance.
A student cohort achieving high grades is a failure of the system, despite the same system being optimised to produce high grades. Schools doing better is bad, even though schools are allocated funding based on their performance.
How can you seek to improve education standards, when the only measure you have that those standards are being met is dismissed as inaccurate? Do you want better schools or not?
It just...doesn't work. And this particular debacle is simply proof of the competing aims in play. No other country does this either. It's a peculiarly British obsession with placing their children in a Catch-22.
If you fail, you're feckless wasters, and if you succeed it's because the exams were too easy, since a load of feckless wasters like you can't have done that well.
And of course none of the above applies to the privately educated, who are obviously not feckless wasters since, after all, their school has a strict policy against even letting feckless wasters through the door!
