Read on Twitter
THREAD
1) I'm getting increasingly worried about the narrative of supposed 'lost learning' and I want to explain why I don't think school closures had much of an impact at all.
It's important, because the sum of public money on the verge of being wasted is becoming insane.
1) I'm getting increasingly worried about the narrative of supposed 'lost learning' and I want to explain why I don't think school closures had much of an impact at all.
It's important, because the sum of public money on the verge of being wasted is becoming insane.
2) The headline above is from an article in which a think tank genuinely tries to argue that the impact on LIFE TIME earnings of a few months off school cd be up to 3.4% per child...and calls for *10 times* more investment.
The evidence just isn't there. It's smoke & mirrors.
The evidence just isn't there. It's smoke & mirrors.
3) When you look at the EEF's summaries of research into this, they all look pretty dramatic. Take this one, the first on the list: apparently Year 2 pupils were '2 months behind'. Two months? Says who? What does two months of learning look like? It's meaningless...
4) It comes from a suspiciously neat conversion table that the EEF has made up. It has just decided that certain effect sizes are equal to a certain number of months of additional or deficient learning. These get put out on press releases and are then seemingly accepted verbatim.
5) Let's take a closer look at that exact study. You'd have thought it would be pretty simple: give a group of children a test after lockdown that children their age took at the same point in their school careers before lockdown.
6) This *is* what they did, but then the results are passed through so many layers of jargon and academic witchcraft that they end up sounding a million times worse than they actually are. Let me illustrate:
7) When you get to the results section in the study, it shows that the mean score dropped from 100 to 98 - which is already being framed as dramatic btw.
But this is the scaled score, not the actual number of questions that children answered correctly.
But this is the scaled score, not the actual number of questions that children answered correctly.
8) Annoyingly, I can't get hold of the conversion table for this exact test. But I have seen every other year group, and they're almost identical.
So let's take Year 3 as our example and apply the exact same drop in scores - a drop of 2 scaled points...
So let's take Year 3 as our example and apply the exact same drop in scores - a drop of 2 scaled points...
9) As you can see, a difference of 2 scaled points could easily mean a difference of 2 raw marks (out of 80). It's worth pointing out by the way that 26/80 was the pass mark BEFORE the pandemic...
10) Now let me show you what this could have meant in practice.
This is what that drop of 2 scaled points could easily have looked like in concrete terms. Two children: one pre-COVID, one post COVID.
Can you even spot the difference?
This is what that drop of 2 scaled points could easily have looked like in concrete terms. Two children: one pre-COVID, one post COVID.
Can you even spot the difference?
11) As you can see, the scores could be completely identical apart from a tiny drop in a specific maths topic. Their maths knowledge in major areas could be largely untouched.
Yet, according to the EEF, this shows a child two months behind where they would have been.
Yet, according to the EEF, this shows a child two months behind where they would have been.
12) Even if we turn the dial up and assume the drop of 2 scaled points represents 6 dropped marks out of 80, it's still hard to spot the difference:
13) If this was your child's post-lockdown score compared to a child who took the test two years ago at the same age, would you be calling for the government to invest an extra £10-15 BILLION on catch up? Or would you shrug it off and do a bit of revision on co-ordinates?
14) The problem is, all the data in these studies is like this - almost completely opaque.
If there has been a significant loss of learning, we should at least be trying to get an idea of what the gaps are. It could literally be one tiny topic within maths - we wouldn't know.
If there has been a significant loss of learning, we should at least be trying to get an idea of what the gaps are. It could literally be one tiny topic within maths - we wouldn't know.
15) While you're here, take a quick look at this data from PixL. These are the average results of 10,000 primary children in Year 2, who took the same test as children their age, pre-COVID, in the same half term:
16) A drop of 3% on one single test paper (which is equivalent to dropping 2 marks out of 60).
This was the ONLY year group which went down - the rest either stayed the same or went UP. Here's Year 3 (the children who were in Y2 during the 1st school closure)
They did better:
This was the ONLY year group which went down - the rest either stayed the same or went UP. Here's Year 3 (the children who were in Y2 during the 1st school closure)
They did better:
17) There is other research btw which shows virtually no change, including some today from the @FFTEduDatalab: https://ffteducationdatalab.org.uk/2021/05/have-disadvantaged-pupils-fallen-further-behind-during-the-pandemic/
18) Why am I so bothered by this? Because the claims are getting more and more absurd (the World Bank estimates a global loss to those children's future earnings of $10 trillion) and the demands for public money more outrageous.
19) I've said it before and I'll say it again: the belief that lockdown affected learning presupposes that without school closures, children would have retained the knowledge they were due to be taught.
There is SO much evidence that this is not true.
There is SO much evidence that this is not true.
20) I know it's unpopular to point it out, but we really do kid ourselves in normal times. Underachievement hides in plain sight across the country. Whether you take recent science results, which show that only 9% of 11yo on free school meals reach the expected standard, or...
21) KS2 question level analysis of SATs papers, which shows enormous swathes of the curriculum completely unlearned, or the crater-sized gaps in knowledge at GCSE which I've blogged about before.
22) When children aren't learning much already, a few months off school doesn't seem to make much difference.
The route to mass fluency and a better education for underprivileged children is not an expensive one. It's not catch up we need, it's an admission of systemic failures.
The route to mass fluency and a better education for underprivileged children is not an expensive one. It's not catch up we need, it's an admission of systemic failures.
