Read on Twitter
We'll be livetweeting @CommonsEd questioning the leadership of @ofqual on the impact of COVID-19 on education and children’s services at around 9.45. You can watch the hearing for yourself at: https://parliamentlive.tv/event/index/a3d523ca-09fc-49a5-84e3-d50c3a3bcbe3 #edselctte
The proceeding has just started. Witnesses are: Roger Taylor, Chair, Ofqual; Dr Michelle Meadows, Executive Director for Strategy and Research, Ofqual; Julie Swan, Executive Director, General Qualifications; Dame Glenys Stacey, Acting Chief Regulator, Ofqual #edselctte
Chair @halfon4harlowMP says @ofqual has said it's "not me guv" over the results issues.
Taylor says he has personally apologised for an "extremely anxiety-making incident" #edselctte
Taylor says he has personally apologised for an "extremely anxiety-making incident" #edselctte
Is it not extraordinary for you to say we were aware of this but were unable to find a solution, Halfon asks.
Taylor says the impact is better for students from lower socioeconimic backgrounds with standardisation approach. #edselctte
Taylor says the impact is better for students from lower socioeconimic backgrounds with standardisation approach. #edselctte
Taylor says they felt it was fairer to use standardisation but accepts the level achieved "was not satisfactory" #edselctte
What authority does @ofqual and @educationgovuk have for decisions, asks Halfon.
Taylor says their role is to have regards for DfE policy, but awarding of grades must be "valid", work between years and command confidence #edselctte
Taylor says their role is to have regards for DfE policy, but awarding of grades must be "valid", work between years and command confidence #edselctte
Who had authority over the standardisation model?
Taylor says the best way at the outset was to hold exams or delay them but neither of these were "acceptable". @GavinWilliamson made the decision to cancel exams without consulting @ofqual, Taylor said #edselctte
Taylor says the best way at the outset was to hold exams or delay them but neither of these were "acceptable". @GavinWilliamson made the decision to cancel exams without consulting @ofqual, Taylor said #edselctte
But Ofqual took all the decisions on the development of the algorithm, admits Taylor #edselctte
Ofqual told Williamson using CAGs could not be delivered and be valid and trustworthy. GW then announced it was govt policy and @ofqual board decided to attempt to find a way to do it #edselctte
Ofqual agreed an approach with the DfE and WIlliamson then published it, before being contacted by GW several hours later, who said it was "not in line with govt policy" #edselctte
Board convened at around 10pm that evening and agreed situation was "out of control" and thought Williamson's decisions were not consistent with their duties. #edselctte
This was the debacle which unfurled on Monday 17 August: https://schoolsweek.co.uk/ofqual-removes-mock-exam-appeals-policy-just-hours-after-publication/ #edselctte
"A couple of days later the decision was made to accept CAGs," Halfon says.
Taylor says there was "no way" to deliver fair grades for students under the previous approach #edselctte
Taylor says there was "no way" to deliver fair grades for students under the previous approach #edselctte
We initially advised against cancelling exams, says Taylor. The algorithm was done by continuous collaboration and they were constructing the system with the DfE and exam boards #edselctte
Committee member @tomhunt1988 says they find it "hard to understand" the policy would command broad public support
Taylor says "from the outset we said it was tremendously difficult" but there were advantages to the way results were decided #edselctte
Taylor says "from the outset we said it was tremendously difficult" but there were advantages to the way results were decided #edselctte
Taylor says they were confident the standardisation model did not hurt disadvantaged communities after Scotland U-turned on using CAGs: https://schoolsweek.co.uk/scotland-withdraws-all-downgraded-exam-grades/ #edselctte
Taylor says they were exploring with exam boards to reach out to students who had a "good case" for appeals, before the mock grades appeals policy came in #edselctte
Did you raise issues with ministers, asks Halfon?
Taylor says the moderation process gave strong candidates from disadvantaged backgrounds better results.
Meadows says they used the pupil database of their source of data and private schools' data is "patchier" #edselctte
Taylor says the moderation process gave strong candidates from disadvantaged backgrounds better results.
Meadows says they used the pupil database of their source of data and private schools' data is "patchier" #edselctte
"Any fluctuation in outcomes this year is extremely similar to previous years," Meadows says Ofqual developed 12 algorithms and tested them against 2019 data
#edselctte
#edselctte
Should you have done your own mock exam with the algorithm, Halfon asks.
Meadows says they tested it "thoroughly" against centre type, ethnicity, socioecominc background etc #edselctte
Meadows says they tested it "thoroughly" against centre type, ethnicity, socioecominc background etc #edselctte
Were the risks of the mode raised with ministers?
"Throughout the process," says Taylor #edselctte
"Throughout the process," says Taylor #edselctte
Of Ofqual's 10-member advisory group, only one worked with exam boards. The rest were from Ofsted, UCAS and other outside organisations, says Meadows #edselctte
Are you going to make all the minutes and comms Ofqual had with the DfE?
Taylor says they can but would need to discuss with DfE. "We don't have anything to hide." #edselctte
Taylor says they can but would need to discuss with DfE. "We don't have anything to hide." #edselctte
What meetings did you have with the DfE between A-level results day and the CAG U-turn?
Taylor said they had conversations with Williamson and policy advisors to him where they fed back concerns about the proposal #edselctte
Taylor said they had conversations with Williamson and policy advisors to him where they fed back concerns about the proposal #edselctte
Any statistical process you were going to use was always going to leave people disappointed, Taylor is asked. He says it became "very difficult" to see how people would accept the approach as fair #edselctte
Taylor's apology "doesn't cut the mustard," says committee member Ian Mearns. "You couldn't see there wouldn't be a huge public backlash to the whole scenario... I've really got to ask: was the prime directive in this process to stop grade inflation?" #edselctte
Given improvement in schools, shouldn't there be some increase in achievement by students year on year?
Taylor says he does not think it would have been appropriate to ignore Williamson's guidance because they did not think it deliverable #edselctte
Taylor says he does not think it would have been appropriate to ignore Williamson's guidance because they did not think it deliverable #edselctte
"The final result of the moderated grades did allow for 2-3pc in inflated grades, which was very significant and larger than would have come about from teaching improvements" #edselctte
No form of assessment is perfect to capture the ability of each pupil, Taylor admits.
Meadows says the level of standardisation is similar to other years #edselctte
Meadows says the level of standardisation is similar to other years #edselctte
Hand of the DfE has been "too intricately" involved if Ofqual is to retain its independence, says Mearns
Taylor says they were in "completely uncharted territory". It is appropriate the decision on how to proceed was for politicians #edselctte
Taylor says they were in "completely uncharted territory". It is appropriate the decision on how to proceed was for politicians #edselctte
As Ofqual took the guidance down on 17 August when Williamson told them, Halfon questions their independence. Taylor said it needed to be "negotiated and managed" but they have full independence #edselctte
What was the degree to which assurances were sought about alternative options for results and how they were considered?
Taylor says they put their advice to try and run exams, delay exams or use calculated grades to the Dept for Health #edselctte
Taylor says they put their advice to try and run exams, delay exams or use calculated grades to the Dept for Health #edselctte
Your advice was calculated grades was "worst case scenario". Did ministers say let's explore the risks?
Taylor says they did list those risks in very vivid detail. Other countries which did exams did encounter resistance #edselctte
Taylor says they did list those risks in very vivid detail. Other countries which did exams did encounter resistance #edselctte
Where there is no independent standardised work from the student, we are in a situation that changes the relationship between the appeals, The notion a student says the teacher got it wrong, Taylor says there was no process for that #edselctte
What about if the student's appeal was signed by the headteacher?
Taylor says the centre could simply tell the exam board they had got it wrong, without the need for an appeal #edselctte
Taylor says the centre could simply tell the exam board they had got it wrong, without the need for an appeal #edselctte
Saying only the institution has the right to appeal, you're saying they can mark their own homework, the committee says, which seems "inherently unjust".
Taylor says there is not an independent piece of evidence to choose between students and the centre #edselctte
Taylor says there is not an independent piece of evidence to choose between students and the centre #edselctte
Swan says they were meeting schools minister Nick Gibb weekly about results. There was also a paper about the predicted "widespread dissatisfaction" about grades. Evidence was also submitted to No 10 Downing Street about the issues #edselctte
"It was clear to us from the very start that students would've thought they would've got a better grade with exams... but we will never know," Swan says #edselctte
We set out range of evidence we asked schools/colleges to take into account. There was no prohibition on putting in a grade for a student if no student had achieved it at that centre before, Swan says #edselctte
Mearns asks: Will Ofqual release all school submissions for grades to a 3rd party like the Royal Statistical Societ so forensic analysis can be undertaken into which centres tried to, for example, game the system? #edselctte
Taylor says they will not publish it but agree there is an enormous amount we can learn from the data submitted from schools. Important third parties have access to that #edselctte
Taylor cannot say when minutes and comms with DfE will be published, but they will publish their records #edselctte
Who constructed the guidance for the algorithm, the algorithm, and was it outsourced?
Meadows says they consulted on the principles of it and set up a working group with exam boards. Decisions taken were Ofqual's and they picked algorithm with most accurate results #edselctte
Meadows says they consulted on the principles of it and set up a working group with exam boards. Decisions taken were Ofqual's and they picked algorithm with most accurate results #edselctte
On socially-distanced exams, could we use them in the future, and is it "regrettable" we could not progress with them?
Taylor says they are looking at how other countries did exams. But it was not clear how long schools would be closed, so exams could not go ahead #edselctte
Taylor says they are looking at how other countries did exams. But it was not clear how long schools would be closed, so exams could not go ahead #edselctte
Prior to this hearing, Ofqual gave a statement to @CommonsEd, which has now been published: https://twitter.com/CommonsEd/status/1301090748797005825 #edselctte
Taylor said Ofqual said it would not have "improved accuracy" to have "given grades away to centre-assessed grades". Scotland's results did not make them any more "acceptable" to students #edselctte
Appeals process, before the govt changed it, was "incredibly narrow," Halfon says, and it was based on bias and discrimination
Taylor says students can ask the centre to appeal on those grounds #edselctte
Taylor says students can ask the centre to appeal on those grounds #edselctte
At what point did it flag concerns that there were several anomalous subject areas?
Taylor says they did not publish data which would allow individuals to find how grades had been calculated #edselctte
Taylor says they did not publish data which would allow individuals to find how grades had been calculated #edselctte
"Our aim throughout has been to be transparent," says Meadows, to show the limitations of using a statistical modelling process #edselctte
One college had over 70pc of their students being downgraded, says committee member Christian Wakeford #edselctte
Halfon says it was a "Charge of the Light Brigade" policy as Ofqual knew there were problems and carried on anyway
Taylor says the notion the algorithm was deficient is the "wrong conclusion" #edselctte
Taylor says the notion the algorithm was deficient is the "wrong conclusion" #edselctte
Differences b/w socioeconomic groups were ironed out, Taylor says. "It is not true to say there were large numbers of large movements in grades." #edselctte
But he admits that to say this was fair, and the level of information was "too low to be acceptable" #edselctte
Cambridge Assessment said they were rebuffed when they asked to review or scrutinise standardisation model.
Meadows says "We have an approach were we bring together exam boards and sign off guidance, so there was an opportunity to raise concerns. None were raised" #edselctte
Meadows says "We have an approach were we bring together exam boards and sign off guidance, so there was an opportunity to raise concerns. None were raised" #edselctte
"There is a misunderstanding what happened with calculated grades," Meadows says, what you will see is there is v little difference in outcomes for disadvantaged groups #edselctte
What evidence has Ofqual collected on the effect on standardisation by region?
Meadows says there are concerns about bias in the model and they kept it under review. We always worry this is something we would have to address #edselctte
Meadows says there are concerns about bias in the model and they kept it under review. We always worry this is something we would have to address #edselctte
Meadows say they will publish information on final results by region and reiterates Taylor's commitment to publishing data to third parties #edselctte
In the absence of papers this year, Ofqual felt the fairest thing to do was to remove caps on students sitting foundation papers #edselctte
"If you can't get more than a grade 5, how can you develop a system so they can get higher than that," Halfon asks.
Meadows calls that an "artificial cap" so they removed it #edselctte
Meadows calls that an "artificial cap" so they removed it #edselctte
Was an equality impact assessment done by Ofqual and can you publish it?
Meadows says yes and it has been published and there has been a "great deal" of equalities work done #edselctte
Meadows says yes and it has been published and there has been a "great deal" of equalities work done #edselctte
On BTECs, students had to wait days for results - do you agree with decision to grade these in line with CAGs was fair?
Meadows says rebuilding public confidence with grades has been paramount so they can understand Pearson's decision to revisit grades #edselctte
Meadows says rebuilding public confidence with grades has been paramount so they can understand Pearson's decision to revisit grades #edselctte
Committee member Jonathan Gullis says the BTEC approach "makes a mockery" - how much consultation was done with Pearson on the algorithm and how much did you talk with them before the U-turn? #edselctte
Halfon calls the decision to align BTECs with CAGs "an afterthought" #edselctte
Taylor agrees it was "not acceptable" for BTEC students and clarifies there was a huge difference between how vocational and general quals work #edselctte
Halfon says they were "hardly mentioned" until it was realised 450k students were affected
Taylor says the use of CAGs in BTECs was "completely different" to general quals. They had "no authority" over how Pearson would respond #edselctte
Taylor says the use of CAGs in BTECs was "completely different" to general quals. They had "no authority" over how Pearson would respond #edselctte
Should Ofqual apologise to BTEC students?
Taylor says he is "very happy" to apologise for the consequences. Swan says there were a few results issued later than Fri because info needed to calculate results had to be extracted from centres #edselctte
Taylor says he is "very happy" to apologise for the consequences. Swan says there were a few results issued later than Fri because info needed to calculate results had to be extracted from centres #edselctte
"Much more work involved" in calculating results for BTECs, Swan adds #edselctte
Mearns says there is an "inherent bias" in the algorithm to ESOL students.
Meadows says they ran the algorithm through 2019 results, including through socioeconomic groups and they all perform "quite differently" in exams #edselctte
Meadows says they ran the algorithm through 2019 results, including through socioeconomic groups and they all perform "quite differently" in exams #edselctte
Mearns says they should guard against using KS2 results after he says Gibb says they are "not at all important #edselctte
What is your message to students in centres who may have received lower grades to their peers? What is your response to calls for centres to resubmit grades without historical performance data?
Swan says some centres were "very cautious" in submitting grades #edselctte
Swan says some centres were "very cautious" in submitting grades #edselctte
Some centres were "much more optimistic" but it is a moot point on whether it is an advantage to receive an inflated grade as they are hearing concerns from colleges that students from optimistic centres have got grades which shows an inaccurate level of ability #edselctte
This is something @FEWeek reported in the wake of A-level and GCSE students receiving CAGs: https://feweek.co.uk/2020/08/20/colleges-ask-for-flexibility-in-covid-19-catch-up-funding-to-support-borderline-students/ #edselctte
Exam boards are having discussions with some centres which believed they made a mistake in their CAGs, Swan says #edselctte
Ofqual is having regular check-ins with exam boards about appeals, says Swan #edselctte
Do you recognise anxiety students feel to challenge result determined by centre puts them at unease?Students have decided not to appeal as they do not understand how to navigate process, Apsana Begum says #edselctte
Swan says there were 60,000 challenges to exam results last year and the difficulty this year is determining if a student is disappointed in the normal course of events #edselctte
How are SEND students being supported, Begum asks
Swan says some students who would've had a reasonable adjustment in exams believe this has not been taken into account. Exam boards looking into it #edselctte
Swan says some students who would've had a reasonable adjustment in exams believe this has not been taken into account. Exam boards looking into it #edselctte
Ofqual has also produced guidance on exam results using videos with sign language to help accessibility #edselctte
Did you threaten to resign because Williamson did not show sufficient support for Ofqual, Halfon asks Taylor, who says they discussed and resolved the issue #edselctte
The DfE expressed confidence in the watchdog after Williamson had declined to: https://schoolsweek.co.uk/dfe-expresses-full-confidence-in-ofqual-despite-williamson-refusing-to-do-so-yesterday/ #edselctte
How much delay do you think you could put into exams to have them marked in time?
Swan says to meet published results day, they could have up to 2 weeks before it gets "challenging" #edselctte
Swan says to meet published results day, they could have up to 2 weeks before it gets "challenging" #edselctte
One problem is asking teachers to give up holidays to mark exams, Swan says. Most markers are active teachers. #edselctte
Swan says they can talk about incentives for centres to support teachers to mark in that way #edselctte
As more students have to isolate, have you considered another paper for them?
Swan says they have guidance from health bodies on how to run autumn exams safely #edselctte
Swan says they have guidance from health bodies on how to run autumn exams safely #edselctte
When do you expect to tell Yr13 students about next year's exams?
Swan says the decision needs to be taken with devolved admins but she recognises "need for certainty" and are working with DfE to get conclusions in weeks #edselctte
Swan says the decision needs to be taken with devolved admins but she recognises "need for certainty" and are working with DfE to get conclusions in weeks #edselctte
Taylor says they have lobbied for traditional exams to take place #edselctte
Gullis says his "big concern" is around local lockdowns - what is Ofqual doing to make sure exams are fair? And what is plan B - CAGs, coursework?
If we do not enter 2021 with clear set of recommendations, Ofqual has to have plan B ready to go - likely to be CAGs #edselctte
If we do not enter 2021 with clear set of recommendations, Ofqual has to have plan B ready to go - likely to be CAGs #edselctte
If God forbid, exams can't take place, would one option be for an independent assessor to check CAGs, Halfon asks.
Taylor says he agrees with Gullis. "Strong view" not to use CAGs next year #edselctte
Taylor says he agrees with Gullis. "Strong view" not to use CAGs next year #edselctte
"Absolutely essential" students can take fair, comparative test, Taylor adds. But if exams can't be done, there are mechanisms incl using online tests #edselctte
Hunt asks on grade inflation, are you concerned about the impact on this year's cohort and future ones?
Taylor says they are v. conscious of the effect and they may see some impact on dropout rates and due to deferrals, this will impact next cohort #edselctte
Taylor says they are v. conscious of the effect and they may see some impact on dropout rates and due to deferrals, this will impact next cohort #edselctte
Halfon says of the thousands who have to be predicted or assessed but can't without a centre, what are you going to do to support them?
Swan says for those the autumn exams will be available to them #edselctte
Swan says for those the autumn exams will be available to them #edselctte
Taylor says they carried out a number of different types of analysis on the impact of the results approach to high achieving individuals in low-achieving centres #edselctte
Were ministers made aware of the model keeping students' grades consistent over time?
Taylor says there would be "inevitably" be a group of students who would have done better in the exams, but Ofqual would not be able to ID them #edselctte
Taylor says there would be "inevitably" be a group of students who would have done better in the exams, but Ofqual would not be able to ID them #edselctte
Taylor says ministers were made aware through letters and about a third of students would have got a grade higher/lower in exams #edselctte
Do you agree a centre in a large entry subject would have gotten its rank order correct?
Meadows says teachers are much more able to accurately rank, but it's impossible to know without testing how inaccurate rank order would be #edselctte
Meadows says teachers are much more able to accurately rank, but it's impossible to know without testing how inaccurate rank order would be #edselctte
"We knew from the outset there would be a large number of changes from what teachers expected," Meadows says #edselctte
Taylor says they did not ignore @CommonsEd's warnings, but Halfon points to how they told Ofqual to publish standardisation model before results day, and the watchdog published it on the day #edselctte
Taylor says they did stop to assess what they were doing before results day, and he says Scotland did adopt combined CAG-standardisation model and this did not produce "any greater public acceptability" #edselctte
Halfon says press have complained Ofqual shut themselves down and complained to talk to media and it was impossible to get through. Why did that happen, shouldn't you have been out reassuring people? #edselctte
Taylor says they were in a "difficult" conversation with the DfE. "We were at variance on the best way to introduce policy and it would not be helpful to be talking to media until this was resolved." #edselctte
Ofqual has a director of communications and 10 officers below them and are about to hire a snr comms person for £80k pa, says Halfon, who questions whether it's "value for money" and asks why they "hid in the Ofqual attic" #edselctte
Taylor says he does think it important communication has to be able to effectively address people's concerns, but Halfon says they didn't and it was "genuinely shocking" they refused to engage with the media - @CommonsEd will look at that in the future #edselctte
Taylor says if they had entered the debate at that point it would have worsened confusion #edselctte
Ofqual's new chief regulator Dame Glenys Stacey says they have their communications under "active review" and a lot of PR officers are "fairly junior" #edselctte
When you were doing analysis did you look at centres using different exam board?
Ofqual says it would have made no difference what exam board a centre was with #edselctte
Ofqual says it would have made no difference what exam board a centre was with #edselctte
On whether Ofqual should be stripped of its independence, Taylor has said this year has been a "huge blow" to public confidence in them, but lists successes around improving exams over the past decade and have shown the real value of an independent exam regulator #edselctte
Dame Glenys says this year has "shown the importance of an independent regulator" and it is a "small organisation doing a very big job" with 15,000 vocational quals as well as general quals #edselctte
"We do have v. important and rare skills and an important job to do," she continues. But says they have Qs they must ask themselves #edselctte
"I will of course look at the way Ofqual works and if there are any changes I will put those in play, even as an acting chief regulator." Says she has the message about value for money "loud and clear" #edselctte
