Several students have asked us if AAMC has addressed concerns about the standardization of the COVID-length MCAT.

They have. But, not publicly.

Here is what they've made available to the med schools

#MedTwitter #PreMedTwitter #MedEd #WaiveTheMCAT

There's a lot to unpack here.

1) The mean MCAT score is UP, not down, when compared to the same timeframe last year. The diff IS statistically significant.
2) They are VERY behind on testing.
3) Some people will have experiences that don't fit this data. That's normal.

There are 4 potential reasons for the score increase. We'll give our theory, too.

1) Flynn Effect aka expected score creep over time
2) The shorter test is easier
3) More time to prep
4) Selection bias

1 is possible, 2 is less likely, 3 and 4 are connected.

1). The longer a high-stakes exam is used, the higher the average score will be. This is why exams are re-normed every few years. We know AAMC does this because their MCAT scores have historically been basically normally distributed.
Source: http://aamc-orange.global.ssl.fastly.net/production/media/filer_public/d9/04/d904b7f4-c3d0-4469-aed1-e5afff500d05/mcat_total_and_section_score_percentile_ranks_2020_for_web.pdf

2) Could the shortened MCAT just be easier? Maybe. But, AAMC has made virtually no information about this available to students, so it's hard to tell. The actual content of the exam has not changed, but the timing and break allotment both have.

We do know there is less time per question, as well as a proportional difference in the amount of time spent answering questions versus taking breaks. There is a statistically significant difference in scores between 2019 and 2020, though likely not due to this.

Eventually, we’ll write about the de-standardization of the MCAT exam this year. But, we at least wanted to point out that there are significant objective changes to the timing. AAMC’s assertion that the two exams are “about the same” isn’t acceptable.

3) Coronavirus disrupted everyone’s life. But, some students may have actually benefitted from this disruption. With no classes to attend and the financial ability to not work and/or search for a job, it’s possible that some students had almost unlimited study time.

4) It’s very possible, likely even, that certain populations of students are self-selecting out of taking the MCAT this summer. Prepping for and taking the MCAT is expensive. Low-SES students might be finding themselves unable to do manage this financial burden right now.

3 and 4 are connected, of course. Students with more resources are going to have the ability to "just keep going" through truly adverse circumstances. That is a product of their privilege that isn't available to many.

How do we know? Well, just based on AAMC's own data regarding the avg MCAT score of students from different SES groups and racial backgrounds, as well as their previous statements on the # of students testing this summer, a mean score of 503.8 is very high. We expected 498-501.

For reference, the calculations above were done using data from this email: https://twitter.com/EthicalStudents/status/1288574330146873351?s=20

There are obviously going to be flaws in our calculations because we are not being given access to the full data that AAMC has available. When we requested it, they sent us a publicly available document on confidence intervals. (Yes, really).

https://www.aamc.org/system/files/2020-07/services_mcat_using-mcat-data-in-2021-medical-student-selection-guide_07082020_0.pdf

Of course, seeing the relevance it has to students, AAMC could just unlock this section of their website. We strongly encourage them to do this, and would even venture to say that our current climate where AdComs are "leaking" the data to SEA doesn't inspire trust.

With that aside over, let's address the issue of test cancelations. AAMC previously assured schools that they would have no problem meeting demand this summer. As we previously pointed out, that data was manipulated. https://twitter.com/EthicalStudents/status/1283238950522572801?s=20

Today, they essentially admitted that. They said they're "doing their best" to reschedule people. But, they did not mention that they have tested 7,264 FEWER STUDENTS than they had during this same time frame last year. https://twitter.com/AAMC_MCAT/status/1298288320171180032?s=20

Do they have room for those 7,264 students to sit for the MCAT before the final test day of the year in September? Theoretically, they might. But, do they have room for them to test in safe, socially-distanced test centers within a reasonable distance of their home? No.

This leads us to our thesis, of sorts...

The mean MCAT score is up this summer because fewer students are testing, and those who are testing are disproportionately from high-SES and ORM backgrounds.

They may try to cover this up by referencing increased rates of URM applicants and FAP recipients. We already know this is a false flag. (We also have many anecdotal reports of students being classified as disadvantaged despite never self-IDing as such). https://twitter.com/EthicalStudents/status/1295489724623314944?s=20

Everything we said was going to happen is happening. AAMC is going to continue to manipulate data and demographic info to try to disprove it. Whether or not the final # s that they publish show it, we know that the medical school class of 2025 will be less diverse than others.

Medical schools are doing harm to their communities by continuing to believe AAMC. They are limiting the diversity of their incoming classes, further deepening the town-gown divide that is already so apparent on many campuses.

If schools are serious about fair testing, diverse classes, and well-rounded students, they will #WaiveTheMCAT. They will reject assurances that "all is well" and they will listen to their applicants as they beg for help.

AAMC will be OK. They can take the financial hit.

But, the students who are being kept out may never come back. That is a loss to the field of medicine, and it is a loss to the communities they may one day serve.

That isn't so easy to recover from.

Worth noting -- this thread hasn't even dug into the issues that accommodated students are still facing. They are also being kept from taking the MCAT exam this summer. They are also at risk of losing thousands of $ s in application fees.

None of this is acceptable.

The MCAT is no longer standardized. They are ~7000 exams behind. Students are going to be left out this year. Medicine will be less diverse.

AAMC's good plan was ruined by 2020. Can't blame them for that. But, we can demand they do better. #WaiveTheMCAT http://www.ethicalstudents.org 

There are a few people we want to thank for their expert data analysis and help here, but we are waiting to make sure that's OK first! So, we are using this as a placeholder to thank everyone who helped with this! Hope to @ some folks soon!