Thread warning: why the "psychometric validity" argument from the MCC is utter nonsense. 1/ https://twitter.com/giulianaguarna/status/1317135548851552258?s=19

Arguing that virtual exams would not be "psychometrically defensible" is analogous to arguing that if the CT scanner's down, and you have a patient with suspected PE, you should just wait a few weeks for the scanner to get fixed and leave the patient in limbo in the meantime. 2/

Let's break down the analogy

Patient = exam candidate
CT angio = traditional MCCQE 2
Other diagnostics (V/Q, US, D-dimer, etc) = best possible exam in a pandemic
Empiric treatment = waiving the exam
Leaving the patient in limbo = deferring the exam

3/

The point here is you can't just refuse to practice medicine when the CT is down. You have to do the best you can in the circumstances you're in. You might give anticoagulants to a few patients who don't actually have a PE. That's ok! 4/

What is *not* ok is privileging the sanctity of your carefully constructed diagnostic algorithm when the most important test isn't available to you. 5/

Medicine is real life. In crisis conditions, you sometimes have to do things differently from normal. Everyone understands that.

The same is true of assessment. It's not ok to just defer this determination indefinitely. 6/

(Particularly because, by this analogy, the pretest probability of PE is about 99%, and the very need for the test is itself in doubt, but this is a digression.) 7/

Furthermore, the "psychometric defensibility" argument presupposes that the traditional MCCQE 2, relative to other possible assessments, has some special validity. 8/

I'm no expert on psychometrics and validation, but the notion that there's some great reference standard against which to compare the MCCQE is just silly. (A bit like pulmonary angiography, come to think of it.) 9/

Yes, you can show that the MCCQE 2 correlates well with the CCFP or RCPSC exams, or with the absence of regulatory complaints, but this doesn't actually establish "validity" in any practical sense. 10/

And it certainly doesn't establish that the historical version of this test performs better than any other reasonable replacement test that might be used. 11/

Again, this isn't the time for arguing about psychometrics. It's a pandemic, and we need to license doctors and get them into practice.

The MCC looks like a bunch of internists arguing about sensitivity and specificity while the patient, unnoticed, suffers a PEA arrest.

12/12.