I listened to @TheUSMLE podcast about Step 2 CS in the hopes of learning about how the virtual CS exam is being validated (as a reliable predictor of clinical skills, let alone whether it catches things med schools miss, protects pts, etc). Some thoughts:

https://rb.gy/jf3fy7 

There's a lot of dubious transparency (translucency?) here; Over the summer, USMLE conducted surveys/focus groups of a range of stakeholders.

But they haven't released results, specified who the stakeholders are (beyond generic categories), or described their research process.

There was no real discussion of how the new CS is being validated:

"Our immediate priority is to relaunch a modified Step 2 CS... but we've also considered a second and separate effort... to continue to enhance clinical skills assessment."

Shouldn't validation precede relaunch?

"What was identified as not working as well?"

- Limited number of test sites
- Cost
- Artificial nature of the interactions
- Limited feedback

They say they weren't surprised to hear any of this. But they don't discuss how any of these things are being fixed prior to relaunch.

They say the goal of the exam is to determine whether medical students can gather, communicate, and reason about clinical information.

But the question is: can they assess this via a few hours of online interactions more reliably than med schools do over four years in person?

^ If this were a clinical trial, there would not be equipoise: no one thinks virtual Step 2 CS is a more sensitive/specific test than years of in-person med school. The burden should be on USMLE to prove that Zoom CS is a value add before they charge [no word on cost] to take it.