How low is the bar for publishing a study showing stay-at-home orders work? Lower than you think.

JAMA edition: fit an absurd exponential model to virus growth, credit the stay-at-home order for the bad out-of-sample fit.

Conclusion: stay-at-home orders saved lives!

According to this model, without the stay-at-home order, CO would have had 422,000 hospitalizations (10% of CO population) by the end of April. Nobody thinks that is plausible.

Placebo test #1— the graph should look different for a state that didn't have a stay-at-home order, right? I fitted the same curve to the same dates in Arkansas (no stay-at-home order).

The result is identical to the graphs in the JAMA paper.

Placebo test #2: If a study this silly found that stay-at-home orders had *no* effect, do you think JAMA would have published it?

This a bad look for science.

It's hard to wrap your head around, but it's likely that (i) social distancing works, and (ii) it is nearly hopeless to measure its effect in a diff-in-diff setting where everyone in the country started social distancing at the same time.

Link to the study: https://jamanetwork.com/journals/jama/fullarticle/2766673