Read on Twitter
This hot take has been getting some attention today, so I think it's worth talking about why I think it's specious, and why the headline it comes with ("Lockdowns don't work") is probably wrong (and irresponsible) given the current empirical evidence.
https://twitter.com/lymanstoneky/status/1252795142026608640
https://twitter.com/lymanstoneky/status/1252795142026608640
First off, @lymanstoneky asserts that no nationwide evidence from the US currently exists that lockdowns are effective. This is materially false. Here's our nationwide analysis showing lockdowns are associated w/dramatic declines in county infection growth rates in the raw data.
Here are the results once we control for all time-invariant features of counties, like baseline population health, & time-varying factors, like the availability of testing nationally, as well as state-level testing when we can.
Our analysis suggests that stay-at-home orders are highly effective, w/approx an 8.6% percentage point reduction in infection growth rates after two weeks. By 3-4 weeks out growth rates go negative, meaning the disease is receding. Updated preprint here: https://www.medrxiv.org/content/10.1101/2020.04.13.20063628v2
Why the different results? The main reason is that the Lyman's analysis is focused on the effect of lockdowns on *deaths* (internationally), while we look at new *infections* (in the US). But looking just at deaths is problematic, for several reasons.
First, notice that Lyman does not use COVID19 death data in the analysis, but rather excess all-cause mortality. Probably b/c there's good reason to think that COVID deaths are heavily underreported internationally, both due to under-diagnosis & due to political pressures.
So even if we agreed that we should just look at COVID19 deaths, that variable is being measured here indirectly and with noise. This is especially a concern if lockdowns are associated with changes in deaths for other reasons (traffic accidents, pollution, domestic violence).
Second, deaths lag infections, and it's not clear by how much. Contrary to what the article asserts, we do not yet have a strong handle on the typical progression of the disease. That makes using deaths data for short-term policy analysis tricky at best.
Not to mention that the "peaks" that are being identified here are based off of extremely noisy data. Confidence bands (are these 95%? 90%? 80%? can't tell) around the estimates are broad. It's hard to know if the peaks even exist, let alone be sure of exactly when they occur.
Bottom line: should you draw strong conclusions from the fact that the peak of all-cause mortality is maybe earlier than could be attributable to lockdowns alone? Probably not.
Third, & this is particularly important: deaths data only capture *at best* the mortality costs of COVID, ignoring morbidity. It's likely that morbidity costs are high as well. Some of the reported side effects, even for those who recover, are horrifying: https://www.washingtonpost.com/health/2020/04/22/coronavirus-blood-clots/
So deaths data is no panacea. Is infection data perfect? No! We have problems with testing in this country. New infections are measured w/ substantial noise. But acting as if that this data is completely useless is specious as well.
But let's say for a sec you don't believe our results. *Even on its own terms*, the article's conclusion is misleading. The analysis argues no evidence for the efficacy of lockdowns exists. It does not follow that "lockdowns don't work". Absence of evidence != evidence of absence
@lymanstoneky does get one big thing right: stay-at-home orders come as a package with other social distancing measures. It's hard to disentangle the different effects of different measures. Our results should be interpreted as the effects of this bundle of policies as a whole.
Which is why the least convincing part of his analysis is the attempt to disentangle the effects of the different measures. We're going to need some fancy identification for this to be credible, & simply regressing lagged deaths on different interventions is not going to cut it.
What we definitely need to understand going forward is which policies are effective and which ones can be relaxed. Given the patchwork of government responses in this country we are likely to get much more evidence on this sooner rather than later.
/fin
/fin
