Read on Twitter
So, after my @slate article on the epidemic of armchair epidemiology, I really didn’t want to spend more time discussing another amateur’s post, but I think there’s a useful lesson here about what qualifies as science. *rolls up sleeves* THREAD https://slate.com/technology/2020/03/armchair-epidemiology-coronavirus.html
2/ First, some context. In my article, I highlighted a medium post by @abestanway. Abe is an industry data scientist and here’s his post
3/ He used public health data on influenza-like symptoms in NYC ERs to conclude that the Corona neighborhood in Queens is the epicenter of the COVID outbreak. I pointed out a flaw in his analysis, which an epidemiologist agreed with. Read my criticism here.
4/ Now, newly released data showing a high number of COVID cases in Corona, Queens, consistent with Abe’s guess. Abe sent me the following tweet, asking for a “retraction” [sic]
![4/ Now, newly released data showing a high number of COVID cases in Corona, Queens, consistent with Abe’s guess. Abe sent me the following tweet, asking for a “retraction” [sic]](https://pbs.twimg.com/media/EUrgAkAUwAEY7kj.jpg "4/ Now, newly released data showing a high number of COVID cases in Corona, Queens, consistent with Abe’s guess. Abe sent me the following tweet, asking for a “retraction” [sic]")
5/ As I stated in the piece, I fully acknowledged that Abe’s guess may turn out to be correct:
6/ So my point wasn’t whether he was right or wrong, or whether any of the armchair epidemiologists are right or wrong. It’s just that what they are doing isn’t science, or really even close. Let me explain.
7/ Many of these amateurs portray themselves as challenging experts who can’t or won’t see the truth in the data. They tout credentials as data scientists to justify their analyses. But the issue was never really one of credentials, or even expertise. It’s one of process.
8/ The reason these pseudoscientific posts are dangerous is because they have many of the trappings of science yet little of its rigorousness. They tout fancy credentials. They have numbers and graphs. They use jargon freely. BUT…
9/ Armchair epidemiologists seem to forget that you can’t claim something as scientific knowledge if you don’t arrive at it via the scientific method.
10/ Science is the act of forming hypotheses premised in previous knowledge. It’s testing them via experiments and/or analyses. It’s rigorously ruling out alternative explanations. These are the hallmarks of actual scientific thinking—not credentials, numbers, graphs, and jargon.
11/ So how well did Abe adhere to these principles? Let’s go back to my original criticism—that Abe needed to compare this year’s data to previous years’ data. One reason to do that is to rule out alternative explanations. For example…
12/ If Corona has a higher number of uninsured people, they might be more likely to seek care at an ER. This may drive up the apparent rate of flu, but wouldn’t necessarily reflect the actual rate of flu. So Abe apparently failed to consider alternative explanations. But then…
13/ After my article, Abe suggested the many hospitals in the area “likely” accounted for the high flu rates. This confused me. If hospital presence is driving the rate, then shouldn't you control for it? Seems like another sign of an unrigorous analysis. But there's more...
14/ Genuinely trying to understand his objections, I asked Abe over Twitter how NYC collected its data. Were visits coded by the patient’s zip code? That would matter. I was surprised to find he didn’t know.
15/ So by his own admission he was drawing conclusions without fully understanding how the data was collected. From a tweet, though, it seemed he assumed the data weren’t coded by patient zip code. I checked directly with the NYC Dept of Health
16/ It’s very hard to be rigorous when you don’t have the details of the methodology.
17/ Just in case I misunderstood something, I also asked NYC Dept of Health as to whether Abe’s evidence-free speculation that hospital presence “likely” explained the high rates in Corona. Here’s what they said (see #2):
18/ The point is that if it’s easy for me—a non-epidemiologist—to find hole after hole in his scientific reasoning, then maybe he’d be better off staying in his lane. I guess this is difficult for some people to accept:
19/ This pandemic isn't a joke; it's is a matter of life and death, and the public needs accurate scientific information. Half-baked amateur analyses, whether the conclusions turn out to be right or wrong, serve no purpose. /fin
