The FDA has approved the first antibody test for COVID-19, from Cellex. It theoretically tells you if you've had it & are, as far as we know, immune for some time.

Sensitivity is 93.8%, specificity 95.6%. Sounds great, right?

Well, sort of. (1/6)

If only a small % have actually had COVID-19 (our best guess now) a "positive" antibody test isn't that likely to mean you're immune.

If only 4.5% of U.S. has had COVID-19, + test only means ~50% chance you really had it. With lots of uninfected, lots of false +s. (2/6)

If 10% were truly infected, a positive Cellex antibody test has a 70% chance of being right. If 30% were infected, a positive test is right 90% of the time. This happens bc when more people were sick, true positives overwhelm false positives. But that's not our situation. (3/6)

So this is test may not be that useful for saying "Zach, you are immune; Jen, you aren't." It might be wrong as often as it's right. Mistakenly telling someone they're immune & clear to return to society...you can see the problem if we do that on a large scale. (4/6)

There are simple equations to correct for this on a population level. So this test is still *very* useful for helping us figure out what % of people have been sick in different areas. And it's the best we've got; deploy it! But realize what it will & won't reliably tell us. (5/6)

Running the test 2x & only telling someone they're immune if they get 2 +s *might* help reduce false +s, depending on the error source. If it's anything systemic - say it's detecting antibodies from a similar virus that don't grant immunity to COVID-19 - no good. @KevinMalogna

Bonus tweet 1: A positive test would be more likely to mean you're truly immune if you're in a high-risk group - healthcare worker, had COVID symptoms, family member had COVID - bc prevalence of infection in these subgroups is higher. So test may be more useful for these folks!

Bonus tweet 2: We might be able to reduce false +s/increase the chance a + test is right by using this test as a screener and another slightly different antibody test to confirm, if they wouldn't both show false + for the same systemic reason like antibodies to a similar virus.

If you'd like to nerd out on this stuff further, here are more details on the various statistics we're talking about and why this test may not do some of the things we need when the prevalence is low.

…https://nflinjuryanalyticscom.files.wordpress.com/2020/04/diagnostic_testing_characteristics.pdf

This is starting to make the rounds so while you're here, stay at home and wash your hands so we can save a few hundred thousand lives and my favorite bar and get sports back and stuff. And give whatever you can ($$$, time) to those with less. That'd be rad. Thanks.