I’m starting to see discussion about serological testing for COVID19 antibodies as one of the ways that we will be able to safely allow people to start to interact more. To have informed public discussions about this, people will need to understand how test “accuracy” works.

Note 1: There’s a whole separate issue here about how immunity may or may not work and whether the presence of antibodies indicates long(ish)-term immunity. That is not the topic here. I recommend this article. https://twitter.com/epiellie/status/1249867382866284544

Note 2: I am not a virologist, an infectious disease doc, nor an epidemiologist. Although I have a background in math & took a lot of epi courses, my PhD was in human factors engineering, focusing on interface design for health information.

Since my expertise is in how to present health information (especially information involving numbers and probabilities) clearly to support evidence-informed decisions that align with your values, I feel like I’m in my wheelhouse here in this thread.

Note 3: This thread presents Bayes’ Theorem in a diagram. I am at ease with Bayes’ Theorem expressed in math (see: aforementioned background in math & engineering, I’ve taught linear algebra, etc.) If you are also comfortable with the math version, this thread is not for you.

This tree diagram method has been shown to be more understandable than the math formula for many. I don’t really care if people know the formula as long as they understand the concept. https://twitter.com/hwitteman/status/1040643483751641088

For the purposes of this thread, let’s talk about the results of serological testing as:
Positive = “you have antibodies! ” (and you may therefore be considered immune)
Negative = “you don’t have antibodies ” (and you may therefore be vulnerable)

The idea of this approach is that people who are certified as immune can then go forward, interact with others, work on the front lines more safely, etc. [pick me!!]

There are three things that matter here: 1) the sensitivity of the test, 2) the specificity of the test, and 3) what percentage of the population actually has antibodies?

1) Sensitivity is a measure of how well a test picks up true positives. If you actually have antibodies, a more sensitive test will more likely detect them.

2) Specificity is how well a test picks up true negatives. So if you *don’t* actually have antibodies, a more specific test will more likely correctly report that.

3) The 3rd thing that matters here is what % of the population actually has antibodies? If lots of people had COVID19 (with or without symptoms) & recovered, lots of people will have antibodies. If fewer people had COVID19 & recovered, fewer people will actually have antibodies.

So, scenario A: let’s say 10% of the population has had COVID19 (with or without symptoms) and recovered, and let’s say we have a test to detect antibodies that is 94% sensitive and 91% specific.

Of the 100 people who actually have antibodies, 94 of them (94%) will get a “you have antibodies! ” result (true positive) while 6 of them (6%) will get a “you don’t have antibodies ” result (false negative.)

Of the 900 people who don’t actually have antibodies, 819 of them (91%) will get a “you don’t have antibodies ” result (true negative) while 81 of them (9%) will get a “you have antibodies! ” result (false positive.)

This means that out of 1000 people, of whom 100 actually have antibodies, a total of 175 people will get a “you have antibodies!” result but it will only be correct for 94/175 = 54% of them.

Scenario B: what if we use that same test but 50% of the population actually has antibodies? Then a “you have antibodies!” result will be correct for 470/515=91% of the people who get that result.

Scenario C: what if it’s still only 10% of people but we have a more specific test? Say 87% sensitivity and 98% specificity? Then a “you have antibodies!” result will be correct for 87/105=83% of the people who get that result.

Scenario D: what if everything’s the same as in Scenario A but we require *two* positive results before being confident that someone has antibodies. In that case, a “you have antibodies!” result will be correct for 88/95=93% of the people who get that result.

So as you start to see discussions of serological testing, this is why you will see scientific discussions re: sensitivity, specificity, how many people actually have antibodies, and whether tests would be done once, twice, or more.

Epi/stats/ID colleagues: please let me know if I’ve misrepresented or oversimplified anything here.

P.S. If you liked this thread, you may also be interested in work my team has done to explain how social/physical distancing works: https://twitter.com/wittemanlab/status/1248158283757748224