Read on Twitter
THREAD **IMPORTANT** (IMO)
SO MUCH Continued CONFUSION about Rapid Antigen Tests - Now about Asymptomatics.
A New @CDC_gov @CDCMMWR study concludes Ag tests didn't work in Asymptomatic ppl. Simply put, It's wrong
Here I got in depth on Rapid Tests & how to evaluate them
1/x
SO MUCH Continued CONFUSION about Rapid Antigen Tests - Now about Asymptomatics.
A New @CDC_gov @CDCMMWR study concludes Ag tests didn't work in Asymptomatic ppl. Simply put, It's wrong
Here I got in depth on Rapid Tests & how to evaluate them
1/x
The study found rapid Ag tests missed 40% of PCR +ve Asymptomatics.
BUT... FAILED to state that 100% of the misses were in ppl with NO CULTURABLE VIRUS (plus ALL had Ct values >30)
Rapid tests CAUGHT 100% of the infectious ppl!
https://www.cdc.gov/mmwr/volumes/69/wr/mm695152a3.htm?s_cid=mm695152a3_w
2/x
BUT... FAILED to state that 100% of the misses were in ppl with NO CULTURABLE VIRUS (plus ALL had Ct values >30)
Rapid tests CAUGHT 100% of the infectious ppl!
https://www.cdc.gov/mmwr/volumes/69/wr/mm695152a3.htm?s_cid=mm695152a3_w
2/x
That's right!
In the very same study that concludes rapid Ag tests do not work in asymptomatic people, the authors failed to state the rapid Ag tests SUCCESSFULLY CAUGHT 100% of ASYMPTOMATIC PEOPLE who had likely contagious virus.
3/x
In the very same study that concludes rapid Ag tests do not work in asymptomatic people, the authors failed to state the rapid Ag tests SUCCESSFULLY CAUGHT 100% of ASYMPTOMATIC PEOPLE who had likely contagious virus.
3/x
In this thread I'll try to place that study in context, as well as discuss another highly publicized study from Liverpool,UK which I believe is hitting up against similar issues-in a less explicit way-causing an unnecessary loss in confidence in a rapid Test.
4/x
4/x
First, Some definitions:
Definitions (i):
Ct=Cycle threshold
High Ct (40) = VERY LOW
Low Ct (10) = VERY HIGH
Each 10 Ct's DOWN = 1000x Higher virus
Eg. If Ct 40=100 viruses, Ct 30 = 100K, 20 = 100MM, 10 = 100BB
5/30
Definitions (i):
Ct=Cycle threshold
High Ct (40) = VERY LOW
Low Ct (10) = VERY HIGH
Each 10 Ct's DOWN = 1000x Higher virus
Eg. If Ct 40=100 viruses, Ct 30 = 100K, 20 = 100MM, 10 = 100BB
5/30
Definitions (ii)
Ag = Antigen: a piece of virus. An Ag test looks for physical part of virus-like looking for a person's eyes
RNA = genetic code of a virus. A PCR COVID19 test looks for virus RNA, like a detective looks for DNA of a human.
6/30
Ag = Antigen: a piece of virus. An Ag test looks for physical part of virus-like looking for a person's eyes
RNA = genetic code of a virus. A PCR COVID19 test looks for virus RNA, like a detective looks for DNA of a human.
6/30
Definitions (iii)
Culture = lab test to determine if there is "live" (likely contagious) virus in the sample
"+ve" & "-ve" simply mean "positive" & "negative"
So, a sample that is "culture +ve" means it has "live" virus detectable in it.
7/30
Culture = lab test to determine if there is "live" (likely contagious) virus in the sample
"+ve" & "-ve" simply mean "positive" & "negative"
So, a sample that is "culture +ve" means it has "live" virus detectable in it.
7/30
Definitions (iv)
A sample that is "culture -ve" means there's no discernable live virus in sample.
If a Sample is PCR +ve & Culture -ve it is very likely that the PCR is detecting old remnant RNA. I explain more below.
8/30
A sample that is "culture -ve" means there's no discernable live virus in sample.
If a Sample is PCR +ve & Culture -ve it is very likely that the PCR is detecting old remnant RNA. I explain more below.
8/30
Back to thread:
Here I show the actual PCR +ve Asymptomatic data from the paper.
As is PLAINLY evident, the rapid Ag tests detected 100% of Culture +ve PCR +ve specimens.
*Culture +ve on decent assay should be MINIMUM bar to consider ppl likely infectious
9/30
Here I show the actual PCR +ve Asymptomatic data from the paper.
As is PLAINLY evident, the rapid Ag tests detected 100% of Culture +ve PCR +ve specimens.
*Culture +ve on decent assay should be MINIMUM bar to consider ppl likely infectious
9/30
So HOW does this go so wrong?
How does a study get so badly distorted??
Simple, We must always DEFINE WHAT IS THE GOAL OF THE ASYMPTOMATIC TESTING!
In case of asymptomatic screening of ppl with no reason to think their infectious, it is to find CONTAGIOUS people.
10/30
How does a study get so badly distorted??
Simple, We must always DEFINE WHAT IS THE GOAL OF THE ASYMPTOMATIC TESTING!
In case of asymptomatic screening of ppl with no reason to think their infectious, it is to find CONTAGIOUS people.
10/30
The goal of Asymptomatic screening is NOT to medically diagnose ppl as having remnant non-infectious virus RNA.
NO! Leave that for medical testing.
Ppl w remnant RNA usually do NOT need to be isolated
**We MUST recognize PCR STAYS POSITIVE LONG AFTER INFECTIOUS**
11/30
NO! Leave that for medical testing.
Ppl w remnant RNA usually do NOT need to be isolated
**We MUST recognize PCR STAYS POSITIVE LONG AFTER INFECTIOUS**
11/30
If NO symptoms, you're testing for contagiousness.
We must stop thinking about tackling this pandemic w medicine but rather w PUBLIC HEALTH tools.
We defined the reasons for testing in @ScienceMagazine here w/ @K_G_Andersen
https://science.sciencemag.org/content/early/2020/12/18/science.abe9187
12/30
We must stop thinking about tackling this pandemic w medicine but rather w PUBLIC HEALTH tools.
We defined the reasons for testing in @ScienceMagazine here w/ @K_G_Andersen
https://science.sciencemag.org/content/early/2020/12/18/science.abe9187
12/30
The @CDCMMWR study linked above failed to define why they were testing and thus got confused.
They said the tests missed 60% of the asymptomatics when in fact they caught 100% of the asymptomatic PCR +ves that they intended to detect!
The culture positive ones!
13/30
They said the tests missed 60% of the asymptomatics when in fact they caught 100% of the asymptomatic PCR +ves that they intended to detect!
The culture positive ones!
13/30
***I implore anyone studying rapid Ag tests to first DEFINE WHY YOU'RE TESTING, and from that, choose the metrics that make sense.
This is crucial.
If your unsure what your goals are, don't do the study or find someone to help. I'll help....
14/30
This is crucial.
If your unsure what your goals are, don't do the study or find someone to help. I'll help....
14/30
There've been many many rapid Ag test studies by ppl who haven't defined their goal. These unsurprisingly tend to ultimately come to a conlclusion that Rapid Ag tests don't work.
15/30
15/30
Meanwhile, studies by experts in this field have largely concluded these tests work well
The study that instigated this thread shows how WELL these tests work- though the data was interpreted with a slightly bent lens - discussed above.
https://www.cdc.gov/mmwr/volumes/69/wr/mm695152a3.htm
16/30
The study that instigated this thread shows how WELL these tests work- though the data was interpreted with a slightly bent lens - discussed above.
https://www.cdc.gov/mmwr/volumes/69/wr/mm695152a3.htm
16/30
Here are a few tips on interpreting Rapid Ag Test studies
a) Define the goal. If goal is screening contagiousness, then study should put any rapid Ag tests in context of virus culturability OR attempt to measure Rapid Ag test sensitivity stratified on viral RNA load
17/30
a) Define the goal. If goal is screening contagiousness, then study should put any rapid Ag tests in context of virus culturability OR attempt to measure Rapid Ag test sensitivity stratified on viral RNA load
17/30
b) Virus Culture +ve does not necessarily mean someone is infectious.
Rather, it should be considered IMIO as a minimum bar to entry that someone might be.
If not culture +ve, then from a public health lens, the person is most likely NOT a major public health threat
18/30
Rather, it should be considered IMIO as a minimum bar to entry that someone might be.
If not culture +ve, then from a public health lens, the person is most likely NOT a major public health threat
18/30
c) Yes, some virus culture systems are more sensitive than others. Of course. But most will do fine to get at this broad metric of whether to consider a sample to have enough live virus to consider likely contagious and a public health threat.
19/30
19/30
d) If evaluating against PCR Ct values - do NOT assume consistent Cts across labs.
Some can be wildly different from next!
Though much discussion about this - it is oddly frequently ignored when assessing Rapid Ag tests.
20/30
Some can be wildly different from next!
Though much discussion about this - it is oddly frequently ignored when assessing Rapid Ag tests.
20/30
For ex, many studies have found a Ct value of ~30 generally reflects the edge of culturability
Thus Ct 30 is often used as useful Ct cutoff for rapid Ag tests to shoot for
This is OK as a guide, but can be way off
This MMWR Study is a decent example of Ct 30 cutoff
21/30
Thus Ct 30 is often used as useful Ct cutoff for rapid Ag tests to shoot for
This is OK as a guide, but can be way off
This MMWR Study is a decent example of Ct 30 cutoff
21/30
If the PCR test being used to validate rapid test is calibrated with a lower Ct value (for whatever reason) this can make a rapid Ag test look much worse than it is.
I believe a real world example is currently sowing confusion in Liverpool, UK
22/30
I believe a real world example is currently sowing confusion in Liverpool, UK
22/30
In most studies that defined Ct ~30 as useful target for culturability/Rapid Ag tests, median Ct values of Asymptomatics is usually ~28-31:
Here is a mishmash of Studies showing Asymptomatic Cts - note medians are all around 30
23/30
Here is a mishmash of Studies showing Asymptomatic Cts - note medians are all around 30
23/30
In these studies 
, rapid Ag test worked well up to Ct of ~30, or even better.
This is important b/c a very public study in Liverpool is showing outlier results where rapid Ag test looks like it is performing poorly... but I think the Ct distribution may be at play.
24/30
, rapid Ag test worked well up to Ct of ~30, or even better. This is important b/c a very public study in Liverpool is showing outlier results where rapid Ag test looks like it is performing poorly... but I think the Ct distribution may be at play.
24/30
This study in Liverpool, UK is finding that Rapid Ag tests are performing well only up to a Ct value of about 22.
This is way different than other assessments of this same test - which have shown it works to Ct ~30, as expected.
However...
25/30
This is way different than other assessments of this same test - which have shown it works to Ct ~30, as expected.
However...
25/30
Inspection of Ct value distributions in Liverpool suggest shift in PCR Ct values:
The median Ct of Asymptomatic screening is a HUGE OUTLIER w median Ct of only ~21!
This is Exceptionally LOW for random screen of Asymptomatics - as shown in the figure a few tweets up.
26/30
The median Ct of Asymptomatic screening is a HUGE OUTLIER w median Ct of only ~21!
This is Exceptionally LOW for random screen of Asymptomatics - as shown in the figure a few tweets up.
26/30
The Ct result is either from a very rare chance of finding most asymptomatic people right at the peak of their infectivity (unlikely given the narrow time frame for this)
OR
More reasonably, the PCR lab may have a readout shifted down by about ~5-10 Cts.
27/30
OR
More reasonably, the PCR lab may have a readout shifted down by about ~5-10 Cts.
27/30
There is other evidence for this.
The lab in the study claims a Ct value of ~30 equals ~100 viral RNA particles/mL
In MOST studies/labs, a Ct value ~100 RNA particles/mL is usually a Ct value of ~40.
Here is the Liverpool study
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/943187/S0925_Innova_Lateral_Flow_SARS-CoV-2_Antigen_test_accuracy.pdf
28/30
The lab in the study claims a Ct value of ~30 equals ~100 viral RNA particles/mL
In MOST studies/labs, a Ct value ~100 RNA particles/mL is usually a Ct value of ~40.
Here is the Liverpool study
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/943187/S0925_Innova_Lateral_Flow_SARS-CoV-2_Antigen_test_accuracy.pdf
28/30
Thus, in the Liverpool study, it is likely that the rapid Ag test is performing as we expect, Just that the PCR Ct values are shifted down from what we're used to seeing.
The study SHOULD evaluate vs culture to know for sure.
29/30
The study SHOULD evaluate vs culture to know for sure.
29/30
To CONCLUDE - we have to be very very careful when evaluating Rapid Antigen Tests.
We must DEFINE the purpose of the test. Determine the best comparison test for evaluation based on that definition, and only then report out how well the test is working.
30/31
We must DEFINE the purpose of the test. Determine the best comparison test for evaluation based on that definition, and only then report out how well the test is working.
30/31
Anything short of that ends up running into major problems with these tests.
This paper here is one of the best papers in terms of methods to evaluate rapid tests. IMO. https://twitter.com/michaelmina_lab/status/1327720372318441473?s=20
This paper here is one of the best papers in terms of methods to evaluate rapid tests. IMO. https://twitter.com/michaelmina_lab/status/1327720372318441473?s=20
