Read on Twitter
The official coronavirus api has changed the way it reports LFD test (rapid test) results, and this presents a unique opportunity to measure how well these tests have been performing. (Thread 1/n) https://coronavirus.data.gov.uk/details/whats-new
Some LFD tests are retested with PCR tests which generates three categories of LFD result: PCR-retest-positive, PCR-retest-negative and no-PCR-retest. Up to yesterday, the last two of these were added together. As of today, PCR-retest-negative is discarded. (2/n)
By subtracting yesterday's numbers from today's, we can work out all three numbers for all past days. (This is a one-off: as it stands we won't be able to see PCR-retest-negative in future. It's something that has been requested.) (3/n) https://docs.google.com/spreadsheets/d/1yItIPgFTItAM29BZ6D8unah53EkOWMkiCW-T5TcGePI/edit#gid=0
From these we can work out some really useful information, one of which is the chance that when you get a positive lateral flow test result, you have a genuine case of Covid-19. This is known in the jargon as the positive predictive value, or PPV for short. (4/n)
The PPV depends on the properties of the test and also the disease prevalence. There has been some concern that at a time of relatively low disease prevalence, the false positives might outnumber the true positives and most LFD positive results would be wrong. (5/n)
However it doesn't look like that's happening. In the four weeks up to 4 April, the PPV is 82%, meaning that there is an 82% chance that your LFD positive result is real. (That's conservatively assuming that there are no PCR false negatives.) (6/n)
Note that the overall PPV is higher at 88.9%, but that's because it includes tests from an earlier era of high disease prevalence. Note also that we're excluding those LFD tests that didn't get a PCR retest. If these are unrepresentative then this could change the answer. (7/n)
What else can we work out from this? We can have a stab at estimating the false positive rate (FPR): the chance that an LFD test on an uninfected individual gives a positive result. A recent official estimate is 0.03% from the DHSC (8/n)
https://www.gov.uk/government/publications/lateral-flow-device-specificity-in-phase-4-post-marketing-surveillance
https://www.gov.uk/government/publications/lateral-flow-device-specificity-in-phase-4-post-marketing-surveillance
though I have wondered whether it may possibly be lower ( http://sonorouschocolate.com/covid19/index.php?title=LFD_specificity_estimate). From this api data we can estimate the FPR over the last four weeks at 0.021%, based on the assumptions: (9/n)
(i) that the LFD tests that were submitted for PCR-retesting are representative, (ii) that there are no PCR false negatives (if there are PCR false negatives then the FPR could be lower still). (10/n)
Something interesting: it's clear that the FPR has been falling since the tests came in in October last year. Looking down column M, we see much higher numbers in the past. (11/n) https://docs.google.com/spreadsheets/d/1yItIPgFTItAM29BZ6D8unah53EkOWMkiCW-T5TcGePI/edit#gid=0
One point that I think is worth mentioning: how can we work out the FPR of a test when it is so low? If you are testing a population that includes a lot of true positives then you quickly run up against the problem that the uncertainty over PCR false negatives dominates, (12/n)
and you won't be able to get a good estimate of the FPR. In the end, the best estimates for a low FPR come from situations where you test a huge number of people you don't expect to be positive, and then you know the FPR can't be higher than the actual positive rate. (13/n)
If you can further refine this further by PCR-retesting the (relatively small number of) positives then you should get a really good bound on the FPR. This is more-or-less what we've been doing with the mass screening programme of secondary schools. (14/n)
In other words, whatever you think about the merits of the mass screening programme, it has the side effect of providing a unique opportunity for measuring the FPR of the LFDs. I suspect we still may not have drilled down to the true FPR, but we're starting to see (15/n)
some very low estimates like 0.02% now. (16/n)
