John Ioannidis, of Most Published Research Findings Are False fame has again updated his preprint on infection-fatality rate for COVID-19

Kudos to him for updating, let& #39;s again look at what& #39;s happened 1/n
3/n You can find my original thread on the paper here: https://twitter.com/GidMK/status/1262956011872280577?s=20">https://twitter.com/GidMK/sta...
4/n So, what& #39;s changed?

At first glance, not much. The median IFR has gone up from 0.26% to 0.27%, based on the inclusion of a bunch more studies
5/n Some of the more inappropriate statements have at last been removed, such as the age-related stuff and the incorrect citation of the case-fatality rate of influenza

Good!
6/n On the downside, we now have this comparison with the death rates from flu, which is also just...weird

If nothing else, the COVID-19 pandemic is STILL GROWING and has ALREADY killed as many people as a & #39;bad& #39; influenza season
7/n Also, many of the issues I& #39;ve raised previously are still there. Many studies have been included that probably aren& #39;t reliable estimates of population seroprevalence are lumped in with very good estimates

That& #39;s...less than ideal
8/n Let me illustrate with an example

This study has been included to calculate IFRs for 4 regions of China - Hubei (not Wuhan), Chonqing, Sichuan, and Guangdong
9/n So the author has taken the percent positive of antibody tests for COVID-19 from each of the regions represented in the study, and used that as a population estimate for the entire region to calculate IFR

But is this reasonable?
10/n The study only tested two groups: healthcare workers and people on dialysis

Now, Ioannidis excludes any testing on healthcare workers, but dialysis patients are...fine?
11/n And these high numbers of seropositive estimates led to inferred IFRs for these four places in China of 0.00%!
12/n If nothing else, the numbers here imply that 99.9996% of all infections in Chongqing were asymptomatic (500 official cases, widespread testing, but seropositivity of 3.8% in the study implying 12 million & #39;true& #39; cases)

Is this plausible???
13/n There are also some numbers in this revised paper that are wrong

This figure should read 44%, not 47%
14/n Moreover, in the example highlighted above, the IFR calculated is for Brooklyn, but this was only true for a tiny subset of 240 patients in this 28,523 patient study. The IFR calculation should& #39;ve been for the whole of NYC, not just Brooklyn!
15/n There are also some worrying inconsistencies in how Ioannidis has split up studies that sampled multiple places within countries
16/n For example, the ENE-COVID and Brazilian studies, which sampled entire countries by region, are only summed up as a single value
17/n On the other hand, several studies that sampled multiple regions (but found MUCH lower IFRs) in other places are split up by area

I cannot see any explanation for this in the paper
18/n On top of this, we& #39;ve got another problem - collinearity

The basic issue is that you shouldn& #39;t lump multiple samples of the same group of people together into one study
19/n But now, in the study we have Wuhan (A), Wuhan (B), and Hubei (not Wuhan)

It& #39;s very poor statistical practice to lump all these estimates together like this
20/n Similarly, we have two estimates from Spain. One is the ENE-COVID study, a rigorous randomized seroprevalence study that is the envy of the world

The other is a non-random sample of pregnant women at one place in Barcelona

These are given EQUAL WEIGHTS in the analysis
21/n The Spain example is even more of a problem because the ENE-COVID (the rigorous study) implies an IFR in Barcelona of ~1%

The survey of pregnant women implies ~.5%

Guess which one is used?
22/n Now, all of this collinearity is particularly troubling for that 0.27% estimate that I mentioned way back at the start of the thread
23/n If we get average the collinear results - where we& #39;ve included the same study or the same sample multiple times - the median jumps immediately to 0.35%

That& #39;s quite a bit higher!
24/n But there are more corrections to be made. In several places, the IFR that is in this paper does not match the IFR calculated by the study authors
25/n For example, Geneva. The original authors calculated an IFR of 0.64%, but this is downgraded to 0.45% in the paper
26/n And this is not the only example. Another study tested over three weeks and found seroprevalence of 3.85%, then 8.36%, then 1.46%. Overall 3.53%

The 8.36% figure is used, giving 5x more infections than the study itself found, and the lowest IFR possible
27/n Taking all this into account, let& #39;s look at the IFRs for only those studies using representative population samples that were correctly calculated
28/n Here& #39;s the revised table. The lowest IFR is, again, Ioannidis& #39; own study, at 0.18%. Nearly half of the estimates are above 1%, and they range all the way up to 1.63% (!)
29/n Somehow, for the third time running, there are innumerable decisions made in the paper that seem to only ever push down the IFR, rather than produce the best estimate
30/n As I& #39;ve outlined, there are also a number of simple errors that make this very problematic as an estimate of the IFR (or the IFR range) for COVID-19
31/n All that being said, the discussion is now MUCH better, and really engages with some of the things I (and others) discussed in previous threads. Too much to go over here, but well worth a read
32/n Ioannidis has also now included some of the government-conducted studies in the paper, which is good to see
33/n All in all, some definite improvements, but a lot of things still in the paper that are really hard to reconcile with best practice
34/n The one thing I would point out - this from earlier in the thread is a classic example of moving the goalposts. The influenza comparison was clearly wrong, so now we have another comparison which is bad but slightly less wrong https://twitter.com/GidMK/status/1283232032085032961?s=20">https://twitter.com/GidMK/sta...
35/n imo much better practice would be to acknowledge that COVID-19 is probably substantially more lethal than influenza, but that quantifying this difference is somewhat challenging
36/n Also, another statement that is incorrect and has remained in each version - that disadvantaged populations/settings are uncommon exceptions in the global landscape

This remains simply untrue
38/n Another addition, this thread goes through some of the headaches with the paper that have remained through every version

TL:DR - it& #39;s not systematic! https://twitter.com/AVG_Joseph96/status/1283236273558294528?s=20">https://twitter.com/AVG_Josep...
You can follow @GidMK.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: