Read on Twitter
Recently, John Ioannidis, of "Most Published Research Findings Are False" fame, published a commentary piece on COVID-19 and global action
I thought it would be good to do a bit of peer-review on twitter 1/n
I thought it would be good to do a bit of peer-review on twitter 1/n
2/n You can find the paper here - it is a classic commentary piece, which means it is mostly the perspective of the author: https://onlinelibrary.wiley.com/doi/10.1111/eci.13423
3/n Given the author's very strong public stance since early March, it is perhaps unsurprising that the main message of the piece appears to be that COVID-19 is not very bad but government actions are
4/n Very broadly, the piece says:
- the death toll from COVID-19 is probably low globally
- government action causes lots of deaths
- therefore, government action should be minimized except for protecting institutionalized elderly people
- the death toll from COVID-19 is probably low globally
- government action causes lots of deaths
- therefore, government action should be minimized except for protecting institutionalized elderly people
5/n Now, as I've said before (many times), the cost-benefit of government regulation is an IMPORTANT issue that we really need to discuss more, so I applaud Prof Ioannidis for looking into the question
6/n Unfortunately, the piece makes a large number of errors that makes the conclusion - that we should "learn to live with COVID-19" - somewhat problematic
7/n One problem is the attribution of deaths. The author raises the point several times that COVID-19 deaths may not be caused BY the disease, but occur WITH COVID-19 and be caused by underlying comorbidities
8/n The reference to this - and indeed the most cited study in this commentary piece - is a short commentary piece written by Ioannidis back in April
9/n And if you follow the references back down the chain, what we find is that the author is using an investigation done in March in Italy that found that the majority of people who died had some comorbidity to support this argument
But it doesn't, really
But it doesn't, really
10/n Conversely, there is fairly good evidence that in the majority of cases COVID-19 was indeed the primary cause of death, despite comorbidities, and that the biggest issue in many places may be UNDERCOUNTING of deaths https://www.bmj.com/content/370/bmj.m2859
11/n The author then goes on to argue that while there have been excess deaths in 2020, these are most likely due to GOVERNMENT ACTION rather than the pandemic
12/n Now, while this is purely speculative anyway, I thought it worth noting that it is incorrect to argue that all disruptions to (say) TB programs are due to government measures. It is likely that much of this disruption would've occurred anyway due to, well, the pandemic
13/n Therefore, the argument made by the author that there is harm caused by "recurrent lockdowns and other draconian measures" is not really substantiated by the evidence at hand
Not great, that
Not great, that
14/n (I'm not saying that there is NO harm from these measures, just that the cost-benefit of government action is quite clearly more complex than the logic presented here)
15/n Moving on, the author basically makes the same argument about comorbidities as before, but in the framework of Years of Life Lost. Similarly, the basic argument is that people who died of COVID-19 were already very sick
16/n The author then argues (again, citing himself liberally in references 3 and 41) that many deaths due to COVID-19 were "avoidable"
None of this is really well-supported as far as I can tell
None of this is really well-supported as far as I can tell
17/n For example, the claim that "many deaths" in Lombardy were due to nosocomial (acquired though healthcare) infections is referenced back to the same April commentary as before, which itself actually suggests that hospital overcrowding was to blame
18/n It's also worth noting that the idea that nosocomial infections are AVOIDABLE in a pandemic is perhaps not entirely reasonable, given the burden that it places on the healthcare system more broadly
19/n The author then looks at the infection-fatality rate of COVID-19, again liberally citing his (as-yet-unpublished) review on the topic
20/n Rather than delving into the reasons why these numbers are likely inaccurate, I'll just link to my previous threads looking at the preprint version of this paper https://twitter.com/GidMK/status/1283232023402868737?s=20
21/n Of note, however, the author makes a classic statistical error in dichotomizing age into two buckets
We now know that the risk from age is CONTINUOUS, and so looking at the arbitrary cutoff of 70 years will lead to an erroneous result
We now know that the risk from age is CONTINUOUS, and so looking at the arbitrary cutoff of 70 years will lead to an erroneous result
22/n The author also makes the interesting claim that the WHO has estimated an infection rate of 10% of the world, referencing a recent news report
23/n However, as @zorinaq has pointed out, this is simply bad reporting, with the WHO arguing that LESS THAN 10% of the world is likely infected
This is, perhaps, an issue with relying on headlines in a scientific paper https://twitter.com/zorinaq/status/1314160523613593600?s=20
This is, perhaps, an issue with relying on headlines in a scientific paper https://twitter.com/zorinaq/status/1314160523613593600?s=20
24/n There are, sadly, further errors in the paper. The author repeats a common myth, that t-cells mean that the threshold for herd immunity is much lower than usually estimated
25/n Interestingly, Ioannidis here cites @profshanecrotty, who has a great thread on why this interpretation is incorrect https://twitter.com/profshanecrotty/status/1313580981341712386?s=20
26/n Now, I actually think that all of these errors are a real shame, because the final parts of the commentary are both interesting and worth considering
27/n The basic idea is that, rather than thinking about the next few months, we should be designing policies based on what will likely happen over the next 5-10 YEARS
This is a good point, and not made often enough!
This is a good point, and not made often enough!
28/n Short-term lockdowns made sense in the early days of the pandemic, but given the months since it is perfectly reasonable to suggest that there may be better ways to manage the disease moving forward
29/n For example, from a GLOBAL perspective, I think that there may certainly be places in which heavy long-term restrictions do not make that much sense
30/n Nigeria, for example, has a median age of 18 years. Less than 10% of the population is over 55. Given how much lower the risk is for younger people, harsher restrictions may not make as much sense https://www.medrxiv.org/content/10.1101/2020.07.23.20160895v6
31/n Conversely, the U.S. has a much older population, on average, and thus is at far greater risk from COVID-19 generally
32/n That being said, this piece appears to use inappropriate evidence, misleading comparisons, and generally underestimate the risk/impact of COVID-19, which makes it problematic as a resource
33/n As one final example, the author makes this statement in the conclusion, that measures taken to halt the pandemic are essentially destroying the world
The reference? His own opinion piece in the Boston Reader
The reference? His own opinion piece in the Boston Reader
34/n I just checked, and Ioannidis cites himself 8 times in this paper, with 4 of those references being media or commentary articles
35/n Oh also, on another note, given the relatively few meta-studies on COVID-19 IFR, it appears that the authors that Ioannidis here describes as "inexperienced" and "overtly wrong" are me and @LeaMerone
Academic civility!
Academic civility!
36/n I should also note - some people have pointed out that the actual numbers in this review are almost certainly inaccurate (e.g. 1.5 mil COVID-19 deaths over 5 years is probably impossible) but I have not focused on the numbers themselves...
37/n ...thing is, the numbers are largely a result of the assumptions. If you extrapolate out an assumed incorrectly low death rate then you'll find the numbers are low
Therefore, I focused on the assumptions, because the numbers themselves are just a consequence of these
Therefore, I focused on the assumptions, because the numbers themselves are just a consequence of these
38/n That being said, it is absolutely worth pointing out that some of these figures that the paper has postulated as reasonable are extraordinarily unlikely and show how the errors have caused a misleading result (remember, this is globally over 5 years!)
