Read on Twitter
In @TheLancet - study today compares outcomes for Covid patients who got usual care or chloroquine/hydroxychloroquine with or without a macrolide antibiotic (e.g. azithromycin). Main finding - these treatments appear to worsen mortality a lot, but ....1/n https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31180-6/fulltext
It's observational - statistical adjusters after the fact try to make people getting and not getting unproven rx similar enough to compare outcomes. Usually the people who 'look sicker' are more likely to get an unproven treatment (called 'confounding by indication'). 2/n
This 'confounding by indication' can be clearly seen in paper. Patients (Table 2) who got the unproven rx were older and heavier. these features were associated (Table 1) with greater risk of death (red line, blue line, respectively. 3/n
These authors, different from the recent @NEJM study that showed zero benefit (but no real harm) from these rx's, used multi-variable time to event ('cox models') to adjust for differences, but repeated their analysis using the @NEJM study method of 'propensity scores'. 4/n
I went over that @NEJM study here 5/n: https://twitter.com/peterbachmd/status/1258777305251557377?s=20
Are the patients after statistical adjustments now similar enough to support the conclusions (they also found incr. rates of cardiac arrhythmias associated with these unproven rx's). How many 'factors' might they have missed of mis-modeled? Could this still be confounding? 6/n
The authors have an answer provided in a 'tipping point analysis'. How big a factor would they have had to miss to make the negative effects of these treatments go away (p. 20 appendix). Big is the answer (not to be too technical) 7/n https://www.thelancet.com/cms/10.1016/S0140-6736(20)31180-6/attachment/84423d57-4cf8-41d0-99ca-0e921f2c80ce/mmc1.pdf
Here's an example from this graph. If some patient factor were present in half of people who get the unproven treatment, and increased their odds (their 'hazard') of death by 50%, the outcomes would still be worse but no longer statistically significant. Here is their chart 8/n
In a well understood clinical scenario the tipping point analysis would be awfully convincing that we are seeing a signal of true harms. The challenge is that in Covid the negative impact of measured factors like age are quite large, an unmeasured one could be too. 9/n
These rx's shld not be used outside a controlled trial. This analysis involved 100,000 subjects. A mere 2% going into RCT's of these rx's would have provided much more certain insights. I discussed this with @IChotiner @NewYorker last week https://www.newyorker.com/news/q-and-a/the-danger-of-rushing-through-clinical-trials-during-the-coronavirus-pandemic 10/10
