New paper on hydroxychloroquine for Covid-19:

"No evidence of rapid antiviral clearance or clinical benefit with the combination of hydroxychloroquine & azithromycin in patients with severe COVID-19 infection", by Molina et al. at Saint Louis Hospital (Paris )

[Thread]

Let's have a look.

To read the paper, just click here:
https://www.sciencedirect.com/science/article/pii/S0399077X20300858

First things first:

This is a journal pre-proof, not a preprint. What does that mean? Well, simply that the paper has been peer-reviewed and accepted for publication in the journal, but it's not yet a definitive version: it still needs copyediting and typesetting.

Second, there are no obvious shenanigans here. For instance, none of the co-authors is a member of the editorial board of the journal.

Still, the manuscript was received on 28 March and available online only 2 days later: that's unusually quick.

What about potential conflicts of interests?

That's a good but thorny question. The authors explicitly state that they have none, and there is no reason to doubt their disclosure.

A quick search on the Transparency in Healthcare database shows that several authors have links with Gilead (remdesevir) or, to a lower extent, Abbvie (kaletra), but nothing out of the ordinary: clinician-scientists have to work with pharmaceutical companies to develop treatments

Now, let's discuss the study.

It is an observational cohort (i.e. not a trial), with data collected prospectively in a single group of 11 consecutive patients who were hospitalized in the department of infectious diseases.

This means that there is no comparison group here.

What about the treatment regimen?

Patients received hydroxychloroquine (600 mg/d for 10 days) and azithromycin (500 mg at day 1, and then 250 mg/d from days 2 to day 5). This dosing regimen is similar to the one used by Gautret et al. ( http://shorturl.at/xMRS3 )

Who were these 11 patients?

- Sex: 7 men, 4 women
- Age: 58.7 years (range: 20-77)
- Comorbidities: obesity (2), cancer (3), hematological malignancies (2), HIV (1)
- Symptoms at baseline: 10/11 had fever and received nasal oxygen therapy

Clinical outcomes (5-day follow-up):

- 1 patient died
- 2 patients were transferred to the ICU
- 1 patient had QT interval prolongation (from 405ms to 460-470ms) justifying treatment discontinuation

Not brilliant...

Biological outcomes:

- Nasopharyngeal swabs were done in 10/11 patients, since 1 patient died.
- At day 5-6 after treatment initiation, 8/10 patients still had SARS-CoV2 positive PCR assays (and 2/10 tested negative)

What does that mean?

In a nutshell, it means that for 8 out of 10 patients the virus was still detectable in their nasal secretions.

For those of you who won't know how these tests are done, the @JAMA_current has done a remarkable illustration.

https://jamanetwork.com/journals/jama/fullarticle/2764238

What is the authors conclusion?

For them, data from these 10 patients "stand in contrast with those reported by Gautret et al.", who found a 70% viral clearance after 6 days among patients treated with hydroxychloroquine (up to 100% among those who also received azithromycin)

Their findings cast serious doubts on the results reported by Gautret et al. and highlights the contrast between the in-vitro antiviral activity of hydroxychloroquine and its clinical reality in real-life patients.

So... how should we (as readers) interpret this paper?
With caution, as always.

First, because it is a single-arm study: by design, it is impossible to know how the disease would have progressed among untreated patients. In this hypothetical control group, maybe 10/10 patients would have tested positive after 5 days. Or maybe 3 would have died instead of 1.

Second, because it is an *observational* study. Even if the authors had been able to compare their results to a control group, the absence of randomisation would have been problematic.

The question that we really want an answer for is: "Had these 11 patients not been treated, would they have had different outcomes?" And since it is not possible to travel back in time, the only we can approximate that is by randomly allocating patients to treatment/no-treatment.

Maybe I lost you just now.

If the notion of "randomisation" is not something that you feel comfortable or familiar with, have a look at this fantastic video from @statsepi https://twitter.com/statsepi/status/1243526933146742784?s=20

Briefly, this means that any claim about the (lack of) efficacy of the treatment based on this study is misleading and unfounded (reminder: the absence of evidence of efficacy is not the same as the evidence of an absence of efficacy)

Third problem: this is a *really* small sample of patients (n=11).

Statistically speaking, it increases the risk of "false negative", namely the probability that we fail to see an association that truly exists.

Of course, taking about type 2 error in a single-arm study is a bit out of scope. But statistical uncertainty does play a role.

What if instead of including 11 patients the authors had included 100 or even 1000 patients? What if we could magically replicate this study x10,000?

If we did such a simulation, assuming that the 11 patients from this study are a random sample of severe Covid-19 patients, the 2/10 viral clearance rate observed by Molina et al. would actually sit anywhere between 12.8 and 29.7 in half of the studies (50% confidence interval).

That’s pretty wide, and it tells you that the sample size simply creates too much statistical uncertainty to make precise inferences. However, it also tells you that there is also very little chance that the viral clearance rate could be 70% in this setting.

Ok, so what? Should we trust this paper?

The article should be considered as a case series report, nothing more. The sample size, the mono-centre and single-arm design, and the lack of detailed data in the paper (e.g. no table, no day-by-day PCR results) are major limitations.

That being said, the description of these 11 cases paints a pretty bleak picture.

Within only 5 days, 1 patient died, 2 were transferred to the ICU, and 1 had to discontinue hydroxychloroquine because of side effects.

At the very least, this proves that the combination of hydroxychloroquine and azithromycin is not the “miracle cure” that the media and some scientists have depicted.

And for once, the authors have a reasonable concluding sentence: “Ongoing randomized clinical trials with hydroxychloroquine should provide a definitive answer regarding the alleged efficacy of this combination and will assess its safety”.

Did I miss anything important @statsepi @tmorris_mrc @jd_wilko @DgCostagliola @profnpicard (and others) ?

And here comes the link to @medmalinf https://twitter.com/medmalinf/status/1244985018109366273?s=20