Read on Twitter
A new randomized trial evaluating the efficacy of hydroxychloroquine has been posted yesterday on @medrxivpreprint. So, what's the deal here? [Thread]
https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1.full.pdf
1/
![A new randomized trial evaluating the efficacy of hydroxychloroquine has been posted yesterday on @medrxivpreprint. So, what's the deal here? [Thread] https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1.full.pdf1/](https://pbs.twimg.com/media/EUb1ctgXgAYTkqV.jpg "A new randomized trial evaluating the efficacy of hydroxychloroquine has been posted yesterday on @medrxivpreprint. So, what's the deal here? [Thread] https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1.full.pdf1/")
https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1.full.pdf
1/
First, let's start by reminding everyone out here that this is a #preprint. It means that the paper has not (yet) undergone the peer-review process that will evaluate its merits and weaknesses.
https://www.medrxiv.org/content/what-unrefereed-preprint
2/
https://www.medrxiv.org/content/what-unrefereed-preprint2/
A quick recap about the design:
- Design: double-blind, randomized parallel-group trial (1:1)
- Intervention: hydroxychloroquine 400 mg/d for 5 days
- Control: usual care
- Recruitment: 4 to 28 February 2020
- Location: Renmin University Hospital, Wuhan (single-centre)
3/
- Design: double-blind, randomized parallel-group trial (1:1)
- Intervention: hydroxychloroquine 400 mg/d for 5 days
- Control: usual care
- Recruitment: 4 to 28 February 2020
- Location: Renmin University Hospital, Wuhan (single-centre)
3/
Inclusion criteria:
- Age â„18y
- CoV-2 confirmed by PCR
- Pneumonia confirmed by chest CT
- Not severely ill
4/
- Age â„18y
- CoV-2 confirmed by PCR
- Pneumonia confirmed by chest CT
- Not severely ill
4/
What about the outcomes? Well, since you ask:
- Time to clinical recovery (body temp and cough from D1 to D5)
- Pulmonary recovery (chest CT scan at D0 and D6)
First time we see actually clinical outcomes, by the way.
5/
- Time to clinical recovery (body temp and cough from D1 to D5)
- Pulmonary recovery (chest CT scan at D0 and D6)
First time we see actually clinical outcomes, by the way.
5/
Statistical analysis:
Compared with the Raoult study (1 sentence), this is very detailed (2 sentences)
Still jarringly short and basic.
6/
Compared with the Raoult study (1 sentence), this is very detailed (2 sentences)
Still jarringly short and basic. 6/
Side note: if you haven't heard of #GraphPad, it's a neat piece of software to plot data if you're not comfortable using a statistical package like R, SAS or Stata. https://www.graphpad.com/scientific-software/prism/
/7
/7
Main results (patients):
Out of 142 COVID patients admitted during the study period, 62 (44%) met the inclusion criteria. Mean age 45 years, 53% women. 31 patients were randomized to the treatment arm, while the other 31 received standard care.
/8
Out of 142 COVID patients admitted during the study period, 62 (44%) met the inclusion criteria. Mean age 45 years, 53% women. 31 patients were randomized to the treatment arm, while the other 31 received standard care.
/8
Main results (outcomes part 1):
Body temperature recovery time:
(a) Treatment group: 2.2 (SD 0.4) days
(b) Control group: 3.2 (SD 1.3) days
/9
Body temperature recovery time:
(a) Treatment group: 2.2 (SD 0.4) days
(b) Control group: 3.2 (SD 1.3) days
/9
Main results (outcomes part 2):
(a) Treatment group: 2 patients worsened (incl. 0 to severe), 19 stable, 25 improved.
(b) Control group: 9 patients worsened (incl. 4 to severe), 5 stable, 17 improved.
/10
(a) Treatment group: 2 patients worsened (incl. 0 to severe), 19 stable, 25 improved.
(b) Control group: 9 patients worsened (incl. 4 to severe), 5 stable, 17 improved.
/10
The authors conclude that in spite of the small sample size, their study "partially confirms" the efficacy of hydroxychloroquine in the treatment of COVID-19.
They add that, in the absence of a better alternative, it is a "promising practice" to use hydroxychloroquine.
/11
They add that, in the absence of a better alternative, it is a "promising practice" to use hydroxychloroquine.
/11
Convinced? I'm not, and here is why. 
/12
/12
Clear deviation from the protocol:
The trial protocol on ChiCTR mentioned 3 groups, with 100 patients in each: (1) HCQ 400mg/d, (2) HCQ 200mg/d, and (3) a placebo. Where the heck is group 2, and why has placebo been replaced by standard care?
http://www.chictr.org.cn/showprojen.aspx?proj=48880
/13
The trial protocol on ChiCTR mentioned 3 groups, with 100 patients in each: (1) HCQ 400mg/d, (2) HCQ 200mg/d, and (3) a placebo. Where the heck is group 2, and why has placebo been replaced by standard care?
http://www.chictr.org.cn/showprojen.aspx?proj=48880
/13
Underpowered study:
This trial was supposed to recruit 300 patients (200 receiving HCQ and 100 receiving placebo). Yet, the analysis reported in the preprint only includes 31 patients in the HCQ arm and 31 in the standard care arm. Of course, no explanation whatsoever...
/14
This trial was supposed to recruit 300 patients (200 receiving HCQ and 100 receiving placebo). Yet, the analysis reported in the preprint only includes 31 patients in the HCQ arm and 31 in the standard care arm. Of course, no explanation whatsoever...
/14
How big should it be?
Assuming no loss to follow-up (which seems to be the case here), 5% type-1 error rate, 85% power and Fleiss CC, a total of 128 COVID-19 patients (64+64) would be required to demonstrate the 55% vs. 81% proportion of patients who "improved" in Table 2.
/15
Assuming no loss to follow-up (which seems to be the case here), 5% type-1 error rate, 85% power and Fleiss CC, a total of 128 COVID-19 patients (64+64) would be required to demonstrate the 55% vs. 81% proportion of patients who "improved" in Table 2.
/15
A multitude of underpowered studies will not be of any use if we want to seriously answer the question "Is hydroxychloroquine beneficial for patients with COVID-19?".
https://twitter.com/statsepi/status/1234576310703398912?s=20
/16
https://twitter.com/statsepi/status/1234576310703398912?s=20
/16
Randomization process is questionable:
(1) Randomization was "stratified by site" (page 4): I was under the impression that there was only one site...
(2) With only N=62, the number of patients randomized to the treatment arm is exactly 31. A happy coincidence, I guess?
/17
(1) Randomization was "stratified by site" (page 4): I was under the impression that there was only one site...
(2) With only N=62, the number of patients randomized to the treatment arm is exactly 31. A happy coincidence, I guess?
/17
Randomization did not achieve equipoise, even though age and sex are nicely balanced:
Fever at day0:
(a) Treatment arm: 22/31
(b) Control arm: 17/22
Cough at day0:
(a) Treatment arm: 22/31
(b) Control arm: 15/22
And, of course, not statistical adjustment whatsoever
/18
Fever at day0:
(a) Treatment arm: 22/31
(b) Control arm: 17/22
Cough at day0:
(a) Treatment arm: 22/31
(b) Control arm: 15/22
And, of course, not statistical adjustment whatsoever
/18
P-values are misused:
(a) Since the authors provided no rationale for the sample size and since the study is underpowered, p-values are uninformative here. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4877414/pdf/10654_2016_Article_149.pdf
(b) In Table 2, using a Chi-2 test with such small cells is inadequate (Fisher's test: p=0.52).
/19
(a) Since the authors provided no rationale for the sample size and since the study is underpowered, p-values are uninformative here. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4877414/pdf/10654_2016_Article_149.pdf
(b) In Table 2, using a Chi-2 test with such small cells is inadequate (Fisher's test: p=0.52).
/19
Conclusion:
The authors' take-home message that "large-scale clinical [...] research is still needed to [...] continuously optimize the treatment plan" is inaccurate.
Large-scale clinical research is still needed to establish (or not) the efficacy of hydroxychloroquine.
/20
The authors' take-home message that "large-scale clinical [...] research is still needed to [...] continuously optimize the treatment plan" is inaccurate.
Large-scale clinical research is still needed to establish (or not) the efficacy of hydroxychloroquine.
/20
I guess I only spotted the most obvious cracks. @statsepi @jd_wilko @tmorris_mrc @ProfSimonGates @DgCostagliola and trial specialists will probably have more to say.
* I meant "other" trial specialists, obviously
