, editor-in-chief of @AmJEpi
. This long thread is for you.
Hello @MuinJKhoury @JustinLessler @shrutihmehta @robertwplatt
, editorial board of @AmJEpi
. This long thread is for you.
Hello @EllenJMacKenzie @dcelent1 @MarthaWerler
, policy board of @AmJEpi
. This long thread is for you.
Hello @leskocar @sunnimum
, editors in residency of @AmJEpi
. This long thread is for you.
Hello @ashley_naimi @brianwwhitcomb
, editors for the methods center of @AmJEpi
. This long thread is for you.
Hello @LucyStats @ProfMattFox @BryanDJames @EleanorJMurray
, associate editors for Social Media of @AmJEpi
. This long thread is for you.
This thread is for you all, editors of @AmJEpi
, regarding the article by Harvey Risch « Early Outpatient Treatment of Symptomatic, High-Risk Covid-19 Patients that Should be Ramped-Up Immediately as Key to the Pandemic Crisis ».
The review of evidence by Harvey Risch in his article is biased and not the fair appraisal that one might expect from a member of the editorial board of the American Journal of Epidemiology.
Peer-review process must be released.
The article must be retracted.
I declare no conflicts of interest. I didn’t, I don’t, and I won’t get any advantage from any pharmaceutical company producing any treatment or candidate treatment of Covid-19.
I read with great interest (at least the title) the paper by Harvey Risch published in American Journal of Epidemiology. https://academic.oup.com/aje/article/doi/10.1093/aje/kwaa093/5847586
I will focus this commentary on the statements by Harvey Risch regarding efficacy of the association hydroxychloroquine (HCQ) and azithromycine (AZ) for the treatment of Covid-19.
Harvey Risch criticizes the NIH and the FDA because, in his opinion, "[they] have omitted the two critical aspects of reasoning about these drugs : use of HCQ combined with AZ or with doxycycline, and use in the outpatient setting".
In his article, Harvey Risch reviews what he claims to be the evidence showing the effectiveness of HCQ+AZ in outpatient setting.
Why did medical agencies focus on HCQ, and why did medical agencies focus on hospitalized patients ?
(spoiler : because the promotors of HCQ said that it’s efficient in hospitalized patients)
At this point, it’s worth reminding some historical facts to Harvey Risch.
Firstly, the hype for HCQ is due to a video posted on february 25th 2020 by Pr Didier Raoult on Youtube in which he says that HCQ is effective on Covid-19.
Therefore, criticizing the fact that other researchers and agencies further tested the efficacy of HCQ and not only of HCQ+AZ doesn’t seem pertinent.
Secondly, and more importantly, the claim made on 25th of february that HCQ was effective relied on no proof at all.
Didier Raoult said that "chinese researches have proven the efficacy of HCQ".
This was a false statement.
No clinical trial was published before 25th of february 2020 on the efficacy of HCQ. This statement relied only on a letter to editor published by Gao et al. in Bioscience trends on 19th of february and titled
« Breakthrough: Chloroquine phosphate has shown apparent efficacy in treatment of COVID-19 associated pneumonia in clinical studies » https://www.jstage.jst.go.jp/article/bst/advpub/0/advpub_2020.01047/_article/-char/ja/
Gao et al. claimed that several clinical trials were led in China to test the efficacy of chloroquine and that results from more than 100 patients have demonstrated that chloroquine phosphate is superior to the control treatment for various outcomes.
However, we must point out that these clinical trials are led in hospitalized patients (as stated by Gao et al.) and therefore will not advocate for the use of HCQ+AZ in outpatient settings.
Furthermore, the letter by Gao et al. don’t show any data, and there was at this time no published clinical trial showing the data of these « 100 patients ».
Thirdly, Pr Didier Raoult led a clinical trial to prove the efficacy of HCQ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7102549/
His trial was a trial of HCQ vs. no-HCQ.
In this trial, he enrolled patients who developped Covid-19 pneumonia.
The authors of this study claim for efficacy of HCQ and even more efficacy of HCQ+AZ… based on a subgroup analysis of 6 patients also treated by AZ because they had superinfection, and not as a treatment of Covid-19.
I will further discuss the methodology and results of this trial.
Harvey Risch intends to give an example of article which do not study the efficacy of HCQ+AZ in outpatient setting. He choses the study by Magagnoli et al. which pre-print has been released in Medrxiv
For the reasons described above (there were no argument for HCQ in outpatients and the promotors of HCQ were saying that it was efficient based on studies on hospitalized patients) the article by Magagnoli et al. focused on HCQ in hospitalized patients.
In his article, Harvey Risch claims that " the Veterans' Administration Medical Centers study (ref 18) examined treated hospitalized patients and was fatally flawed (ref 19)"
Reference 19 cited by Harvey Risch is a letter to Magagnoli et al. by Didier Raoult published on the website of the IHU Méditerranée Infection. https://www.mediterranee-infection.com/wp-content/uploads/2020/04/Response-to-Magagnoli.pdf
It seems that Harvey Risch did not read both the paper by Magagnoli et al. and the reply by Pr Didier Raoult. In fact, the comments by Pr Didier Raoult on the article by Magagnoli et al are all nonsense :
a. Didier Raoult claims that Magagnoli et al. conclude their article saying that HCQ treated patients die twice as much as no-HCQ patients.
This is a false statement : the authors do not draw such a definitive conclusion. After discussing the bias of their own study, their main conclusion is that RCTs are needed.
b. Didier Raoult then focuses his criticism on the difference in lymphopenia between HCQ and no-HCQ groups. This is knocking an open door since Magagnoli et al. discussed this issue in the discussion paragraph of their article.
And the issue of different comorbidities between HCQ and no-HCQ groups was taken into account by a propensity score analysis which is a gold-standard analysis for observational studies.
c. The last criticism by Pr Didier Raoult shows that he didn't understand Magagnoli et al. study at all. Pr Didier Raoult claims that it is not clear how the groups were constituted.
It appears that Didier Raoult did not understand that there were 3 groups of patients (HCQ, HCQ+AZ, no-HCQ) made by two different methods, such as in a sensitivity analysis.
The first method accounts for treatment exposure over the whole study period.
The second method accounts for treatment exposure only before mechanical ventilation.
The first method is the normal method to study drug exposure. The second method is obviously performed to favor the HCQ groups if HCQ is effective on early stages (before mechanical ventilation, as was claimed by Didier Raoult).
Criticizing this sensitivity analysis and concluding to « scientific fraud » is nothing but completely stupid since this analysis was performed to give a chance to HCQ if HCQ is efficient as claimed by Didier Raoult himself.
It’s disturbing to read that Harvey Risch endorses such pointless criticisms and cite this response letter by Didier Raoult without any distanciation.
Finally, Harvey Risch claims in his article that "the same point about outpatient use of the combined medications has been raised by a panel of distinguished French physicians (ref 20)".
I wonder on what basis he calls them "distinguished"? The authors of the study in reference 20 are totally unknown, they have never published anything, and their study is an illegal (no authorization of the French regulatory agency and no approval of an ethics committee) trial.
At the time Harvey Risch wrote his article, this study was published on a pre-print server.
In fact the authors managed to publish their article... in a predatory journal. https://www.journalajmah.com/index.php/AJMAH/article/view/30224
This publication shows that all the authors but one have no true affiliation.
The only author that has a true affiliation is Pierre Levy, from iPLESP (Institut Pierre-Louis de Santé Publique, which is a French institute of Public Health).
iPLESP issued a statement on this topic : « The management of the IPLESP as well as its supervisory bodies (Inserm, Sorbonne University) and the AP-HP refute the methodology and the conclusions of the manuscript entitled
« Azithromycin and Hydroxychloroquine accelerate recovery of outpatients with mild/moderate COVID-19 » published in the Asia Journal of Medicine and Health and one of the co-authors of which is affiliated with the research center.
This article published in a predatory journal does not allow us to conclude that azithromycin administered alone or with hydroxychloroquine has any favorable impact on the course of the COVID-19 disease.
Finally, the regulatory status of the study as described in the paper raises questions and should be clarified ».
End of the statement by iPLESP.
Furthermore, the words « distinguished French physicians » used by Harvey Risch sound like an appeal to authority fallacy that should not take place in a scientific journal.
On what evidence of efficacy of HCQ is based the original article by Harvey Risch ?
(spoiler : a bunch of most biased studies on HCQ ever)
Harvey Risch writes that "in reviewing all available evidence, I will show that HCQ+AZ and HCQ+doxycycline are effective in preventing hospitalization for the overwhelming majority of such patients".
Let’s have a look at these studies and how Harvey Risch comments on them.
The first study picked by Harvey Risch is Gautret et al. 2020 clinical trial published in International Journal of Antimicrobial Agents (IJAA). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7102549/
The authors of this study claim that HCQ and HCQ+AZ are effective against Covid-19.
Harvey Risch claims that this study shows efficacy of HCQ+AZ.
He claims to have "reanalysed the raw study data" and found the same conclusion.
Harvey Risch adds that the only valid criticism against Gautret et al. is the lack of randomization.
He rejects the criticism about the "small study size", saying that criticism on size matters only when no evidence is found.
I wonder how the American Journal of Epidemiology can accept that.
"Lack of randomization" and "small study size" were not the only valid criticism against Gautret et al.
Other valid criticisms were :
a. Open label (even if this criticism must be tempered by the fact that it may have been difficult for the authors to obtain a placebo of HCQ)
b. The authors chose the surrogate endpoint of viral load. A patient died at day 3 with a negative viral load at day 2.
c. All the patients of HCQ group were treated at IHU of Marseille, all the patients of control group are from other cities
d. Follow up was not the same for all patients, with way more missing data for control group than HCQ group
e. According to study protocol, the study was planned for lasting 2 weeks. Authors ended the study after 6 days and never provided follow up data
f. The method used to deal with missing data was highly in favor of HCQ group, resulting in "PCR not performed" considered postive in control group and negative in HCQ group
g. Authors compared 3 groups with only one global Chi-squared test. They didn't perform post-hoc analysis. For a simple reason : post-hoc analysis shows that HCQ is more effective than control but HCQ+AZ is not more effective than HCQ alone.
(note : this is surprising that Harvey Risch did not see that, because in his review of safety he reanalysed safety data and performed post-hoc analysis with Bonferoni correction)
All these criticisms are swept away by Harvey Risch who only focuses on the 6 patients (1 died, 3 transfered to ICU) who were excluded from analysis (which is what can be called a true scientific misconduct).
It is worth noting that the peer-review process at IJAA lasted less than one day. The International Society of Antimicrobial Chemotherapy (ISAC) issued a statement in which it says that « the article does not meet the Society’s expected standard » https://www.isac.world/news-and-publications/official-isac-statement
Elsevier and ISAC published a statement on Gautret et al. paper in which they say that « additional independent peer review is ongoing to ascertain whether concerns about the research content of the paper have merit » https://www.journals.elsevier.com/international-journal-of-antimicrobial-agents/news/joint-isac-and-elsevier-statement-on-gautret-et-al-paper
This statement was published on april 11th.
At the time Harvey Risch wrote his paper, this statement had not been updated.
Now it has been updated : ISAC and Elsevier asked Fritz Rosendaal to peer-review the article by Gautret et al.
His conclusion : « this is a non-informative manuscript with gross methodological shortcomings. The results do not justify the far-reaching conclusions about the efficacy of hydroxychloroquine in Covid-19, and in the view of this reviewer do not justify any conclusion at all »
Instead of retracting the article by Gautret et al, ISAC and Elsevier decided to publish the comments by Rosendaal along with an editorial and another comment on another article by Raoult published in IJAA.
This happened 4 months after the initial publication : the review by Frits Rosendaal was written 19th of April 2020 and published 13th of July 2020 in IJAA.
It's disturbing that Harvey Risch didn't see the big methodology problems of the article by Gautret et al.
Furthermore, Harvey Risch seems to not understand the importance of a large sample size. A large sample size is important to detect a difference, but it is way more important to have a robust estimator with tight confidence interval.
This clinical trial by Gautret et al. was first released on 16th of March on Youtube
Harvey Risch claims that this study shows that HCQ+AZ must be given very early.
I must object that this study was led on hospitalized patients, and included patients who were not given medication early.
Didier Raoult himself, on the basis of his clinical trial, said on March 25th that « patients must be treated when they have moderate Covid-19 or starting aggravation Covid-19 », not only asymptomatic or early Covid-19. And not only « high risk patients ». https://twitter.com/raoult_didier/status/1242727178950893568
The second study highlighted by Harvey Risch is the study by Million et al https://www.sciencedirect.com/science/article/pii/S1477893920302179
This study is once again from IHU Méditerranée Infection.
It was first published on April 20th on the website of IHU Mediterranee Infection, then on May 5th in Travel Medicine and Infectious Disease.
There would be much to say about this study, especially on the differences between the pre-print version and the final version.
But one criticism is sufficient to exclude this study of the list of studies showing efficacy of HCQ+AZ : this study has no control group.
The authors of this study conclude to efficacy of the association HCQ+AZ.
A study with no control group can’t conclude to efficacy.
I wonder how Harvey Risch can ignore that.
The third study is Zelenko’s study, which data are not even published.
The commentary by Harvey Risch relies on a 2 pages statement by Dr Zelenko on Google Doc.
How can Harvey Risch comment on a study which is not even published?
The fourth study is a "controlled non-randomized trial of HCQ+AZ in 636 symptomatic high-risk outpatients in São Paulo,Brazil" published on Wordpress.
This study was initially touted by Didier Raoult https://twitter.com/raoult_didier/status/1251450740641542144
And finally the fifth study highlighted by Harvey Risch is "a small study ongoing in a long-term care facility in Long Island, NY".
And that's all.
Conclusion : This review by Harvey Risch is nothing but cherry picking of worst studies ever published on HCQ for Covid-19.
Is there a chance that HCQ works ?
(spoiler : no)
The fact is that efficacy of HCQ or the combination HCQ+AZ has not been proven.
It has not been proven in hospitalized patients.
It has not been proven in outpatients.
It has not been proven in early / mid / late Covid-19.
The true question is : is there even a chance that HCQ be effective ?
To answer this question, one should take into account that the FDA has published an article showing that HCQ doses that have been found active in vitro cannot be reached in vivo. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7314136/
And that is the reason why the RECOVERY study (even if it has been criticized by stupid people who don't have any knowledge on pharmacology) used high doses of HCQ, which are not even enough to reach the goal of in vitro studies. https://www.recoverytrial.net/files/recovery-protocol-v6-0-2020-05-14.pdf
When a drug shows no benefit, should one care for its harms ?
(spoiler : no, because it should not be prescribed and patients should not take it)
At this point, it should not be necessary to analyze the review of evidence of safety of HCQ+AZ by Harvey Risch.
Because Harvey Risch’ review failed to demonstrate efficacy of HCQ+AZ even in outpatient setting, therefore advocating that risks are rare is pointless when benefits are nonexistent.
Nevertheless, it is of interest to remind some simple key notions of pharmacology :
a. Every drug have side effects.
b. Focusing on rare and fatal side effects and ignoring frequent side effects may be interesting when a drug is shown to be very efficient. This is not the case of HCQ or HCQ+AZ in Covid-19.
c. When a drug has no benefit, it only has side effects
d. Clinical trials are bad to show rare side effects. Trying to conclude on rare side effects on the basis of small clinical studies is stupid.
e. Pharmacovigilance can show rare side effects. Pharmacovigilance showed rare side effects of HCQ and the combination HCQ+AZ.
Harvey Risch concludes his review saying that « it is not ethic to give no treatment to patients ».
This is hard to believe that such a sentence was published in an epidemiology journal.
One of the basic principles of bioethics is « primum non nocere ». This principle implies that drugs who have not shown benefit should not be prescribed to patients.
How did Harvey Risch react to comments on his article ?
(spoiler : with even more bullshit)
I am well aware that American Journal of Epidemiology published comments on this paper by Harvey Risch.
However, the way that Harvey Risch reacted to these comments is disgusting.
Instead of accepting criticisms, he responded disdainously to Korman.
I will focus on his response to @DgCostagliola
Peffer-Smadja and Costagliola reanalyzed the data of Gautret et al. and found no statistical difference between HCQ and HCQ+AZ groups (p>0.05).
Harvey Risch argues that « Peiffer-Smadja and Costagliola need to use the outcome event times in Cox regression in order to obtain proper P-values ».
This is beyond dishonnest for two reasons :
a. Cox regression was not the analysis planned in the protocol of the study, and Peffer-Smadja and Costagliola performed the same statistical tests as the authors of Gautret et al.
b. Performing Cox regression on such poor data (small number of patients, missing values for outcome, PCR switch from positive to negative and then positive again) is inappropriate.
Another comment by Peffer-Smadja and Costagliola was that Gautret et al. was not conducted in outpatient (which is a big problem when Harvey Risch pretends to show outpatient efficacy of HCQ when he cites this study).
Harvey Risch responds to this comment that « Many of the patients in the Gautret et al. study were asymptomatic or very mildly symptomatic and would be treated as outpatients in most circumstances. ».
In other words : « Ok, the patients of this study were not outpatients, but they could have been ! ». What kind of dishonnest answer is that ?
In the study by Gautret et al., two patients out of 20 patients treated by HCQ were asymptomatic.
Is that « many » ?
How many patients were « very mildly symptomatic » ? We don’t know, because Gautret et al. does not mention that. How does Harvey Risch know ?
Harvey Risch writes : « Peffer-Smadja and Costagliola label a carefully performed, sequential-patient non-randomized controlled clinical trial an “unpublished, poorly designed studies whose quality is even lower than [Gautret et al. and Million et al studies]”. [...]
[...] I disagree with this characterization as it is unsupported by the evidence ».
Peffer-Smadja and Costagliola are right. As I show above, « Zelenko study », « Sao Paulo study » and the « small study ongoing in a long-term care facility in Long Island, NY » are unpublished.
Given they are unpublished, how can Harvey Risch claim that they are « carefully performed non-randomized controlled clinical trial » ?
For instance, is Zelenko study a clinical trial ? Where has it been registered ? I can’t find it in clinical trials register database.
Finally, Harvey Risch adds even more cherry picking, citing 7 other studies that, in his opinion, show efficacy of HCQ in high risk outpatients.
He cites for example the study by Lagier et al. (from Didier Raoult’s team).
In this study, Lagier et al. compared the patients treated by HCQ in their facility in Marseille (treated group), to the patients that they didn’t treat (control group).
Didier Raoult and the MDs of IHU Méditerranée Infection issued a statement in which they say that they will treat all patients by HCQ in their facility. https://www.mediterranee-infection.com/epidemie-a-coronavirus-covid-19/
So there is absolutely no way that patients treated by HCQ in their facility can be compared to patients not treated.
Because if they did not treat some patients, it’s obviously because they were at higher risks or had contra-indication to treatment.
Furthermore, in this study Lagier et al. mixed hospitalized patients with non-hospitalized patients.
In the control group, 39.3% were hospitalized.
In the treated group, 13.8% were hospitalized.
At IHU Méditerranée Infection, the rule was to follow patients in day care hospital (= outpatients).
When the patients worsened, they could be hospitalized in conventional units.
But Lagier et al. did not provide the percentage of patients who were initially treated as outpatient and further needed hospitalization.
They provided only the number of patients hospitalized, which includes the patients who were initially hospitalized (without a follow up as outpatient prior to hospitalization).
Way more importantly, to conclude to efficacy of HCQ+AZ, the authors of this article removed the patients treated by HCQ+AZ who aggravated early and they put these patients in the control group !
There is obviously an immortal time bias in this study, which cannot be corrected by multivariate analysis.
Of course, this retrospective study is a total mess.
The true question is : since Pr Raoult and colleagues treated more than 3000 patients by HCQ or HCQ+AZ, why didn’t they randomize some of them in a control group ?
And why an epidemiologist such as Harvey Risch only cites studies without any critical appraisal ?
In his response to Peffer-Smadja and Costagliola, Harvey Risch writes that « a meta-analysis of studies to-date completely demonstrates this benefit  ». Reference 14 cited by Harvey Risch is a link to the website of the IHU Méditerranée Infection.
The link reaches a response from Didier Raoult to Elsevier.
As I described above, ISAC (the society editing IJAA) and Elsevier published a joint statement to criticize the article by Gautret et al.
They asked for an epidemiologist to make an independent peer-review of the article and finally published the review by Frits Rosendaal.
Ref 14 cited by Harvey Risch is the response from Didier Raoult to Elsevier.
It is worth noting that this response was not published in IJAA along with commentaries by Frits Rosendaal.
It's easily comprehensible : this response is ridiculous.
In this response, Didier Raoult cites in Figure 1 and ref 20 a meta-analysis by Million et al. (Didier Raoult last author as always) https://www.sciencedirect.com/science/article/pii/S2052297520300615
(Yes, the meta-analysis that Harvey Risch cites in his response to Peffer-Smadja and Costagliola is in fact performed by Didier Raoult’s team.)
Harvey Risch cites this meta-analysis by citing the reply from Didier Raoult to Elsevier instead of citing the meta-analysis itself.
This shows that there is no distance at all in the appraisal by Harvey Risch : he reads something by Raoult and he cites it, and that’s all.
In fact, this meta-analysis is not a meta-analysis. It’s a poor litterature review by Million et al. published in New Microbes and New Infections, a journal created by Didier Raoult himself to publish his articles that have been rejected elsewhere. https://science.sciencemag.org/content/335/6072/1033.summary
A rapid search in pubmed shows that more than 30 % of the articles published in NMNI are signed by Didier Raoult https://twitter.com/Clara_Locher/status/1270379123697840128
Harvey Risch wasn’t able to detect that?
Furthermore, this « meta-analysis » has been widely mocked on twitter... because it is not a meta-analysis
In fact, the figure copied by Harvey Risch in his response to Peffer-Smadja and Costagliola isn’t even part of the meta-analysis published by Million et al. It’s from nowhere but the response from Didier Raoult to Elsevier.
This proves again that there is absolutely no distancing when Harvey Risch reads something by Didier Raoult. He accepts it as truth without even veryfiying it is consistant.
Time to conclude
For all these reasons, I think that Harvey Risch’ article should be retracted from American Journal of Epidemiology.
It's not dishonorable to retract a bad article.
This review by Harvey Risch is a bad article.
It should not have been published in a journal of epidemiology.
You ( @AmJEpi
and editors) should also explain why the comments by Korman, Fleury, Costagliola and Peiffer-Smadja on the article by Harvey Risch are published 2 months later. Is this acceptable according to the Covid-19 pandemic?
You ( @AmJEpi
and editors) should release the peer-review process of the article by Harvey Risch (the names of the reviewers and the comments they made on his article).
Now Harvey Risch, an editor of @AmJEpi
, publishes an article in France Soir, a fake journal whose journalists were all fired one year ago and whose director is promoting essential oils for Covid-19.
Now Harvey Risch, an editor of @AmJEpi
, is writing his opinion in Newsweek relying on the fact that he managed to publish his fake review of evidence in @AmJEpi
You ( @AmJEpi
and editors) should react.
It is not a matter of « is HCQ efficient or not ? ».
It is a matter of credibility for @AmJEpi
It is a matter of « is this critical appraisal by Harvey Risch honnest and worth publishing in a world leading journal of epidemiology ? ».
And it’s definitely not.
, this is not freedom of speech.
This is freedom of bullshit with the seal of @AmJEpi
A shame for @YaleSPH
« Hydroxychloroquine works in high-risk patients, and saying otherwise is dangerous »
« saying otherwise is dangerous » -> Is this still « scientific debate », @SVermund @AmJEpi
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.