Time to share some thoughts and questions following the . @medrxivpreprint first release of results from the #SolidarityTrial that compared local standard care to #remdesivir, #HydroxyChloroquine #lopinavir_ritonavir and #interferon for treatment of patients with #COVID19

I have read the #solidarityTrial preprint: https://www.medrxiv.org/content/10.1101/2020.10.15.20209817v1

Its supplementary material: https://www.medrxiv.org/content/10.1101/2020.10.15.20209817v1.supplementary-material

its posted protocol: https://www.who.int/publications/m/item/an-international-randomised-trial-of-additional-treatments-for-covid-19-in-hospitalised-patients-who-are-all-receiving-the-local-standard-of-care

The CRF: https://www.who.int/publications/i/item/WHO-2019-nCoV-Clinical_CRF-2020.4

The final report from #ACTT1 for contrast: https://www.nejm.org/doi/full/10.1056/NEJMoa2007764#.X49WAB_MTQ8.twitter

Everyone has focused on the results, which is understandable, yet understanding the specifics in #SolidarityTrial design compared to the #ACTT1 design, may bring light into the differences reported between trials and may give a way to do better in #COVID19 #therapeutics

I am reviewing the preprint as if it had come to me from a colleague or journal for peer review, so maybe some of the questions that do need to be clarified are answered before the final publication of the #solidarityTrial.

Here is a first table comparing some basic design characteristics between #ACTT1 and #SolidarityTrial

- major difference the open-label design in Solidarity vs. concealed allocation and double-blind design in ACTT1, which is of higher quality and less risk of bias.

If you plug the different key components of both trials in into the . @cochranecollab risk of #bias tool (RoB 2), this is what you get

ref: https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-bias-tool-randomized-trials

Solidarity is flagged in several domains as a high-risk of bias trial, despite its relatively large sample size.

This is important because the risk of bias could be towards one treatment, but also towards the null, and these biases that are not at the point of randomization can affect results deeply.

As you dive into the methods, issues pop that need to be clarified for proper assessment:

#solidarityTrial has no explicit definition in the protocol or preprint of what is #Covid19, only mentions that patients with #definitive Covid19 were enrolled, and the results mention the number of patients were excluded because #Covid19 was categorically excluded. Pretty basic.

In #ACTT1 and other trials, the definition was explicit in term of molecular testing and imaging. Here we don't know (yet). In #RecoveryTrial in the UK, which followed the same #Oxford approach as #Solidarity, >10% of patients did not have a confirmed infection with #SARSCoV2.

This is not trivial, as if patients were enrolled into the trial sick, but without #Covid19, then no #SARSCoV2-specific intervention will improve the patients outcome and the risk of bias towards the null will increase with the proportion of patients without confirmed disease.

Other eligibility issue is that it was up to the “physician’s view” to decide if a particular drug was contraindicated. No guidance or explicit criteria for any of the 4 drugs is stated.
This sounds cool & simple, would is really a serious issue when using investigational drug.

The #solidaritytrial protocol mentions if a patient is registered, but has a contraindication to any of the drugs available, then the patient would not be randomized at all. The preprint does not mention how many patients were excluded and not randomized, basic #CONSORT stuff.

My sense from the #solidaritytrial preprint is that it refers only to the drug contraindicated, but that is not what the protocol states (clarify)
It's not clear if patients were stratified by site, protocol is silent on this, the preprint suggests it did. Another major bias risk

Another basic definition point to keep in mind is that the #solidaritytrial defined severe #COVID19 as need for ventilatory support, #ACTT1 as need for any supplemental oxygen.

The primary endpoint was in hospital #mortality. The protocol is explicit that patients discharged alive are not followed.This is tricky when you look at the CRF, outcome includes transfer (worsened?), palliative discharge. May explain the overall lower mortality c/w #Recovery.

And that type of truncation of events may work to a bias towards the null as we look at the now famous preprint main manuscript figure.

Why am I concerned about this as a potential issue?

Take a look at the supplemental figures that separates ventilated vs. non-ventilated patients, put them in the same order as the figure above for comparison. Except for lopinavir, patients in active arms had higher mortality.

Different from the #ACTT1 comparable population where no effect was noted with #remdesivir among patients in need of ventilatory support compared to those who just needed supplemental oxygen.

It is perplexing that the WHO proposed the use of the ordinal scales to capture the totality of effects of any intervention, even if they were not lifesaving (see link) that everyone else followed, but they themselves decided to do a mortality only trial. https://www.who.int/publications/i/item/covid-19-therapeutic-trial-synopsis

Then there are some basic Good Clinical Practice/Protection of Human Subjects points in the methods that concerned me.
Most striking is the admittance that consent could be retrospective "where locally approved." Consent patients after they are administered investigational drugs?

I don't think this is justifiable, no matter if locally approved, and blows over a basic human subjects protection issue that WHO should defend at all costs, even during a #pandemic!

How do we value a trial that does not respect this basic principle of participation?

then we have the problem that it was up to the doctor to decide what treatments the patient could be randomized, and I don't think doctors were immune to the huge therapeutic mood swings in the field, that could sway them to pick what they (or the patient) would prefer.

Finally we have the issues with documentation. If everything was paperless, consents were not retained, and randomization decisions were left to the "physician's view" in a stressful and polarized environment, how do you reconstruct the trial, audit the results?

Big, Robust results, can be imposing, but they don't necessarily mean they are better, big can be beautiful, but big can be monstrous, even if randomization occurred.

I think we tried to shortcut our research interventions by oversimplifying our trial designs, and we risk not having anything meaningful at the end, come short for our patients.

Hard to toss the results of #ACTT1 given its high quality design and conduct.

So time to re-propose to all involved the #ReDexCoVer trial.

Given the results and mood swings of the last 2 weeks, we need to seriously consider it.