Read on Twitter
New (short) paper out - A Bayesian re-analysis of the convalescent plasma (CP) arm of RECOVERY with @DrToddLee @karlahemming @rjlilford and @DavidT_Arnold.
Key question: Is CP futile in hospitalised patients with COVID-19?
below... https://www.medrxiv.org/content/10.1101/2021.04.01.21254679v1
Key question: Is CP futile in hospitalised patients with COVID-19?
below... https://www.medrxiv.org/content/10.1101/2021.04.01.21254679v1
CP has been used for centuries to treat infection - and relies on using antibodies from previously infected/vaccinated people to treat people. it has been tested in COVID-19, where it seems early Ab response is critical in recovering from infection: https://www.hematology.org/covid-19/covid-19-and-convalescent-plasma
Our interest is whether CP is beneficial in patients hospitalised with COVID-19. Recently,a slew of small RCTs have been published (meta-analysed below), that even in sum, are too small to answer the question either way: https://jamanetwork.com/journals/jama/fullarticle/2777060
But luckily, RECOVERY, led ably by @PeterHorby and @MartinLandray, leveraged the whole NHS infrastructure to allow a large (n=11,558!), pragmatic, study on this exact question. Appropriately, the analysis plan was pre-specified, and stopping criteria etc led by the DSMB
(And it is worth pausing at this point to congratulate the effort of everyone involved in RECOVERY - a fantastic achievement, and one I am proud to play a tiny, tiny, part in!)
This study concluded there was no effect of CP on mortality, and concluded no effect (their conclusion on R). They also found (again, pre-specified hypothesis testing) no effect on any subgroup, in particular, in seronegative vs seropositive patients.
Pre-specified testing and analysis plans are great, and clearly reduce false positives ( @MarcusMunafo is worth a follow on this), but in this case, we think the correct answer is not: evidence of NO effect, but NO evidence of an effect. This might seem a minor point, but..
Given the number of patients with COVID-19, and the importance of the outcome (death), we would argue that even a small (say 1% absolute risk difference or number needed to treat of 100) would be relevant to many clinicians. Would you take a drug with an NNT of 100 in COVID?
So; what is the chance that CP *actually* has an NNT of 100 for death? We did a re-analysis of their raw data using multiple priors (see paper for methods, or run your own using our code): it's around 20%: The study is consistent with a 20% chance of CP having an NNT of 100!
Now, is that enough to keep trialing the drug? Maybe yes/maybe no - but not (in our view), evidence of NO effect. We then went on to look at the seronegative subgroup. Why? Because, we are donating antibodies in CP. It is v likely the effect will be different in this group
Now, subgroup analyses are complex, and Twitter too short, but, in this case, we would suggest that given the treatment is antibodies to kill covid, stratifying by antibody status is pretty reasonable. Many clinicians would not argue there was equipoise in seropos patients.
And this was pre-specified in RECOVERY (for the same reasons). Here are the results for the subgroup analyses- the KM for the primary outcome, and the forest plots for the secondary outcomes
So: in the pre-specified analyses: no formal effect in main analyses, but benefit of CP in seronegative inboth secondary outcomes (discharge at d28, ventilation or death). Now, this is a dangerous game - subgroup analyses and secondary outcome cherry picking is not fair, but..
I highlight to show the heterogeneity and advantages of a Bayesian approach. Let's ask the question instead: what is the probability that CP has an NNT of 100 or more in the seronegative arm: it's ~73%! So, more likely than not! As I said, there are complexities with subgroup...
analyses, and the RECOVERY team should be commended for their pre-specified approach, but concluding a NULL effect for CP in the seronegative arm is not the interpretation we would take. Instead, we would say: it is possible (even probable), that CP has a benefit in seroneg
hospitalised patients with COVID-19. We should put this data in context with excellent early data of monoclonals in early disease (RRR >50%), and failure in intensive care patients (in REMAP-CAP). In summary: CP may well have a benefit in seroneg patients with C19.
We should not throw the results away and say no benefit - there is NOT evidence of absence here. I should say, as a conflict of interest, I hate CP, it is a messy blood product and is a real hassle for lots of reasons. I have no vested interest in its success, but it may well..
work! Many thanks to everyone involved, especially @DrToddLee for helpful comments and @karlahemming for stats. All code available to run your own analyses.,
