Read on Twitter
1/
My appraisal of the RECOVERY trial: This was a (very) positive study.
How could chance or bias affect the validity of these results?
What was missing?
How should we apply these results?
Let’s take a deeper look.
#HowIReadThisPaper
My appraisal of the RECOVERY trial: This was a (very) positive study.
How could chance or bias affect the validity of these results?
What was missing?
How should we apply these results?
Let’s take a deeper look.
#HowIReadThisPaper
2/
Regarding bias: Strict inclusion/exclusion criteria can introduce selection bias by creating a highly selected study population.
This was NOT so in RECOVERY.
In figure 1, we see that 83% of recruited* patients were ultimately included.
(*assuming “recruited” = “screened”)
Regarding bias: Strict inclusion/exclusion criteria can introduce selection bias by creating a highly selected study population.
This was NOT so in RECOVERY.
In figure 1, we see that 83% of recruited* patients were ultimately included.
(*assuming “recruited” = “screened”)
3/
Amazingly, ~15% of all patients hospitalized for COVID19 in the UK during the study period were included in this trial.
No other Tx trial for COVID19 even comes close to that level of representation.
→These results should generalize broadly to hospitalized patients.
Amazingly, ~15% of all patients hospitalized for COVID19 in the UK during the study period were included in this trial.
No other Tx trial for COVID19 even comes close to that level of representation.
→These results should generalize broadly to hospitalized patients.
4/
The choice of all-cause mortality at 28 days as the primary outcome was bold. All-cause mortality is:
-Likely to UNDERestimate COVID19 cause-specific mortality
-Difficult to show a reduction in
-Patient-centered
(the protocol confirms the outcome was not changed).
The choice of all-cause mortality at 28 days as the primary outcome was bold. All-cause mortality is:
-Likely to UNDERestimate COVID19 cause-specific mortality
-Difficult to show a reduction in
-Patient-centered
(the protocol confirms the outcome was not changed).
5/
Regarding chance: Through simple bad luck, dex patients were on average ~1 year older than usual care patients.
Since ↑age is a risk factor for COVID19 death, we would expect those patients to do worse, attenuating any benefit of dex that might exist.
Regarding chance: Through simple bad luck, dex patients were on average ~1 year older than usual care patients.
Since ↑age is a risk factor for COVID19 death, we would expect those patients to do worse, attenuating any benefit of dex that might exist.
6/
Not anticipating this, the authors adjusted for age in their regression.
Is this bad? No.
Does it need to be planned a priori? Yes.
For a fantastic discussion of why and how we should do this, see this post from @ESteyerberg on @f2harrell’s blog: https://www.fharrell.com/post/covadj/
Not anticipating this, the authors adjusted for age in their regression.
Is this bad? No.
Does it need to be planned a priori? Yes.
For a fantastic discussion of why and how we should do this, see this post from @ESteyerberg on @f2harrell’s blog: https://www.fharrell.com/post/covadj/
7/
Why does this need to be specified up-front?
Planning this removes the possibility of choosing to adjust for a covariate only when doing so makes a positive result more likely.
The unadjusted analysis confirms age adjustment did not “create” these results:
Why does this need to be specified up-front?
Planning this removes the possibility of choosing to adjust for a covariate only when doing so makes a positive result more likely.
The unadjusted analysis confirms age adjustment did not “create” these results:
8/
Speaking of age, what does it mean that ventilated patients were, on average, ten years younger than non-ventilated patients?
This is not what I would expect if the decision to intubate was based solely on disease severity.
Speaking of age, what does it mean that ventilated patients were, on average, ten years younger than non-ventilated patients?
This is not what I would expect if the decision to intubate was based solely on disease severity.
9/
Another possibility: it may have been easier to decide to intubate a younger, healther patient than an older, sicker patient (↑ risk of harm).
We know that early in the pandemic, early intubation was common - maybe too common: https://www.statnews.com/2020/04/21/coronavirus-analysis-recommends-less-reliance-on-ventilators/
Another possibility: it may have been easier to decide to intubate a younger, healther patient than an older, sicker patient (↑ risk of harm).
We know that early in the pandemic, early intubation was common - maybe too common: https://www.statnews.com/2020/04/21/coronavirus-analysis-recommends-less-reliance-on-ventilators/
10/
Despite that, controlling for age and stratifying by level of respiratory support increases my confidence that neither of those confounded the association between dex and reduced mortality.
Despite that, controlling for age and stratifying by level of respiratory support increases my confidence that neither of those confounded the association between dex and reduced mortality.
11/
Regarding application: Clearly, patients who A) required intubation and B) had symptoms for >7 days derived the greatest benefit from dex.
Can we disaggregate which of A or B is more important? Help me here, #medtwitter #epitwitter - I don’t have a good answer.
Regarding application: Clearly, patients who A) required intubation and B) had symptoms for >7 days derived the greatest benefit from dex.
Can we disaggregate which of A or B is more important? Help me here, #medtwitter #epitwitter - I don’t have a good answer.
12/
The suggestion of harm from steroids in patients not requiring O2 raises the hypothesis that the host inflammatory response is a key driver of hypoxemia, and that normoxemic patients may be less inflamed. An area for future study.
The suggestion of harm from steroids in patients not requiring O2 raises the hypothesis that the host inflammatory response is a key driver of hypoxemia, and that normoxemic patients may be less inflamed. An area for future study.
13/
Based on this, other covariates I would have liked to see included in the model:
- Date of symptom onset (to disaggregate duration from severity)
- Date of admission (since in-hospital mortality rate and likelihood of intubation likely changed as outbreak progressed)
Based on this, other covariates I would have liked to see included in the model:
- Date of symptom onset (to disaggregate duration from severity)
- Date of admission (since in-hospital mortality rate and likelihood of intubation likely changed as outbreak progressed)
14/
As the authors eloquently put it, these data suggest the benefits of steroids in COVID19 are dependent on choosing:
- the right dose (not too high)
- the right time (not too early), and
- the right patient (not oxygenating too well).
As the authors eloquently put it, these data suggest the benefits of steroids in COVID19 are dependent on choosing:
- the right dose (not too high)
- the right time (not too early), and
- the right patient (not oxygenating too well).
15/
Bottom line: Dex was associated with an absolute ↓ in 28-day all-cause mortality among patients hospitalized with COVID19 requiring any level of respiratory support, but not those on room air.
Benefit was greatest in patients who were intubated or had symptoms >7d.
(End)
Bottom line: Dex was associated with an absolute ↓ in 28-day all-cause mortality among patients hospitalized with COVID19 requiring any level of respiratory support, but not those on room air.
Benefit was greatest in patients who were intubated or had symptoms >7d.
(End)
