Pondering whether COVID-19 will be evidence-based medicine’s nemesis. Having saved lives for 25 years, EBM (narrowly interpreted) will now tend to increase deaths. Not yet convinced, but worried. 1/

As I’ve said before, I’m a survivor of a poor-prognosis cancer, caught early and treated promptly. I owe my life to a series of methodologically rigorous observational studies and randomised controlled trials. I literally wouldn’t be alive but for EBM. 2/

I was one of the early protagonists of EBM. I published my first papers on EBM in 1996. I’ve written one of the world’s best-selling textbooks on EBM (it’s sold over 100,000 copies and is in its 6th edition). Repeat, I’m not in principle anti-EBM. 3/

Here’s my worry. EBM is methods-focused. And it was designed for a particular purpose: making treatment decisions for individual patients based on the findings of RCTs of therapy (and diagnostic decisions based on findings of validation studies). 4/

Of course, there’s a lot more to EBM than that (‘splainers, please, like I said I’ve written books about this subject). But broadly speaking, EBM taught us to hold back from putting a patient on a drug till we had definitive RCT findings. 5/

This is a good principle. It’s guided my clinical practice for 25 years. It’s appropriately cautious. It resonates with the tony bit of Latin that most doctors can still speak: primum non nocere (first, do no harm). 6/

This principle worked well for me when I had cancer. It saved me from a mutilating operation that had been standard practice until a RCT showed it was harmful not beneficial. It gave me the chemotherapy and targeted therapy that eradicated my cancer. 7/

But the principle of waiting for the definitive RCT before taking action should not be seen as inviolable, or as always defining good science. On the contrary, this principle, inappropriately applied, will distort our perception of what “good science” is. 8/

A RCT is intended to assess the effect size of a particular intervention under controlled conditions. It follows simple, cause-effect logic. Drug A has an effect size of x when compared to drug B (perhaps placebo). 9/

There is an extensive literature on whether and when it’s appropriate to do RCTs of complex social interventions. Social scientists are split. As a rule of thumb, RCTs will tend to underestimate their impact and observational studies to overestimate it. 10/

Literature on complex systems highlights a more organic model of causality with multiple interacting influences. Ask not “what is the effect size; is it statistically significant, controlling for other variables?” but “does this intervention *contribute* to a better outcome?” 12/

Using the logic of complex systems, multiple interventions might each contribute to an overall beneficial effect, even though none of these interventions individually would have a statistically significant impact on any predefined variable. 12/

How do we study multiple interventions in complex systems? Well, we can *model* them (since interventions interact and baselines change over time). We can *observe* them (routinely collected data etc). And we can look historically and contemporaneously at natural experiments. 13/

Our EBM training teaches us to downgrade studies that have a “lesser” research design than a RCT. But if we over-apply this rule of thumb, we end up dismissing the very study designs that might provide the richest insights into how the complex system is behaving. 14/

Giving excessive weight to RCTs also biases us against engaging with disconfirming data from the real world. Who cares about a choir practice in Seattle? It wasn’t a RCT! Who cares about basis science studies of droplets and aerosols? Only RCTs count! 15/

So here’s my coda: I say EBM (narrowly defined and applied) was good for 20th-century medicine where simple causality held good most of the time. It could be perilously misleading in 21st-century pandemics characterised by complex chains of causality. 17/

More specifically, taking a “primum non nocere” (= don’t act till we’ve got RCT certainty) stance when hundreds are dying daily makes no scientific or moral sense. It is neither scientifically nor morally reckless to try out policies that have a plausible chance of working. /ends