Thread by @samedwardsmd, Our letter to the editor in @NEJM on the @CamdenHealth Hotspotting RCT [...]

Our letter to the editor in @NEJM on the @CamdenHealth Hotspotting RCT is out today.

https://www.nejm.org/doi/full/10.1056/NEJMc2001920#.Xs8sLXg5gik.twitter

A">https://www.nejm.org/doi/full/... thread explaining our thinking.

Our letter to the editor in @NEJM on the @CamdenHealth Hotspotting RCT is out today. https://www.nejm.org/doi/full/... thread explaining our thinking.

Our basic point is that RCTs are not the most informative way to evaluate complex interventions embedded in complex systems.

We were inspired by @trishgreenhalgh and Papoutsi& #39;s piece & associated articles on complexity science in @BMCMedicine in 2018. https://rdcu.be/b4rtT ">https://rdcu.be/b4rtT&quo...

Don’t worry, we love RCTs! We understand randomization, and basic epidemiology.

RCTs excel studying linear, predictable relationships.

Like evaluating the efficacy of medicines that have potential harm

say... treating people for COVID-19 w/ off the shelf lupus drug

https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Rightwards arrow" aria-label="Emoji: Rightwards arrow">need RCT

Also, we believe the findings, i.e. that the intervention did not reduce hospitalizations at 6 months.

This is not surprising, MANY studies of complex interventions that try to reduce hospitalizations of “high risk” patients have similar findings.

https://rdcu.be/b4sIw ">https://rdcu.be/b4sIw&quo...

More examples

@DonnaZulman et als RCT of Intensive Primary Care VA
https://ja.ma/3eujZSf

Yoon& #39;s">https://ja.ma/3eujZSf&q... eval of PACT intensive management
https://www.acpjournals.org/doi/10.7326/L18-0461

Elegant">https://www.acpjournals.org/doi/10.73... studies with similar results, little to no impact on hospitalizations. But WHY don& #39;t they "work"? What happened?

Augmenting Primary Care for High-Need Veterans Affairs Patients

This randomized clinical trial of a Veterans Affairs intensive outpatient care program evaluates how augmenting the Veterans Affairs medical home with an intensive management program influences...

https://ja.ma/3eujZSf

Here& #39;s a complexity definition from Greenhalgh

“Hotspotting” is a complex intervention (multicomponent, adaptive) that is embedded in a complex system (an impoverished, diverse city w/ limited employment & fragmented care)

The intervention and context are inseparable.

Intervention function may depend on co-evolution with contextual factors that are changing constantly. Instead of seeking an & #39;objective& #39;, faux laboratory, context free zone, why don’t we study what is actually happening, in all its messy glory? Again from @trishgreenhalgh

We advocate for rich, mixed-methods evaluations that seek to understand how and why interventions function in context.

Here Evelyn Chang et al. describes how a multisite intensive primary care intervention evolved over time.
https://www.sciencedirect.com/science/article/pii/S2213076417301021">https://www.sciencedirect.com/science/a...

Here @blchan7& #39;s qualitative study in @AnnFamMed of a hotspotting program in an FQHC beautifully describes how flexible, committed interdisciplinary teams stick with patients through chaos and define success as engagement, not reduced health care use.

http://annfammed.org/content/17/6/495">https://annfammed.org/content/1...

This does not preclude using RCTs to answer causal questions about pre-defined outcomes, but such studies use a different theoretical lens, and mixing methods and research philosophies should be done cautiously.

Brilliant table from same article from @trishgreenhalgh

Overall, different research questions require different methods. In complex systems, RCTs can only address a small proportion on what we need to know.

We need to rethink what defines "rigor" in health services research.

https://www.annfammed.org/content/12/5/447.long

END">https://www.annfammed.org/content/1...

Latest Threads Unrolled: