Read on Twitter
BREAKING! Retrospective study finds that men with prostate cancer have better survival with surgery vs. radiotherapy!
A thread on our newly published work quantifying treatment selection bias (TSB) in comparative effectiveness research (CER): https://www.nature.com/articles/s41391-020-00291-3
A thread on our newly published work quantifying treatment selection bias (TSB) in comparative effectiveness research (CER): https://www.nature.com/articles/s41391-020-00291-3
When analyzing cancer registry data, TSB skews estimates of overall survival (OS) when comparing treatment modalities. Ex. healthier patients may be selected for surgery and live longer than those selected for less invasive treatment like radiotherapy.
Researchers (hopefully
) all know that TSB exists in CER using registry data, but the magnitude of the effect of TSB on OS is extremely difficult to quantify in national registries. i.e. what does “surgical fitness” buy you in terms of 10-year OS? +2%? +5%?
) all know that TSB exists in CER using registry data, but the magnitude of the effect of TSB on OS is extremely difficult to quantify in national registries. i.e. what does “surgical fitness” buy you in terms of 10-year OS? +2%? +5%?
To estimate TSB effect, we studied a population uniquely suited for such a question: men with low-risk prostate cancer. Why? b/c 1) 10y prostate-cancer specific survival (PCSS) is ~100% and 2) randomized data from ProtecT (i.e. no TSB) showed equal OS and PCSS between treatments.
Thus, OS differences between treatments using registry data in this population are due to TSB. We used SEER (2005-2015) to compare OS and PCSS for men with low-risk prostate cancer receiving radical prostatectomy (RP), external beam radiotherapy (EBRT), or brachytherapy (BT).
Sure enough, despite ~99% PCSS for the SEER low-risk prostate cancer cohort, we found MASSIVE (>16%) differences in 10y OS between modalities: The more invasive the procedure, the better the OS (RP>BT>EBRT).
As you might expect, there were measurable baseline differences between patients receiving EBRT vs. BT vs. RP:
older age, less insurance, and more black patients were in the EBRT cohort (p<0.001) (also alluding to important disparities outside the scope of this study).
older age, less insurance, and more black patients were in the EBRT cohort (p<0.001) (also alluding to important disparities outside the scope of this study).
After controlling for these "measured" factors with propensity score matching, substantial OS differences persisted. We propose that TSB = measured TSB (controllable) + unmeasured TSB (cannot be controlled). Figure 4 shows the composition of the TSB effect on OS:
Yes, SEER lacks important covariates (i.e. performance status) which may change the composition of the TSB, but even in EXTREMELY detailed retrospective databases including functional information, TSB still persists https://www.redjournal.org/action/showPdf?pii=S0360-3016%2818%2933529-6
The bottom line: think twice 
when drawing conclusions from CER using registry data because TSB is lurking and hoping you'll make a false conclusion! And consider these large TSB effect quantities on OS (Figure 4) when interpreting prostate cancer CER using national registries
when drawing conclusions from CER using registry data because TSB is lurking and hoping you'll make a false conclusion! And consider these large TSB effect quantities on OS (Figure 4) when interpreting prostate cancer CER using national registries
Thank you to all my mentors and collaborators including @BenjaminKannMD @HenryParkMD @jamesbyu @DrPaulNguyen @sky__john @TalcottMd @VikramJairamMD @mleapman and others! Nothing better than working on an interesting project with colleagues that you genuinely enjoy.
