Read on Twitter
How good are doctors at diagnosis?
This is the most relevant paper I have written. Not perfect but addresses a huge issue I think could change medicine if acknowledged
…it has changed how I think about diagnosis
https://ja.ma/3rQNtjv
@JAMAInternalMed
@drjohnm
1/n
This is the most relevant paper I have written. Not perfect but addresses a huge issue I think could change medicine if acknowledged
…it has changed how I think about diagnosis
https://ja.ma/3rQNtjv
@JAMAInternalMed
@drjohnm
1/n
summary:
Clinicians widely overestimated chance of disease especially after testing
Cardiac ischemia after + ECG—EBM 2-11%, median answer 70%
UTI after + urine cx—EBM 0-8.3%, answer 80%
Breast CA after + mammo—EBM 3-9%, answer 50%
Pneumonia after + CXR EBM 46-65%, answer 95%
Clinicians widely overestimated chance of disease especially after testing
Cardiac ischemia after + ECG—EBM 2-11%, median answer 70%
UTI after + urine cx—EBM 0-8.3%, answer 80%
Breast CA after + mammo—EBM 3-9%, answer 50%
Pneumonia after + CXR EBM 46-65%, answer 95%
Gerd Gigerenzer, David Eddy, @StevenWoloshin @arjunmanrai & others asked how well doctors do at the math of understanding diagnosis, and found they aren’t great.
Many of issues w/ real life tests covered by @deeksj @d_spiegel @dan_diekema
Many of issues w/ real life tests covered by @deeksj @d_spiegel @dan_diekema
We asked questions of diagnosis sense making and test interpretation for boringly common scenarios that often require some degree of patient shared decision making (and therefore require explanation of risk by doctors)
Thanks #NIHHighRisk @NIHDirector for DP2 funding
Thanks #NIHHighRisk @NIHDirector for DP2 funding
We assembled a big team that asked many hard questions up front, we finally used survey v34
Great collaborators (those on twitter)
@DKorenstein @AndrewFoy82 @masnick @SWeisenberg @ldscherer @LeykumLuci
Great collaborators (those on twitter)
@DKorenstein @AndrewFoy82 @masnick @SWeisenberg @ldscherer @LeykumLuci
We ask for interpretation of scenarios of pneumonia, UTI, cardiac ischemia and breast cancer screening.
All involved varying risk and questioned the use of first line testing.
No tricks, bread and butter cases
The challenge was estimating gestalt probability
All involved varying risk and questioned the use of first line testing.
No tricks, bread and butter cases
The challenge was estimating gestalt probability
We enrolled 553 doctors & NPs/PAs across 8 states w/ 76% response rate
(response high mostly d/t in person enrollment w/ in person follow up—frequently from me or collaborators—this was a LOT of work as survey not easy to complete)
(response high mostly d/t in person enrollment w/ in person follow up—frequently from me or collaborators—this was a LOT of work as survey not easy to complete)
We asked about a common scenario and the chance of a disease before testing and after a positive or negative result.
Patients have nuance, so here is description
We searched lit. for best evidence-based medicine (EBM) answer to compare
The graphs show distribution of answers
Patients have nuance, so here is description
We searched lit. for best evidence-based medicine (EBM) answer to compare
The graphs show distribution of answers
Cardiac ischemia
Pretest chance—EBM 1-4.4%, median answer 10%
Post stress + ECG—EBM 2-11%, median answer 70%
Post stress - ECG—EBM 0.4-2.5%, median answer 5%
Pretest chance—EBM 1-4.4%, median answer 10%
Post stress + ECG—EBM 2-11%, median answer 70%
Post stress - ECG—EBM 0.4-2.5%, median answer 5%
Urinary Tract Infection (UTI)—case of asymptomatic bacteriuria
Pretest chance—EBM 0-1%, median answer 20%
Post + urine cx—EBM 0-8.3%, median answer 80%
Post – urine cx—EBM 0-0.1%, median answer 5%
After + urine 71.1% would give Abx. After – urine 7.8% would give antibiotics!
Pretest chance—EBM 0-1%, median answer 20%
Post + urine cx—EBM 0-8.3%, median answer 80%
Post – urine cx—EBM 0-0.1%, median answer 5%
After + urine 71.1% would give Abx. After – urine 7.8% would give antibiotics!
Pneumonia
Pretest chance—EBM 25-42%, median answer 80%
Post + chest x-ray EBM 46-65%, median answer 95%
Post – chest x-ray EBM 10-19%, median answer 50%---After negative CXR 27.5% would NOT treat with antibiotics
Pretest chance—EBM 25-42%, median answer 80%
Post + chest x-ray EBM 46-65%, median answer 95%
Post – chest x-ray EBM 10-19%, median answer 50%---After negative CXR 27.5% would NOT treat with antibiotics
Breast cancer screening
Pretest chance—EBM 0.2-0.3%, median answer 5%
Post + mammo—EBM 3-9%, median answer 50%
Post – mammo—EBM <0.05%, median answer 5% (no change from pretest!)
Pretest chance—EBM 0.2-0.3%, median answer 5%
Post + mammo—EBM 3-9%, median answer 50%
Post – mammo—EBM <0.05%, median answer 5% (no change from pretest!)
Just like @arjun Medicine’s Uncomfortable Relationship With Math https://ja.ma/39TRPA8
and Casscells (NEJM 1978) when asked
The most common answer was 95%,
BUT 2% was correct
and Casscells (NEJM 1978) when asked
The most common answer was 95%,
BUT 2% was correct
And, these simple decisions each of us makes many times a day, across all clinicians = hundreds of thousands of decisions.
If we’re all biased towards big overestimates, this could explain a big portion of the ~1/3 of medical care that is #overuse #lowvalue care
@lowninstitute
If we’re all biased towards big overestimates, this could explain a big portion of the ~1/3 of medical care that is #overuse #lowvalue care
@lowninstitute
It is not that doctors are bad—but diagnosis and medicine are difficult.
The answer isn’t better math but a gestalt that is generally right
We need to use the clinical evidence we have to make diagnosis more scientific and not a simple “art” done differently by each doctor
The answer isn’t better math but a gestalt that is generally right
We need to use the clinical evidence we have to make diagnosis more scientific and not a simple “art” done differently by each doctor
Diagnosis is often an afterthought—list icd-10 codes with the right words for $$
If a code is present, make sure patient gets the medication or documentation for performance metrics
@fda & @cms have minimal test oversight @drJoshS https://ja.ma/2ReZOBf
If a code is present, make sure patient gets the medication or documentation for performance metrics
@fda & @cms have minimal test oversight @drJoshS https://ja.ma/2ReZOBf
The billing/metrics false-simplification of clinical medicine worsens patient care and is a barrier to nuance and #SDM @RichardLehman1
I think the answer lies with clinicians being encouraged to doctor @adamcifu , @Bob_Wachter
I think the answer lies with clinicians being encouraged to doctor @adamcifu , @Bob_Wachter
The first step is acknowledging we have a problem w/ #diagnosis
leading to a #diagnosticerror of COMission
not described much in @theNAMedicine report https://www.nationalacademies.org/our-work/diagnostic-error-in-health-care
@HardeepSinghMD
leading to a #diagnosticerror of COMission
not described much in @theNAMedicine report https://www.nationalacademies.org/our-work/diagnostic-error-in-health-care
@HardeepSinghMD
Solutions may not be that difficult if we:
-teach probability & clinical research > pathophysiology
-guide pretest probability—history/prevalence/exam
-teach test interpretation w/ natural frequencies NOT 2x2 table
https://ja.ma/3tpejRr
@VPrasadMDMPH
@AdamRodmanMD
-teach probability & clinical research > pathophysiology
-guide pretest probability—history/prevalence/exam
-teach test interpretation w/ natural frequencies NOT 2x2 table
https://ja.ma/3tpejRr
@VPrasadMDMPH
@AdamRodmanMD
cont..
-have reference clinical sensitivity & specificity http://testingwisely.com
-implement tests w/ dx stewardship https://ja.ma/2VGYs2F
-understand and accept #uncertainty ( @greyscalespaces)
-have reference clinical sensitivity & specificity http://testingwisely.com
-implement tests w/ dx stewardship https://ja.ma/2VGYs2F
-understand and accept #uncertainty ( @greyscalespaces)
