An epidemiologist's primer to DIAGNOSTIC TESTING

- sensitivity & specificity
- positive and negative predictive values
- likelihood ratios
- pre- and post-test probabilities

Sound familiar?
Sound really confusing?

In this I'll try to tackle how these terms fit together.

1/

AGREEMENT ON TERMS AND ABBREVIATIONS

- sensitivity (SENS)
- specificity (SPEC)
- positive predictive value (PPV)
- negative predictive value (NPV)
- likelihood ratio positive (LRP)
- likelihood ratio negative (LRN)

2/

SENS & SPEC

- when we speak of COVID-19 viral tests like PCR and antigen tests, we want them to be ACCURATE

- SENS and SPEC are the accuracy characteristics of your test

- to measure these, we need to know whether people actually have the outcome of interest (disease)

3/

SENS & SPEC (cont'd)

- SENS reflects how good the test does in returning a POSITIVE result in people who actually have the "disease"

- SPEC reflects how good the test does in returning a NEGATIVE result in people who actually DO NOT have the "disease"

4/

SENS & SPEC (cont'd)

- a SENS = 90% means that of every 100 people WITH the "disease", the test reads + for 90 (it gets 10 wrong)

- a SPEC = 95% means that of every 100 people WITHOUT the "disease", the test reads - for 95 (it gets 5 wrong)

5/

For SARS-CoV-2 viral tests, you might see metrics like:

PCR
- 98% sensitivity
- 99.9% specificity

Point-of-care antigen (best-case)
- 96% sensitivity
- 97% specificity

These metrics are used to say that PCR is a "more accurate" test than currently available antigen tests.

6/

DIAGNOSTIC ABILITY IN THE "REAL WORLD"

- What is most valuable to clinicians/public health is how a test with a certain SENS and SPEC will perform when administered to people whose true "disease" status is unknown.

7/

YOU BE THE DOCTOR

- a patient comes in from the community
- prevalence of SARS-CoV-2 is 5% in the community
- seems like an "average person"

Before giving the test, you might guess their chance of having SARS-CoV-2 is 5%

- this is the PRE-TEST PROBABILITY

8/

HOW DOES THE TEST HELP?

- the test should help you revise the likelihood that the patient has SARS-CoV-2

- this is the POST-TEST PROBABILITY (PTP)

For a test with 100% SENS & SPEC (perfect accuracy)
- a POS result would PTP to 100%
- a NEG result would PTP to 0%

9/

NOMOGRAM

- using a schematic to illustrate what a test should do

- let's say patient has 50% chance of disease BEFORE the test (coin flip)

- you have a test with 90% SENS & 90% SPEC

- a POS result chance of disease to 90%
- a NEG result chance of disease to 10%

10/

NOMOGRAM (cont'd)

- a better test would give you more confidence

- Increase SENS & SPEC from 90% to 99% each

- a POS result chance of disease to 99%
- a NEG result chance of disease to 1%

Not perfect confidence, but MUCH better because test is more accurate.

11/

IMPACT OF PREVALENCE (PRE-TEST PROBABILITY) ON TEST PERFORMANCE

- take the same scenario from above (99% SENS & SPEC), but instead of a 50% pre-test probability, it's 5%

- a POS result chance of disease to 84%
- a NEG result chance of disease to 1 in 2000

12/

Pre-test prob is also based on patient factors, but let's assume it's an avg person (so we use prevalence)

- Higher prev improves how much a + test result helps (FALSE +)

- In the prev example, the low prev (5%) meant that 16% of people with + test were actually NEG!

13/

WHAT IS THE POSITIVE PREDICTIVE VALUE?

- PPV is the % of people with a + test result who actually have the disease

- PPV is the same as the post-test probability after a + test result

- The false POS rate is (100% - PPV)

- Lower prevalence -> PPV and false + rates

14/

WHAT IS THE NEGATIVE PREDICTIVE VALUE?

- NPV is the % of people with a - test result who actually do not have the disease

- NPV is the same as 100% minus the post-test probability of disease after a - test result

- The false NEG rate is (100% - NPV)

15/

WHAT IS LIKELIHOOD RATIO POSITIVE (LRP)?

- You have a test with SENS=90% & SPEC=97%
- Patient's pre-test probability of disease = 5%

- The LRP=30, which means a + test result the odds the patient actually has the disease by a factor of 30!

- The higher the better!

16/

WHAT IS LIKELIHOOD RATIO NEGATIVE (LRN)?

- Same ex as

Prob of disease
- Pre-test: 5%
- Post-test: 0.5%

- The LRN=0.1, which means a NEG test result the odds the patient actually has the disease by a factor of 10 (1/0.1)!

- The lower (closer to 0) the better!

16/

WHY DO WE HEAR THAT SPECIFICITY "RULES IN"?

- SPEC means fewer false +
- for the same prevalence, SPEC results in higher PPV
- so, a + result gives strong evidence the patient has the disease (rules in)

SPEC more important for minimizing FALSE POSITIVES

17/

WHY DO WE HEAR THAT SENSITIVITY "RULES OUT"?

- SENS means fewer false -
- for the same prevalence, SENS results in higher NPV
- so, a - result gives strong evidence the patient does not have the disease (rules out)

SENS more important for minimizing FALSE NEG

18/

EXAMPLE: DIFFERENCE IN PCR AND ANTIGEN (1)

Assume:
- PCR: 98% SENS, 99.9% SPEC

Prevalence = 5%

Administer to 100,000 people

4,995 will test + (95 will be wrong, a false +)
- PPV 98%

95,005 will test - (100 will be wrong, a false -)
- NPV 99.9%

19/

EXAMPLE: DIFFERENCE IN PCR AND ANTIGEN (2)

Assume:
- Antigen: 96% SENS, 97% SPEC

Prevalence = 5%

Administer to 100,000 people

7,650 will test + (2,850 will be wrong, a false +)
- PPV 63%

92,350 will test - (200 will be wrong, a false -)
- NPV 99.8%

See the diff

20/

AGAIN: PREVALENCE MAKES A DIFFERENCE

At 5% prev:
- PCR: 98% PPV, 99.9% NPV
- Antigen: 63% PPV, 99.8% NPV

Increase true prev to 15%:
- PCR: 99.4% PPV, 99.6% NPV
- Antigen: 85% PPV, 99.3% NPV

This is why, for many tests, we use in a "high-risk" group. prev -> PPV

21/

Often, although we want the highest possible SENS & SPEC, there may be a trade-off.

Do you want to minimize false + or false - MORE?

Depends on situation. Think about HIV tests.
- the stress of a false +
- the ramifications of a false -

22/

Just scratches the surface, but covers key terms people have been asking me about.

TL;DR
- Tests with SPEC better to FALSE POSITIVES
- Tests with SENS better to FALSE NEGATIVES
- The % of false + can get when prevalence is , even for test w/ great accuracy

23/

Those who MAY care to share the diagnostic testing primer

@nataliexdean @EpiEllie @zorinaq @_stah @aetiology @meganranney @EricTopol @berthahidalgo @jessicamalaty @bethlinas @jaimiegradus @michaelmina_lab @DrTomFrieden @angie_rasmussen

24/