Read on Twitter
I had the opportunity to write a comment about findings from a large vaccination cohort study in Scotland. I use the comment to discuss some of the challenges of observational vaccine studies and the potential for lingering bias.
THREAD 1/11 https://twitter.com/TheLancet/status/1386639838162599939
THREAD 1/11 https://twitter.com/TheLancet/status/1386639838162599939
Confounding is a key challenge in observational studies. One way to gain confidence in findings is to check for bias in results where we know the answer (usually, where we know there is no relationship). 2/11
. @_MiguelHernan describes using this approach in an Israeli cohort. They verified that they didn't see a protective effect of vaccines earlier than observed in randomized trials. At first they did see this, so they adjusted for more covariates. 3/11 https://twitter.com/_MiguelHernan/status/1364700322892034048?s=20
In the Scotland study in the Lancet, they observed an early benefit against hospitalization within 0-6 days after hospitalization. This is not biologically plausible, so it suggests some residual bias. 4/11
https://www.sciencedirect.com/science/article/pii/S0140673621006772?via%3Dihub
https://www.sciencedirect.com/science/article/pii/S0140673621006772?via%3Dihub
The authors discuss how people were advised to take strict precautions prior to vaccination, so this may explain the initial benefit. They posit that this benefit is transient and so later findings are more reliable, but there may be other sources of confounding. 5/11
There is some signal that the nature of this (hopefully transient) bias varies across age groups. We see different 7-13 day effects for 18-64 year olds than for 65+ year olds. 6/11
This presents another challenge, in that the two vaccines studied (Oxford/AZ, Pfizer/BioNTech) were deployed differently in Scotland. Pfizer/BioNTech was rolled out first. When both were available, Oxford/AZ was largely used in care homes/for 80+, and Pfizer largely in HCWs. 7/11
Although we can adjust for age, it does makes it harder to compare the two vaccines directly. This preprint from Public Health England does a nice job stepping through some of the challenges. 8/11 https://www.medrxiv.org/content/10.1101/2021.03.01.21252652v1
In the above preprint, instead of seeing an early protective effect of the vaccine, they observe the opposite, suggesting that people who were at baseline HIGHER risk were being targeted for vaccine. 9/11
Over time, they see this effect lessen, with reduced evidence of bias. They suggest that this occurred because the initial vaccine rollout was highly targeted at high risk individuals, but as vaccines became more widely available, these differences were less notable. 10/11
Observational vaccine studies are a valuable source of information, but they're challenging! Bias lurks in many corners. So integrating bias checks (like the type in this great cholera study 
) is a smart strategy to increase confidence in results. 11/11 https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(14)70368-7/fulltext
) is a smart strategy to increase confidence in results. 11/11 https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(14)70368-7/fulltext
