We (epi & biostat folks) have a major conflict of interest when it comes to evaluating COVID research that I’m not sure we’ve fully acknowledged. 1/

Mostly, when we assess studies or evidence, we are at arm’s length from the problem we’re studying. Yes, we may know people who have a condition, but our findings don’t have implications for our lives. 2/

Indeed, we are justifiably skeptical of whether someone who is heavily invested in the outcome of a study can be objective in evaluating its quality. This is why we have COI declarations. 3/

But in COVID times, we are regularly asked to evaluate the quality of evidence in studies whose outcomes could have a major impact on our own lives. 4/

There’s no better alternative, but it does complicate the scientific dialogue. 6/

Most of us have strong views about what would best for our lives: send kids to school (or not), visit family (or not), etc. These views are Informed by our own personal degree of risk tolerance. 7/

Our assessments of the strength of evidence on various COVID-related topics will inevitably be filtered through this risk tolerance lens. 8/

For example, if you have a low risk tolerance, you are probably more likely to be critical of a study which presents imperfect evidence suggesting that a particular activity is relatively safe. 9/

This isn’t at all irrational; since the definition of “safe” is multi-dimensional, it is entirely reasonable to demand stronger evidence to be convinced that the level of risk is below your (low) personal threshold. 10/

Faced with the same evidence, someone with a higher risk tolerance might be equally justified in saying that the evidence is sufficient to conclude that the activity in question is sufficiently safe for their personal purposes. 11/

The problems arise when we bring these arguments to the public sphere. Two experts might actually agree about the strength of evidence, but argue for different conclusions because of their own personal risk lens. 12/

I don’t have any solutions, other than to suggest that we need to acknowledge that we are all in conflict of interest when it comes to COVID, and should think about how that affects the way we do our science. end/

Wanted to highlight some great points that people have made: 1) Many people work on problems that affect them, particularly researchers of color and those identifying as LGBTQ. We have a lot learn about personally-engaged research from these scientists.

2) Similarly, it is possible to be personally invested in one’s research without it being your lived experience.

3) The term “conflict of interest” isn’t perfect here, and brings a lot of negative connotations with it that I didn’t mean to imply.

4) This thread reflects my own impressions based on the public discussions I’ve seen around COVID. My perspective is doubtlessly North American-centric and leans more to the biostat side of the epi/biostat line.

Thanks to everyone who has added to the discussion!