One of the most valuable things I’ve realised since starting my PhD is how many studies that have pretty terrible methodologies or unrepresentative samples are reported by the media in pretty absolute terms

This is Exhibit A for me: reported as “hard data” that “proves” something, but it no only does it not prove the thing the headline claims, the methodology is terrible (uses Twitter followers as a proxy for supporters!!) https://www.salon.com/2020/03/09/there-is-hard-data-that-shows-bernie-bros-are-a-myth/

I’m glad people want evidence, but the evidence has to be robust and meaningful, the methods have to stand up to scrutiny. So many research projects seem designed to confirm biases.

Another one that I saw yesterday was about the median household wealth of Black families in Boston being $8. I have no doubt that the number in reality is very, very low, but the data it was using had a sample of about 70. That’s just not enough to be representative.

So now, I try to find the sources. And often, the whole basis of the article is, if not completely obliterated, at least undermined.