In April I tweeted a/b whether we had evidence for COVID-19 mental health crisis. See summary of longitudinal studies since then

I'm not a mental health expert, but watching COVID19 mental health studies has motivated me to reflect on many things for my research [thread] 1/ https://twitter.com/Daisy_Fancourt/status/1293826796295720960

1. Measurement consideration (h/t @farid_anvari)

I wonder more how the way people respond to measures. For ex: (in)equivalence of responses during COVID vs. pre-COVID.
How Q is asked also matters. For ex: asking "are you anxious right now?" vs. "does covid-19 make you anxious?"

2. How data is sampled?

I think more carefully about drawing conclusions from convenience vs. representative vs. probability samples, and become more skeptical about conclusions drawn from comparing means between groups unless sampling is probability.

3. How data is distributed

I started to look at data beyond the mean and SD, and looked more carefully at how data points are distributed. For ex: when it comes to mental health, does everyone change relatively uniformly or do we see a few extreme cases influencing the mean?

4. My criteria for evidence

I acknowledge I've been conditioned to see evidence rather than accepting inclusive evidence; so I find myself becoming more reflective of my effect sizes. For ex, can I say there is change if I see a jump of .21 on a 7-pt scale that yields p < .001?

5. Cumulation of evidence

I pay more attention to consistency across studies rather than focusing on "one hit wonder". It is a rewarding experience to start out with not knowing what I will discover and zooming in on certain evidence that keep consistently showing up in the data

6. Communicating my results

I had a few opportunities during this pandemic to talk w science writers/journalists about my research. I learn to be more confident in motivating the people I've spoke w to critically reflect on the nuances rather than clinging onto definite claims

The silver lining of this pandemic is how much I've grown as a scientist & adjusted my thinking a/b research

I feel more passionate a/b the process of discovery rather than the final result, and motivated to include those outside of my expertise to join me in that journey

End/