Read on Twitter
New paper w/ @gsheag and @SpencerPiston out at APSR. In it, we advocate for pretest-posttest designs in which the DV is also measured prior to the experiment *in the same survey wave.* 1/5 https://www.cambridge.org/core/journals/american-political-science-review/article/abs/increasing-precision-without-altering-treatment-effects-repeated-measures-designs-in-survey-experiments/2E1637FFE36DFD0BA4AD0822634D8337
People, including us, have long worried that prepost designs might create consistency effects or otherwise affect results. But there hasn't been good, systematic evidence for these concerns. 2/5
In 6 experiments, we compare the prepost design to the standard posttest-only design and find no reliable differences in results. We find similar results for a "quasi" design in which a variable closely related to the DV is instead measured pretreatment. 3/5
The benefit is that the prepost design leads to large increases in the precision of our estimates. As a result, the design allows the detection of smaller effects (or use of smaller samples). These gains are much larger than those from standard controls (PID, ideo). 4/5
Finally, we discuss how prepost designs can help us understand our effects. Here we show that most partisans are unaffected by partisan cues and that most movement involves change in intensity rather than position. This could be particularly useful for studying backlash. 5/5
We encourage others to adopt the prepost design! But, of course, we can't generalize our results to all studies. It might not be wise when studying particularly sensitive topics, for example. Nonetheless, we think we've all been too paranoid. 6/5
