Read on Twitter
The following may be of interest to those who use Prolific and/or Lucid for surveys.
Ran a study yesterday about election-related opinions (plus some other stuff - data is a bit depressing, coming tomorrow) using Lucid & Prolific's "nationally representative" sample function...
Ran a study yesterday about election-related opinions (plus some other stuff - data is a bit depressing, coming tomorrow) using Lucid & Prolific's "nationally representative" sample function...
Both sources use quota-matching to filter people into studies who match U.S. demo's on age, gender, ethnicity, and (for Lucid) region. However, there were some notable differences and similarities between the samples.
(Note: Target N for each was 500, study was ~10 min long.)
(Note: Target N for each was 500, study was ~10 min long.)
I included a very simple initial attention/bot check: "Puppy is to dog as kitten is to _____?" with an open-ended text box to respond. This came at the very start. Two other fairly simple attention checks came later in the survey. We also asked directly if ppl responded randomly.
Lucid:
24% failed cat Q.
6% of the resulting sample did not finish.
27% failed 2 out of 3 screeners
Prolific:
1% failed cat Q (5 out of 503 people)
1% of the resulting sample did not finish
2% failed 2 out of 3 screeners.
24% failed cat Q.
6% of the resulting sample did not finish.
27% failed 2 out of 3 screeners
Prolific:
1% failed cat Q (5 out of 503 people)
1% of the resulting sample did not finish
2% failed 2 out of 3 screeners.
So, Prolific looks pretty good there. However, if you're interested in politics (as I am), the following isn't great
Lucid:
49.8% Biden voters, 36.3% Trump voters
42.5% Dems, 37.5% Reps, 18.3% Ind
Prolific:
67% Biden voters, 18.4% Trump voters
56.3% Dems, 16.8% Reps, 23.3% Ind
Lucid:
49.8% Biden voters, 36.3% Trump voters
42.5% Dems, 37.5% Reps, 18.3% Ind
Prolific:
67% Biden voters, 18.4% Trump voters
56.3% Dems, 16.8% Reps, 23.3% Ind
A key Q is whether the Republicans on the two platforms differ. Turns out they sorta don't (caveat: N's aren't huge, ~130 for Lucid & ~80 for Prolific). Age & education are very similar. Prolific Reps are more economically conservative but equally socially conservative
However, there are larger differences b/w the samples when it comes to Democrats and Independents. In short, both groups are more liberal on Prolific than on Lucid.
Thus, comparisons with Republicans will be (likely artificially) inflated in Prolific samples!
Thus, comparisons with Republicans will be (likely artificially) inflated in Prolific samples!
Happy to answer further questions if people are curious. Study was run with @DG_Rand, who suggested I write this thread and is therefore responsible for interrupting people's celebratory tweeting
