Read on Twitter
Some interesting (to me) details about the retracted Clark et al. article. One thread per detail.
This is thread #1: The preregistration.
(There may only be one thread, depending on my motivation levels; it's a beautiful day here and the park beckons.) /1
This is thread #1: The preregistration.
(There may only be one thread, depending on my motivation levels; it's a beautiful day here and the park beckons.) /1
Clark et al. received Psychological Science's "preregistered" badge. Here are the criteria for awarding that badge, from the journal's web site. /2
Let's look for the results of any preregistered analyses in the article. The strings "prereg" and "pre-reg" find only a few hits. Here's the first: a keyword! Strong start there. This article is clearly going to be a feast of open science practices. /3
(By the way, that image also contains the date on whiuch the manuscript was received: January 25, 2019. Keep that in your mind.) /4
The next three mentions of "prereg" are in the Methods section, where for some reasons the authors seem to have become somewhat lukewarm about preregistration. ¯\\_(ツ)_/¯ /5
Still, there must be plenty of other examples, right? After all, "preregistered" is a keyword. Let's find the next occurrence of "prereg".
Oh. It's in the coda.
Still, we get a link to this doubtless extensive preregistration. I might start getting excited again. /6
Oh. It's in the coda.
Still, we get a link to this doubtless extensive preregistration. I might start getting excited again. /6
Here's the preregistration. It was "preregistered" on June 3, 2019. That's 130 days ***after the manuscript was submitted***. It describes a preregistered analysis of... the possible effect of adding an extra explanatory variable to the models that are already in the paper. /7
So the article got the "preregistered" badge (arguably the journal's problem), and the authors themselves used the keyword "preregistered", on the basis of exactly one exploratory analysis, suggested by the reviewers, and written up in the supplement as having unclear effects. /8
Aside: A friend who works for a large consumer products company once told me that when they make a 10,000-litre batch of their "herbal extract" shampoo, they add 10 millilitres of actual herbal extract to the vat. That's what this feels like. Semi-homeopathic open science. /9
I hope the reader will forgive me if I say that my reaction to this is more likely to be spontaneous combustion than spontaneous applause. /10 /end
