Read on Twitter
Do you like open science and reproducible research? 
Of course you do. You should read this thread.
#openscience #rstats #academia #linguistics #phonetics
Of course you do. You should read this thread. #openscience #rstats #academia #linguistics #phonetics
I taught a graduate course (Spring2019) on open science and reproducible research for linguists 
. The goal was to teach students in our SLA/Bilingualism PhD program about open science and the tools used to do reproducible research.
. The goal was to teach students in our SLA/Bilingualism PhD program about open science and the tools used to do reproducible research.
I wanted the course to be practical for students w/ limited programming exp. so they would finish feeling like more independent researchers and also be beneficial in the sense that they would walk away w/ something tangible, i.e. conf pres
I believe replication studies are particularly beneficial for pedagogical purposes, a position advocated by @TimoRoettger and Baer-Henney in https://doi.org/10.3765/pda.v1art4.13
We decided to do a conceptual replication of a study in the journal Cognition, extending it to a new pop. (adult L2 learners)
https://doi.org/10.1016/j.cognition.2018.08.021
https://doi.org/10.1016/j.cognition.2018.08.021
We contacted the authors of the study we planned to replicate (ht Kalim Gonzales and @Krista_BH) and were able to get the original stimuli used in their experiment 
.
.
We learned about the replication crisis in Psychology (and agreed it's likely much worse in #Linguistics 
) and some tools being used to make research more transparent. Here's a great video on the topic:
) and some tools being used to make research more transparent. Here's a great video on the topic:
Together we went through the previous lit and then drafted a pre-registration using #github and #OSF.
It was an ideal opportunity to learn the basics of #Git as well as some ins and outs of scientific collaboration.
It was an ideal opportunity to learn the basics of #Git as well as some ins and outs of scientific collaboration.
Students learned about the preparation needed in the experimental design stages in order to complete a #prereg. We thought about our statistical analyses, power/sample size, stopping rules, etc.
We built the experiment using open source software, #PsychoPy3, and ran it online via #pavlovia and @gitlab
We finished collecting the data halfway through the semester and carried out our statistical analysis. We then began writing up a manuscript using #papaja in #Rmarkdown (ht @FrederikAust)
We were able to present preliminary results and got great feedback from @kleinschmidt and some of the members of his lab 
It was a bit tight, but we finished the manuscript by the end of the semester and submitted it for publication on the last day of class 
We also submitted abstracts to two conferences.
We also submitted abstracts to two conferences.
The paper was under review for a little over a year 
, but we also uploaded a pre-print to @PsyArXiv and got some helpful feedback at a later stage (ht @TimoRoettger).
, but we also uploaded a pre-print to @PsyArXiv and got some helpful feedback at a later stage (ht @TimoRoettger).
In the meantime we presented at the two national conferences. For some of my co-authors this was the first conference presentation for their CVs. The slides were HTML documents prepared using #xaringan in #Rmarkdown (ht @xieyihui)
Here comes the happy ending 
: in June (2020) the paper was accepted for publication!
: in June (2020) the paper was accepted for publication!
The project lives on OSF so data, code, and Rmd files are available here: https://osf.io/cp9bs/
The preprint is available here on psyarxiv: https://psyarxiv.com/adkn8/
And the publication in Studies in Second Language Acquisition is available here: https://doi.org/10.1017/S0272263120000273
Congrats to all my co-authors (I believe @Crislocrispis is the only one on twitter) and thank you to Kalim Gonzales, @Krista_BH, @TimoRoettger, and @kleinschmidt.
Looking back on the whole process now, I am extremely happy with how everything worked out 
. I am planning on doing a replication study like this every time I teach this grad class.
. I am planning on doing a replication study like this every time I teach this grad class.
My main takeaways: doing a replication study with a class takes a lot of work, but is worth it, it's good for science, and it's good for grad students. 10/10 would repeat. 
