Read on Twitter
Do you like open science and reproducible research? 
https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤔" title="Denkendes Gesicht" aria-label="Emoji: Denkendes Gesicht"> Of course you do. You should read this thread.
#openscience #rstats #academia #linguistics #phonetics
#openscience #rstats #academia #linguistics #phonetics
I taught a graduate course (Spring2019) on open science and reproducible research for linguists https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤓" title="Nerd-Gesicht" aria-label="Emoji: Nerd-Gesicht">. The goal was to teach students in our SLA/Bilingualism PhD program about open science and the tools used to do reproducible research.
I wanted the course to be practical for students w/ limited programming exp. so they would finish feeling like more independent researchers and also be beneficial in the sense that they would walk away w/ something tangible, i.e. conf pres
I believe replication studies are particularly beneficial for pedagogical purposes, a position advocated by @TimoRoettger and Baer-Henney in https://doi.org/10.3765/pda.v1art4.13">https://doi.org/10.3765/p...
We decided to do a conceptual replication of a study in the journal Cognition, extending it to a new pop. (adult L2 learners)
https://doi.org/10.1016/j.cognition.2018.08.021">https://doi.org/10.1016/j...
https://doi.org/10.1016/j.cognition.2018.08.021">https://doi.org/10.1016/j...
We contacted the authors of the study we planned to replicate (ht Kalim Gonzales and @Krista_BH) and were able to get the original stimuli used in their experiment https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤝" title="Handschlag" aria-label="Emoji: Handschlag">.
We learned about the replication crisis in Psychology (and agreed it& #39;s likely much worse in #Linguistics https://abs.twimg.com/emoji/v2/... draggable="false" alt="🙀" title="Mattes Katzengesicht" aria-label="Emoji: Mattes Katzengesicht">) and some tools being used to make research more transparent. Here& #39;s a great video on the topic: https://youtu.be/0Rnq1NpHdmw ">https://youtu.be/0Rnq1NpHd...
Together we went through the previous lit and then drafted a pre-registration using #github and #OSF.
It was an ideal opportunity to learn the basics of #Git as well as some ins and outs of scientific collaboration.https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤗" title="Umarmendes Gesicht" aria-label="Emoji: Umarmendes Gesicht">
It was an ideal opportunity to learn the basics of #Git as well as some ins and outs of scientific collaboration.
Students learned about the preparation needed in the experimental design stages in order to complete a #prereg. We thought about our statistical analyses, power/sample size, stopping rules, etc.
We built the experiment using open source software, #PsychoPy3, and ran it online via #pavlovia and @gitlab
We finished collecting the data halfway through the semester and carried out our statistical analysis. We then began writing up a manuscript using #papaja in #Rmarkdown (ht @FrederikAust)
We were able to present preliminary results and got great feedback from @kleinschmidt and some of the members of his lab https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤜🏽" title="Nach rechts zeigende Faust (mittlerer Hautton)" aria-label="Emoji: Nach rechts zeigende Faust (mittlerer Hautton)">https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤛🏼" title="Nach links zeigende Faust (mittelheller Hautton)" aria-label="Emoji: Nach links zeigende Faust (mittelheller Hautton)">
It was a bit tight, but we finished the manuscript by the end of the semester and submitted it for publication on the last day of class https://abs.twimg.com/emoji/v2/... draggable="false" alt="😅" title="Lächelndes Gesicht mit offenem Mund und Angstschweiß" aria-label="Emoji: Lächelndes Gesicht mit offenem Mund und Angstschweiß"> We also submitted abstracts to two conferences.
The paper was under review for a little over a year https://abs.twimg.com/emoji/v2/... draggable="false" alt="🤬" title="Gesicht mit Symbolen über dem Mund" aria-label="Emoji: Gesicht mit Symbolen über dem Mund">, but we also uploaded a pre-print to @PsyArXiv and got some helpful feedback at a later stage (ht @TimoRoettger).
In the meantime we presented at the two national conferences. For some of my co-authors this was the first conference presentation for their CVs. The slides were HTML documents prepared using #xaringan in #Rmarkdown (ht @xieyihui)
Here comes the happy ending https://abs.twimg.com/emoji/v2/... draggable="false" alt="🥳" title="Partying face" aria-label="Emoji: Partying face">: in June (2020) the paper was accepted for publication!
The project lives on OSF so data, code, and Rmd files are available here: https://osf.io/cp9bs/ ">https://osf.io/cp9bs/&qu...
The preprint is available here on psyarxiv: https://psyarxiv.com/adkn8/ ">https://psyarxiv.com/adkn8/&qu...
And the publication in Studies in Second Language Acquisition is available here: https://doi.org/10.1017/S0272263120000273">https://doi.org/10.1017/S...
Congrats to all my co-authors (I believe @Crislocrispis is the only one on twitter) and thank you to Kalim Gonzales, @Krista_BH, @TimoRoettger, and @kleinschmidt.
Looking back on the whole process now, I am extremely happy with how everything worked out https://abs.twimg.com/emoji/v2/... draggable="false" alt="👍🏽" title="Thumbs up (mittlerer Hautton)" aria-label="Emoji: Thumbs up (mittlerer Hautton)">. I am planning on doing a replication study like this every time I teach this grad class.
My main takeaways: doing a replication study with a class takes a lot of work, but is worth it, it& #39;s good for science, and it& #39;s good for grad students. 10/10 would repeat. https://abs.twimg.com/emoji/v2/... draggable="false" alt="👊🏽" title="Fisted hand (mittlerer Hautton)" aria-label="Emoji: Fisted hand (mittlerer Hautton)">
