Thread by @zerdeve, I want to highlight some non-mainstream work on reproducibility, open science, replication [...]

I want to highlight some non-mainstream work on reproducibility, open science, replication crisis, meta-science by women. Reading and drawing from a diverse set of authors and ideas will help push this stream of work forward and help make science more open and inclusive.

I will only be able to highlight a few select work by these awesome scholars but follow them, look up their work, and cite them. And please do add what I have missed to my thread. I'll keep adding more as well.

1. @o_guest has written on open science and computational reproducibility. See for example https://twitter.com/o_guest/status/1230144174751698946?s=20 and this one with @andrea_e_martin https://www.sciencedirect.com/science/article/pii/S1389041713000314

https://twitter.com/o_guest/status/1230144174751698946?s=20

2. @IrisVanRooij has written on preregistration and the theory crisis. See https://featuredcontent.psychonomic.org/psychological-science-needs-theory-development-before-preregistration/ and https://twitter.com/IrisVanRooij/status/1233509131111325696?s=20

3. See also this paper on preregistration with @IrisVanRooij @djnavarro and @TrishaVZ among the authors https://psyarxiv.com/x36pz/

Is preregistration worthwhile?

Proponents of preregistration argue that, among other benefits, it improves the diagnosticity of statistical tests [1]. In the strong version of this argument, preregistration does this by solving...

https://psyarxiv.com/x36pz/

4. @djnavarro has written on the interplay between science and statistics https://link.springer.com/article/10.1007/s42113-018-0019-z and on preregistration https://djnavarro.net/post/paths-in-strange-spaces/

Paths in strange spaces, part I

The first half of a two-part post on preregistration. In this first half I argue against the proposition that preregistration is a good tool to prevent ‘p-hacking’. In the second half, I will switch...

https://link.springer.com/article/10.1007/s42113-018-0019-z

5. @chbergma has written on reproducibility https://onlinelibrary.wiley.com/doi/full/10.1111/infa.12182 and preregistration https://psyarxiv.com/es2gx , the latter together with @NaomiHavron

Preregistration in infant research - a primer

Preregistration, the act of specifying a research plan in advance, is becoming a central step in the way science is conducted. Preregistration for infant researchers might be different than in other...

https://onlinelibrary.wiley.com/doi/full/10.1111/infa.12182

6. @kirstie_j does great work on open science tools, resources, and workflow. See https://search.proquest.com/openview/f4042b3119a79ae8a561ba641bbed7f8/1?pq-origsite=gscholar&cbl=2045579 and https://twitter.com/kirstie_j/status/1118401811910746112?s=20

7. @cailinmeister does work on social structures in science, epistemic diversity, and false beliefs. See http://cailinoconnor.com/wp-content/uploads/2019/08/Misinformation_Groupthink.pdf and https://www.journals.uchicago.edu/doi/abs/10.1086/697744

8. @SabinaLeonelli has written on reproducibility as a criterion for research quality https://www.emerald.com/insight/content/doi/10.1108/S0743-41542018000036B009/full/html#loginreload

9. I am working on advancing a theoretical understanding of reproducibility and scientific process. See https://twitter.com/zerdeve/status/1128734143347445760?s=20 and https://arxiv.org/abs/1811.04525

https://twitter.com/zerdeve/status/1128734143347445760?s=20

10. See also this paper on robust modeling together with @_a_c__, @dora_matzke, Jennifer Trueblood, myself, and others https://link.springer.com/article/10.1007/s42113-019-00029-y

Robust Modeling in Cognitive Science

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform...

https://link.springer.com/article/10.1007/s42113-019-00029-y

11. https://twitter.com/chbergma/status/1234910959178854401?s=20

https://twitter.com/chbergma/status/1234910959178854401?s=20

https://twitter.com/rolandlangrock/status/1234922858062721024?s=21

13. @Nicole_C_Nelson does work on STS. See examples: https://estsjournal.org/index.php/ests/article/view/345 and http://nicolenelson.net/research/

14. @cantabile has written on mathematical reproducibility https://link.springer.com/chapter/10.1007/978-981-15-1960-4_8 and scientific workflow https://arxiv.org/abs/1910.06964 . Watch for more cool papers to come out of her dissertation work!

Truth, Proof, and Reproducibility: There’s No Counter-Attack for the C

Current concerns about reproducibility in many research communities can be traced back to a high value placed on empirical reproducibility of the physical details of scientific experiments and...

https://link.springer.com/chapter/10.1007/978-981-15-1960-4_8

15. @victoriastodden has a lot of work on computational reproducibility. See https://www.pnas.org/content/115/11/2584.short and https://www.crcpress.com/Implementing-Reproducible-Research/Stodden-Leisch-Peng/p/book/9781466561595 and look up the rest.

Implementing Reproducible Research

In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code...

https://www.pnas.org/content/115/11/2584.short

16. @LorenaABarba has written on reproducible computer science: See https://ieeexplore.ieee.org/abstract/document/8661798 and https://ieeexplore.ieee.org/abstract/document/8842605

17. Shirley Wang https://twitter.com/cailinmeister/status/1234972615552331779?s=20

https://twitter.com/cailinmeister/status/1234972615552331779?s=20

18. Don’t miss this article on open access publishing model by @CNSteltenpohl @AmyJAnd and Katherine Daniels https://www.gjcpp.org/en/article.php?issue=33&article=199

Wanted to make this thread bec of the apparent gender bias in the citation practices in the metascience/reproducibility lit. Many folks are not aware of the work done by women in the field. If you’re curious about your own citation patterns, here’s a tool: https://twitter.com/max_bertolero/status/1234965161062039560?s=21