I'm curious to hear what y’all think about coronavirus models, peer review, and open science.

In general, the expectation with mathematical models in a field like epidemiology is that the authors post their code as supplementary material or on a site like GitHub,

1/n

so that their simulations can be replicated by others in the scientific community. This is virtually a requirement to get published in a reputable journal. Much of our policy approach to coronavirus has been shaped by a few influential reports,

2/n

for example, the Imperial College report by Fergusson et al. The Imperial College team has made the puzzling decision to delay posting their code. Fergusson tweeted the following: https://twitter.com/neil_ferguson/status/1241835454707699713?s=20

3/n

I think it is alarming that the code is thousands of lines of undocumented C based on a 13+ year-old flu model. It seems a blatant disregard of scientific standards to not immediately make this code available to the entire scientific community.

4/n

Ferguson makes an excuse for the delay, talking about hiring companies to clean it up and make a front-end for it. That is ridiculous. Modelers don't want a front-end. We want to see the code. We don't need perfect documentation, because we can read it,

5/n

match it up with the methodology described in the paper, and see how things fit.

Anyway, I think that this fundamentally undermines public confidence in modeling, and should be called out by the scientific community. Thoughts?

6/n, n = 6

I clearly set off a firestorm of interest with this thread.

Here is a Guardian article that discusses similar themes of open science and coronavirus modeling: https://www.theguardian.com/commentisfree/2020/apr/02/government-publish-data-coronavirus-deaths?