“The models were all wrong.”

I have been seeing this argument a lot online, sometimes on Twitter but more often on Facebook, as a claim meant to support the idea we shouldn’t be sheltering in place at the expense of the economy.

This is dangerous and wrong thinking.

I am not an epidemiologist and don’t work with those models. But I have done enough study and working with models to know how they work.

The problem isn’t the models, but rather the public’s understanding of them (and quite possibly how reporters cover model projection claims).

Models provide a range of projection based on certain inputs we call variables. They are constructed based on factors we know about, and then those factors have a numerical value based on data we have.

So right there: how robust are the imagined inputs, and how good is the data?

Inputs can be things like demographics, population density, underlying population health, etc. It’s a mix of social science and science, and it is hard. Epidemiologists spend years trying to figure out the most critical factors when building models, and weight them properly.

But even if your model is good, bad data screws it up.

Witness what’s happened in Georgia with messed up reporting. https://bit.ly/2LKS6IY 

Or Florida, with claims the state is encouraging tainted data reporting. https://bit.ly/2yfkDmX 

But even if the data is good, here’s the thing: any change to inputs affects the model.

People saying “We were told 6m deaths, then 2m deaths, then 500k deaths! They were all wrong!” misses the point.

We changed the inputs. We shut things down. That changes the model.

My statistics profs in grad school said several times, “All models are wrong, but some are useful.”

The point was, we change the model as we learn more. We understand factors better, and which factors matter most. We get better data to understand the impact.

The point is, the model being wrong is a feature, not a bug.

There. Is. No. Such. Thing. As. A. Static. Model.

So how can the public better understand models through better reporting? Here is what I’d suggest:

1. Models have ranges. The death toll projection is not the headline. They give a high/low range because models aren’t certain.

The public needs to understand this, and that it’s a best guess based on understood factors.