The trouble with sustainability science – I have been active in several sustainability modeling domains and have observed several others. I see common patterns in research going “wrong”, at least by my view of what it should be. Here are thoughts on what is going wrong and why.

1.Conflicting results: Researchers often find very different results for what should be the “same” thing. Does corn-based ethanol increase or reduce CO2 emissions compared to gasoline? Is the learning rate of wind positive or negative? Depends on whose paper you read.

It is normal in science to have conflicting results. The problem is a lack of 1. recognition of conflicts and 2. community debate to understand from where differences originate and 3. Development of best practices. Not saying this never happens, just not nearly enough.

2.Irreproducible results: There is a pervasive deficiency of reporting of models and data. Most papers I read don’t report nearly enough to make them reproducible. Many are egregiously lacking, with only high level discussion of method and a few aggregate/summary results.

The rise of online supplements to journal articles has helped some, but generally still vastly underutilized. This relates to the first issue because one can’t resolve why two results conflict if you don’t really know how each tackled the problem.

3.Lack of analysis and debate on uncertainty: Even in the journal Science, it is not hard to find papers claiming to estimate a new and complicated thing without once mentioning uncertainty, validation, caveats, or related concept. This just shouldn’t be.

Doubt is the most important ingredient in science. Quantification of that doubt is often beyond the reach of a single journal paper. But sustainability science poses many difficult questions, whose answers are uncertain. Admitting assumptions and caveats should be de rigueur.

The first step is to admit doubt. The next is to debate it. And someday, quantify and reduce that doubt. The dream is that as we now know the speed of light to many digits, we will someday be able to answer the messier questions of our future with some degree of certainty.

Next week: Thoughts on why the above is going “wrong”.