In an earlier thread, I said we can use the predictable nature of Whakaari eruptions to run a real-time alert system. In fact, we have been running one for the last 5 months. Here’s how it works. 1/17 https://twitter.com/DavidEDempsey/status/1284944999998615552

The building blocks are the precursor features found earlier. 4-hr RSAM bursts, 40-min RSAM burbling, peak RSAM and some others. We also know the precise date of 5 previous eruptions. 2/

Using data from 4 of those eruptions, we trained a Random Forest computer model to answer the question “given features computed from the last 48-hrs of RSAM, will an eruption occur in the next 48-hrs?” 3/

This type of model returns a number between 0 (“no”) and 1 (“yes”). The trick is to decide at what threshold to issue an alert – we ended up choosing 0.8 for reasons below. 4/

We tested the Random Forest model on data leading up to the unseen 5th eruption to understand what threshold would have been needed to issue an alert. For last year’s eruption, anything below 0.94 would have worked. The higher the threshold, the shorter the warning. 5/

Then, we repeated the model train-and-test (called cross validation) four more times, determining the minimum required alert threshold for each unseen eruption. The 0.8 value mentioned earlier would hypothetically have provided timely warning of 4 of the 5 eruptions. 6/

(The eruption in Oct 2013 was actually 3 eruptions over 8 days. At a threshold of 0.8, we catch 2 of them. So 5 caught eruptions out of 7 if you like.) 7/

This forecaster has trade-offs. Setting a threshold of 0.8 means that there is only a 1 in 12 chance for an eruption to occur if an alert is issued. On average, the alerts last 5 days and occur every 2 months. 8/

But what is the right threshold to use? Who should enforce the alerts? Will it ever be safe enough for Whakaari visits to restart? These are value judgments - scientists can help inform those decisions, but not make them. 9/

These systems are also not guaranteed to catch every future eruption. The 2016 eruption did not have similar precursors to the others and wasn’t caught using our approach. 10/

This was a pseudoprospective study. It emulated forecasting conditions (no use of ‘future’ data) but still had the benefit of hindsight. The gold standard test is to freeze the forecast model and evaluate it on future data. 11/

We have been evaluating a frozen model since February. There haven’t been any eruptions in that time and it hasn’t issued any alerts either. 12/

(It did almost trigger an alert in June. That was a bit of a butt-clenching moment until @scronin70 pointed out it had seen strong shaking from the M 7.4 Kermadec earthquake ) 13/

We are talking with GNS about whether a version of this forecaster would be useful for them. Our one runs on the @nectar_cloud, emailing and tweeting summaries each day or broadcasting if/when an alert is issued. 14/

We don’t know yet if this will work for other volcanoes. However, we have made the code open source in case it could be useful for other people to try. https://github.com/ddempsey/whakaari
15/

Next, we’ll be looking to see if it could work at Tongariro or Ruapehu, which receive tens of thousands of visitors each year. I took this photo flying over them last month (thanks to the pilots at @FlyAirNZ who know how to set a flight path). 16/

The loss of loved-ones at Whakaari last year is awful beyond words. We hope systems like these could help to avoid future tragedies, but there is some work to do yet. 17/17