Read on Twitter
@HIQA report into excess deaths in Ireland is almost brilliant. Commendable use of complex statistical techniques and presentation of results.
One big problem: the highly precise number of 1,072 excess deaths is likely an underestimate based on techniques used.
Here's why
1/ https://twitter.com/HIQA/status/1278958011697057792
One big problem: the highly precise number of 1,072 excess deaths is likely an underestimate based on techniques used.
Here's why
1/ https://twitter.com/HIQA/status/1278958011697057792
Calculating excess deaths requires two things: actual deaths and expected deaths.
Actual deaths tend to be (relatively) easy to calculate. Excess deaths are a little more tricky.
2/
Actual deaths tend to be (relatively) easy to calculate. Excess deaths are a little more tricky.
2/
HIQA have decided to take a time-series approach to calculating expected deaths. If you're not familiar with time-series analysis, it looks at data points over time to try predict or estimate what will happen next.
This is generally fine, but really not in the case of RIP.
3/
This is generally fine, but really not in the case of RIP.
3/
The ratio of RIP to CSO deaths has been increasing over time as things move more online. We can't expect this ratio to continue increasing indefinitely, but this feature leads to issues with time-series analysis and will tend to overestimate expected deaths for 2020.
4/
4/
It's also great when researchers publish sensitivity results. We can see the model is based on data from 1/1/15 to 18/2/20. Bizarrely, if the final 6 weeks of data are removed from the model (an entirely reasonable assumption), excess deaths would be 45% lower (or 485 fewer).
5/
5/
It shows just how sensitive HIQA's analysis is to the time-series model selected.
The criteria for model selection was the model with the lowest AIC (generally a solid statistical decision), but you have stand back and ask yourself, "does this make sense"?
6/
The criteria for model selection was the model with the lowest AIC (generally a solid statistical decision), but you have stand back and ask yourself, "does this make sense"?
6/
Looking at weekly mortality, results are much more stable (albeit still with some bias regarding trends). They are also higher than the summary figure presented in the paper.
Weekly mortality appears to remove much of the volatility caused by daily numbers.
7/
Weekly mortality appears to remove much of the volatility caused by daily numbers.
7/
And this is before we get to the issues with using RIP data. It's not strictly correct to present the HIQA analysis as excess deaths, but rather excess posts to a 3rd party funeral website that doesn't capture everyone.
But it's the best available data we have.
8/
But it's the best available data we have.
8/
I've done some analysis using the FT/ @jburnmurdoch's methodology. This suggests we had ~1,550 'excess posts'.
This method, however, also has its flaws and likely overstates excess deaths.
So perhaps a better range is 1,250 to 1,500 excess posts.
9/ https://twitter.com/IrishDataViz/status/1274643762485002249?s=20
This method, however, also has its flaws and likely overstates excess deaths.
So perhaps a better range is 1,250 to 1,500 excess posts.
9/ https://twitter.com/IrishDataViz/status/1274643762485002249?s=20
Another issue with the paper is it claims we're currently experiencing a period of lower deaths than expected. But this is based on HIQA's expected deaths which we've seen has issues.
10/
10/
There is another excellent thread on the issues with the HIQA analysis here:
11/ https://twitter.com/Mad_iguana/status/1278990273356484608?s=20
11/ https://twitter.com/Mad_iguana/status/1278990273356484608?s=20
So what is Ireland's excess mortality? We don't know, and won't for some time to come. We need to wait for the CSO's official figures for this period along with some actuarial analysis of what expected deaths would have been. This is still many months away however.
12/
12/
So in summary, our media's interpretation and downplaying of our death figures has been poor at best. While we have possibly overstated our deaths attributable to COVID-19, Ireland's excess deaths are likely greater than what the HIQA report suggests.
13/
13/
The general statistical analysis of the available data is highly impressive however and the final conclusion of the HIQA report is one we can all agree with: monitoring of excess mortality would benefit immensely from more timely death registrations.
14/14
14/14
