NHS analysts and policy people. a short thread with some questions about the data the system collects. Why is it so irrelevant and so poor?

...

The government is currently ruminating about the next spending review and wondering what it is worth trying to achieve.

Should they attempt to fix waiting lists or A&E performance...

And where would money have to go to do that?

Waiting lists and A&E are both troubled by a lack of free beds (can't admit electives if the beds are full and can't meet 4hr in A&E if no beds are available when you need them...

So, do free beds correlate with A&E performance? For example. Should more be spent to increase free bed numbers?

Good questions. Possibly question where good data might help...

Now we hit the big problem. The data is both irrelevant to the big problem and about as shonky as a three-legged donkey pulling a cart with square wheels...

The best public data we have on free beds comes from the KH03 returns hospitals have to make every quarter. This counts average available and average occupied beds...

Sounds useful. But quarterly? Less so. And the data is collected by measuring midnight occupancy and not measuring it very frequently (it certainly isn't average hourly occupancy)...

And A&E waits occur not because you have no free beds at midnight, but because there are no free beds at, typically, lunchtime when the peak emergency admissions happen...

Average quarterly free beds is not a very useful metric. It is quite possible to have plenty of free beds at midnight and none at lunchtime. And many hospitals don't even know their live, hour by hour, occupancy ever...

But let's look at the data anyway, just for fun.

This is what hospitals reported in calendar Q4 2019 (some are missing because trial sites for the new A&E target are not reporting)...

There is what looks like a vague relationship between free beds and A&E performance. But it is not statistically significant. Worse, it looks like thee might be a data quality problem...

Note all the trusts reporting >100% average bed occupancy. I don't think bunk-beds have become a thing in the NHS just yet, so this is of uncertain cause...

We can look at quarterly data going back to 2012. Maybe this will reveal a pattern or explain the odd data on bed occupancy.

Bugger...

The relationship is just as weak if you look at the underlying stats. And there are plenty of quarters over the whole period where many trusts report bed use >100%. ...

i know that some analyst somewhere has built a model fitting this to a queuing theory equation. But the visualisation whos the futility of that model: the data is a mess and there is little to reveal except noise...

But how can the government and DHSC decide what to do? would more beds help? And, if so, how many do we need?

Well, as the irishman asked for directions said, "I wouldn't start from here"...

Hospital capacity is complicated. A&E waits are complicated. And none of the long term metrics England publishes do much–if anything–to help us understand them...

If we really wanted good data on beds and A&E we would insist that hospitals collect real-time bed occupancy. Knowing the typical patterns across the day would tell far more about the nature and cause of the problem than KH03...

But, hospitals might say, "that would be a prohibitive operation burden." And they are right except that every industry outside health has automated the data collection for stuff like this. That costs, but the logistical benefits are worth it...

Amazon does not achieve next day delivery by sending teams round its warehouses twice a day to count the parcels and where they are. Hotels don't send a porter to find free rooms when you turn up at reception...

But that is what hospitals often do. And politicians encourage them to stay in the logistics stone age. "What we need is more doctors, not more computers or more managers" even if that means doctor productivity is 50% of where it should be...

And the policy makers can't find good ways to decide what policy would have the biggest effect. How can they when they have spent so long discouraging the tools that would give them reliable data to make decisions?...

Running the NHS well needs good evidence. Improving it needs better evidence. But it looks like nobody at the top has the wit to stand back and make that the first goal for the NHS...

So, as is too often the case, we will get major spending and policy decisions based on nonsense. Because being seen to do something (even ineffective) is far more politically important than gathering the data to do the right thing.