Writing in the well-known Blue Top, the BMJ, Margaret McCartney, a fellow scourge of Bad Medicine, recently described the 16% higher chance of death if you are admitted to hospital over the weekend as a zombie statistic. The essence of a zombie statistic is not necessarily that it is wrong, but that it won’t go away, even when it is shown to be at least spurious, possibly wrong, and almost certainly misleading. Zombie politicians, who tend likewise to be at least spurious, probably wrong and almost certainly misleading, but still wont go away, love zombie statistics, as does the zombie press, which attracts zombie stats as a dunghill attracts flies. Shortly before the election, David Cameron, increasingly the zombie party leader as BJ hots up the mustard, pumped up the 16% higher mortality statistic, and true to form it just won’t go away. Today the zombie minister Jeremy Hunt will use the 16% zombie statistic to prop up his case for seven day zombie working in the NHS.
A picture will be painted of five-day consultants putting in weekend hours on the golf course while their admissions languish in hospital, getting sicker by the hour. Meanwhile, the 16% zombie statistic creates its own voodoo, a weekend nether-world hospital where zombie patients are admitted to zombie beds. While the consultants are on the links, Baron Samedi stalks the wards. Outside in the car park, under grey skies, porters are tarring chicken feathers to the telegraph wires. Inside, the nurses cabal in their offices, taking turns to stick needles into patient dolls. The weekend admissions have had their cards marked, by the grim emblem of death. Some way down the line waits Dr Foster, the grim reaper of hospital mortality statistics, which he turns into reports, which in turn generate the zombie statistics, zombie statistics about zombie patients.
What Dr No has just written is of course the product of an over-heated imagination. Mortality – typically the cut off is up to thirty days from admission – is higher for patients admitted over the weekend. But this higher mortality, which as far as we know is real, and has been found to apply in other health care systems run on very different lines to the NHS, is also almost certainly misleading, even spurious. As McCartney points out, the 16% increase is a relative risk, and when pitched against the absolute risk – 1.32% – of death following admission to hospital, the figure looks less alarming, though that is unlikely to be of comfort to patients concerned that the nurses are eyeing up a doll with their bed number on it. Indeed, savvy patients have been know to beg not to be admitted to hospital over the weekend for these very reasons. What makes the statistic potentially misleading is our natural and proper tendency to impute, or, if you are a zombie minister, ascribe, reasons, or causes, for the higher mortality among weekend admissions.
This is our old friend ‘association is not causation’ at work again. Weekend admission, and attendant circumstances, like a hospital running at less than full throttle, may be associated with higher mortality, but that does not prove that the attendant circumstances caused the higher mortality, no matter what zombie ministers may want us to believe. Another perfectly plausible cause is that patients admitted over the weekend are iller and sicker. Perhaps other NHS services, such as primary care and palliative care, are less accessible over the weekend, pushing up acute hospital admissions, some of whom are moribund. Rather curiously, although you are more likely to die at some point if you are admitted over a weekend, mortality itself actually falls a bit over the weekend, with Wednesday being the peak day for deaths. Those savvy patients who beg not to be admitted over the weekend would do as well to beg to be discharged by midnight Tuesday.
The 16% higher mortality figure’s potential to mislead is further aggravated because it is based on fancy stats done on dubious data. Dr No has warned before that if the statistical working presented in research is beyond the ken of the averagely intelligent scientifically literate and numerate reader, then the researchers can say what they like, and the rest of us have no viable means of verifying it. If Dr No tells you that he has used a binomial proportional risks hazards adjusted log transformed multivariable accelerated cubic spline Box logistic regression model to predict that earthworms from one end of his garden have a 16% chance of being eaten by a mole before they reach the other end of his garden, there is not much you can do to challenge that finding, unless you happen to know a lot more about fancy stats than Dr No does.
Even if by a stroke of luck or genius, the researchers have managed to dial the right comprehensible analysis into their statistical programme, the GIGO – garbage in, garbage out – problem remains. The data behind the 16% figure are the infamous hospital episode statistics. The numbers are huge (millions of episodes, which means p values will be very small, which in turn allows the researchers to say their results are ‘highly significant’ – highly statistically significant, that is, which is just another way of saying the results are unlikely to have arisen by chance, but on its own that tells us nothing about cause); but while hospital episode statistics deliver on quantity, they tend to fail on quality.
Factors like the patients age and sex are relatively easy to get right (and age in particular matters, because age is a major determinant of mortality – if we admit 10% more older patients over the weekend, we might not be surprised to find 10% higher mortality among those weekend admissions) but, crucially, recording accurate data on how ill patients are on admission, and how many pre-existing, and yet to be discovered co-existing conditions, the patient has is difficult, and the data notoriously unreliable. If, as seems likely, we admit a higher proportion of iller patients over the weekend, then perhaps we should expect a higher mortality among those admissions, without a need to impugn poorer weekend care as the cause for the higher mortality.
The grand-mother of all the English NHS studies on mortality by day of admission, and the source of the 16% figure, is that published by Pagano et al in 2012. Analysing over 14 million admissions, the headline result was a ‘hazard ratio for Sunday [admission] vs Wednesday [of] 1.16’. So far, so straight-forward (the 16 after the decimal point is what became the 16%). Make sure you get admitted on Wednesday. Presumably Saturday admissions, and possibly Friday, especially late on Friday, admissions also have higher mortality, and Monday admissions the lowest (a whole working week to sort the patient out before heading for the golf course). Actually, no. Saturday admissions do have an 11% increased mortality at 30 days compared to Wednesday admissions, but after that the wheels start coming off the golf cart. Not only is actual mortality by day of the week slightly lower at weekends (hullo? if care is poorer over the weekend, surely the mortality rate should go up at the weekend?), but when analysed by day of admission, Fridays – the day before the weekend – are one of the admission days with the lowest mortality, while Monday admissions, with the longest stretch of full throttle hospital working ahead of them, actually have a slightly increased mortality, by 2%.
These weekday admission mortality figures, particularly for Monday and Friday admissions, do not sit comfortably with the poor care at weekends kills patients hypothesis. On the other hand, a plausible hypothesis might be that Monday admissions include a number of ill-over-the-weekend-but-leave-it-to-Monday-morning admissions, but that does nothing to explain the lower mortality for Friday admissions, patients whose acute immediately after admission care might well run into the weekend. All in all, although we know day of admission is associated with mortality risk in the following 30 days, we don’t know how or why; instead, we have zombie hypotheses chasing zombie statistics. Rather than pumping up seven day working and annoying consultants, the zombie minister for zombie patients would do better to commission a proper prospective study that includes decent admission data on how ill the patient is at admission, with a view to determining what it is that marks the cards of weekend admissions. Once the real cause of the higher mortality is better known, the proper and practical steps, if there are any, to lower the risk can be taken. Merely ordering consultants onto the wards at weekends – a zombie policy chasing a zombie statistic – may even backfire: zombie consultants chasing zombie patients. May have legs as a horror movie proposition, but no way to run the NHS.