The Hospital Manager’s Association
Top Secret – Eyes Only
The Hospital Manager’s Guide to Massaging HSMRs
Members will be aggrieved to hear that the Doctor Foster Intelligence Unit and its lottery hospital standardised mortality ratios (HSMRs) are here to stay, despite several recent papers showing the methodology to be unsound.
Members will appreciate that they supply the raw data used by Dr Foster, thus providing opportunities to ‘cook’ the figures before they are passed to Dr Foster. The Association does not condone directly tampering with the data; however, faced with the intractable use of flawed statistics, the Association does believe members are entitled to ‘game’ the system to their advantage.
Members will further recall that the HSMR is the ratio of observed deaths to expected deaths, conventionally multiplied by 100. The secret of successful gaming is not only to reduce observed deaths, but also to arrange matters so as to increase the number of expected deaths. This can simply be achieved by weighting matters in such a way as to favour adjustment factors – such as age and co-morbidities – that increase expected mortality.
The Association has researched options for members, and recommends any or all of the following:
1. Get rid of dying patients. Nothing boosts HSMRs more than dead patients – so dump them before they die. It doesn’t matter where you dump them – just get them off the ward before they die. Try tapping into the nostalgia for dying at home – start a campaign through your local paper.
2. Shift dying patients that can’t be dumped into ‘unseen groups’. Remember, Dr Foster only includes only 56 diagnostic groups that account for 80% of in-patient deaths. If a patient’s diagnostic group falls outside the 56 that are included, the death won’t be ‘seen’ by Dr Foster.
3. Rack up those co-morbidities. HSMRs are heavily adjusted for case-mix (concurrent illnesses), so the more co-morbidities the better. Consider taking on extra clerical staff to review case notes for missed co-morbidities.
4. Admit as many old but not expected to die patients as possible – so called ‘Golden Oldies’. It may make the wards smell a bit more than usual, but it does wonders for lowering HSMRs.
5. Avoid elective admissions like the plague. Maintain bed occupancy as 110% so as to force cancellation of elective admissions. Discretely facilitate bed-blocking with old (but not moribund) patients. Liase with local Social Services to ensure that discharge arrangements are slow and complicated.
6. Dr Foster now adjusts for palliative care. Open a large palliative care unit and admit as many dying patients as possible to it. A large high profile palliative care unit has the added bonus of making your hospital appear trendy and caring.
Finally, always bear in mind the most enthusiastic gaming can fail, and a high HSMR result. In such circumstances, the only option is to discredit the Dr Foster methodology. Have ready at all times a briefing pack explaining why Dr Foster HSMRs are not to be trusted. Useful background papers include:
Lilford R, Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ 2010;340:c2016. [link]
Black N. Assessing the quality of hospitals: hospital standardised mortality ratios should be abandoned. BMJ 2010;340:c2066. [link]
Mohammed MA, Deeks JJ, Girling A, Rudge G, Carmalt M, Stevens AJ, et al. Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals. BMJ 2009;338:b780. [link]
I wonder if you have seen the Francis Inquiry, appendix 9 P. 440. A report from two US based experts who review HSMRs with no bias. They come to the conclusion that HSMRs are a pretty decent thing that should be used alongside other measures to check to see if there might be an issue that needs investigation within a hospital.
It also roundly condemns the paper from Lilford and Mohammed A Mohammed, the conclusions of which were the basis for prof. Lilford’s last paper. Chief amongst these was the constant risk fallacy that you yourself espouse in a previous blog.
I only mention it as you don’t seem to have written a blog about it, you only mention the negative stuff.
Anon – thanks for your thoughtful response. Yes I have read Appendix 9 of the Francis report (Shahian and Normand), and seen the summary of the University of Birmingham report they critique. Shahian and Normand certainly don’t endorse the UoB report, but neither do they roundly condemn it. It is more of a ‘maybe/not really/so what?’ sort of response; and so Dr Foster methodology (ie indirect standardisation) remains ‘OK’.
The constant risk fallacy is a tricky one (so tricky that Shahian and Normand keep enclosing it in inverted commas!). I don’t think I actually mentioned it in a previous blog. I was going to mention it this time, but it is complicated enough to get one’s head round (the ‘key’ paper is less than accessible shall we say), let alone explain in words, and so I decided to go for a different potential flaw (gaming the system) which isn’t a flaw in the methodology per se, but a vulnerability that means it can be massaged by the unscrupulous (and yes such things do happen eg booting patients out of A&E into spurious units/beds to meet waiting time targets).
There is also the question of coding depth – poor coding depth leading to high HSMRs (ie the opposite of item 3 in the list above). It appears that Mid Staffs coding was poor, and then got better – and HSMRs came down (entirely as expected). Some authors do in fact cite this as an example of the constant risk fallacy – I see it more as poor data collection than a bias is the methodology itself.
However, as you have mentioned the constant risk fallacy, I shall describe how I understand it (and it’s potential effects). My earlier post of standardisation Stiff Counting was specifically about indirect vs direct standardisation. The central point is that direct standardisation weights local rates according to a single standard population distribution (so each hospital’s rates get weighted in an identical manner, and so the results are directly comparable), while indirect standardisation weights national rates according to the local population structure, which obviously varies from place to place, causing different weighting for different hospitals, such that the results aren’t comparable. I try and show this in Stiff Counting by showing that two hospitals with identical age specific mortality rates (which should therefore have identical HSMRs) end up with differing HSMRs when their population structures differ. The errors arise because different weightings are applied in each age band because the number of patients in each age band differs; and so the results are not directly comparable.
The constant risk fallacy – at least my understanding of it – is in a way the opposite side of the coin. Instead of using varying weighting (different population structures) when a standard (one set of weights for all), it applies one standard weight (the ‘constant risk’) to all, when in fact the weight of the risk varies between populations. Two examples: the first easier to grasp, the second less so.
First example: let us say one of the risk factors we adjust for is palliative care admission/not palliative care admission (as indeed Dr Foster does). The (not unreasonable) assumption is that being a palliative care admission increases the risk of death (by a fixed ie constant amount); so if one hospital has more palliative care admissions, then it will expect to have more deaths, and it is only right that the expected number of deaths be increased to reflect this, such that the final HSMR will be lower (to reflect the fact more likely to die patients were admitted). However – let us say that for some reason or other one hospital (A) uses palliative care beds in the conventional way; while its neighbour (B) also – in addition to palliative admissions – also admits chronic pain or whatever patients – who are not likely to die: in other words the presumed constant risk attached assumed by being a palliative admission is not in fact a constant risk across all palliative care admissions. When the calculations are done (ie this is a methodology bias), hospital B’s expected deaths will be increased (because all the palliative care patients will be assumed to have the higher risk of death attached to being a palliative care patient) and as a result hospital B’s HSMR will be spuriously lowered – even when in reality both hospitals were (in this example) identical (apart from their use of palliative care beds) in every other way.
Second (hypothetical) example: let us say that diabetic mortality is higher in male (or whatever) patients, because of some interaction between maleness and diabetes (genetic maybe?), even when everything else is controlled for. Hospitals A and B are identical in every respect, including quality of care, but A admits twice as many male patients. HSMRs are calculated, standardising for the usual suspects, including sex and co-morbidities (which includes diabetes). The standardisation applies constant risks and correct standardisation is done for sex (expected deaths are appropriately inflated for males) but applying the constant diabetic risk to both hospitals A and B deflates the expected deaths for hospital A, because, even after standardisation for sex, male diabetic mortality is higher, and so we should expect more deaths, but we don’t, because we applied a constant risk when in fact the risk is not constant. In this case, lower expected deaths at hospital A will spuriously increase that hospital’s HSMR (because it will have more observed deaths, because of the higher mortality in its larger number of male diabetic patients).
So far so good. Where the second example gets more complicated is in teasing out the real risks and effects. Are males more likely to die because the get diabetes more, or because they are more vulnerable to diabetes – or both – in other words, is there an interaction between sex and diabetic mortality? Is the risk true risk male risk plus diabetic risk (eg four plus four equals eight) or male risk multiplied by diabetic risk (four times four equals sixteen)? The hypothetical example had the advantage of being made up – in the real world even the most stringent statistical modelling starts to creak at the seams when trying to isolate the varying contributions to and interactions in risk. Most simply give up at a certain point – and thus important effects can be – and almost certainly are – missed.
It is the oldest and most intractable problem in epidemiology – how do you control for risks you are not aware of? The answer of course, in observational studies (which is what HSMRs are) is you can’t (and that is why we need to remain sceptical); in more controlled settings, randomisation is of course the classic method of dealing with that which we do not know about – but it ain’t going to happen for hospital admissions (however hard the government of the day may try to make it so…).
Dr No is not against measuring hospital mortality – obviously – but he is wary of its apparent beguiling simplicity and consequent interpretation, for example in the press (a point Francis’s report picks up on). In many ways it is like screening (in fact, routine HSMRs are a form of screening) – a no-brainer to politicians and Joe Public – but when we look a little more closely at the science and methodology we begin to see worrying uncertainties – and that is why Dr No has decided it is worth looking at the matter more closely.
I agree with this assessment. I tried to have a long discussion with Prof J but he is not willing to entertain contrary views. My concern is more fundamental – the HES data is historically considered to be inaccurate.
Dr No is doing the world a service by this dissection.
Rita Pal
Ward 87.blogspot.com
I am not medically trained and so do not understand HSMR’s.
Statistics though can always be massaged.
I attended a meeting of a Scrutiny Committee in Wigan to discuss the High Unexpected Death Rate of Wrightington Wigan and Leigh NHS Hospital.
I was very concerned when the CE and Medical Director of the Hospital produced some very professional looking graphs and, to cut to the chase, it sounded as though their idea of reducing Unexpected Deaths was to code or stamp more patients files with ‘expected to die’! So just watch out if you need to go into Hospital for a simple op or minor problem!
Also I am being increasingly made aware of the fact that elderly patients who have started with those end of life symptoms, urine infections and then other infections that seem to end up as pneumonia, are being ‘offloaded’ to care or nursing homes to literally die half an hour after arrival!
I attended a Cure the NHS event on Thursday (29 April 10)at Mid Staffs, Professor Jarman gave a talk at this event. I put a question to Professor Jarman thus “How will you combat creative coding of patients”, Professor Jarman recons he has this covered, i’m not convinced.
This is an excellent piece in regard to manipulation of HSMR figures by these trusts. Hope you don’t mind but the I will adopt the points raised and put them to the CQC and Monitor in regard to my trust.
Excellent piece.
M.C.
To be absolutly honest, I think Professor Jarman wrote the first comment in regard to this article.
If you did produce the first comment in regard to this piece Professor Jarman, I think you have to agree with all Dr No states. It is not the what Dr Foster do with the data that is the main issue here, the main issue being what do these trusts do with the data prior to dispatch to Dr Foster via SUS.
Like Ritz states HES is historically unreliable. Read the September 2006 report by the Royal College of Physicians “Engaging clinicians in improving data quality in the NHS” You can have the best methology in the world in regard to HSMR but if you use flawed data to produce HSMR’s then questions will need to be asked.
Professor Jarman you need incorporate a fiddling factor into your methology. Like Dr No states these trusts have not got the best record in regard to honesty, if they can fiddle then they will. The date used to formulate HSMR figures is wide open to abuse. Professor Jarman join me down here on the street for some wising up in regard to human nature.
M.C.
Prof Jarman didn’t write the first post, I did, I’m sure he would have been much more coherent and statistically significant than I. I agree that it is a complex issue but the key assumption everyone makes is that the coding of palliative care and co-morbidities significantly affects the HSMR.
By far the overriding affect has to be from age, method of admission and diagnosis. I’d be surprised if the chaps like Prof J and Imperial didn’t have something up their sleeves to prove such a point.
However the other key point raised here, which is completely valid, is what are the trusts doing to the data before they get it to Dr Foster? What we need is transparency to see how the trusts’ coding has changed, if at all, over the last few years.
Are there any unusual patterns that might be a trust gaming its coding or simply coding more effectively? I don’t think we should automatically assume trusts are all trying to disguise the quality of their care, that these killing factories are hiding their evil ways. Most trusts seem to want to understand their figures and improve – and that’s why I like HSMRs. No one says they are perfect or a single overall reliable measure, but they do help the trust to investigate in the right areas and they do give the public a chance to ask the right questions. I don’t think the intellectual capacity of the general public should be underestimated.
Having come to learn a certain amount about HSMR’s through Miguel of http://www.wrightingtonwiganandleighexposed.com and after meeting Professor Jarman in London, I am interested in the ‘creative coding’ or lying aspect of NHS Statistics.
However I comment now on 2 statements made by Anonymous:
1. “I don’t think we should automatically assume Trusts are all trying to disguise the quality of their care”.
Sorry but from what I have experienced and heard whilst speaking to NHS ‘customers’ from different parts of the UK, I must disagree with you. Time and time again ‘serious’ complainants are commenting on how their Trust’s were ‘disguising’ facts.
So as it seems to be ‘par for the course’ to disguise or lie in one area then this leads to mistrust of facts in others!
2. “They do give the public a chance to ask the right questions”.
Sorry but from what I and others have experienced, try to ask a straight and simple question of any NHS Trust and they are unable to answer with a straight answer – they disguise and lie about the simplest of things.
So I would not trust what any NHS Trust says!
Sorry Dr. No for straying off the main topic!
I have no doubt some hospital managers are altruistic, but gaming the system is human nature. Even Hon. Members, even those from ‘the other place’ do it. And we know as a matter of fact (some) hospital managers do it (eg A&E targets) – and when some do it, then the others have to do it, to keep up.
Given the drive towards pseudo-commercialism in the NHS – internal ‘markets’, PBR, PRP etc etc etc – it is inevitable that gaming will become endemic – if not epidemic. It is not the only problem facing HSMRs, but it is a serious, possibly fatal (sic) one.
I’m just curious as to when (or possibly whether) this blog will be updated anytime soon. I always worries that my favourite medical bloggers have been rounded up in some sort of midnight raid and sent to the Siberian Salt mines for defamation of the status quo.
Dr No is not in a salt mine (yet). He is bidng his time.
DN
Waiting for you.
Anon – rest assured – DN is alive and well. He has just been enjoying a holiday he couldn’t refuse. But – he’ll be back! Soon!
house makeover games for girls seeeeeeeeeeeeex games casino royale quotes imdb casino highest online paying fanny mae bonus game longest poker bingo casino station blackjack playing conrad jupiters hotel and casino online surgery game superbowl live betting my poker face music video fee boat online games poker rooms in florida redriver casino spierman games 900pay casino playtech sterling casino cruise casino dice game russian roulette vodka online poker how-to expansion expansion module slot porno games free world poker tour online tournaments compulsive gambling treatment taxes gambling loss memphis colonial middle principal took money casino atlantis casino paradise island holdem holdem online poker room texas texasholdempoker.info free american roulette l auberge du lac hotel casino south carolina education lottery not going towards education cherokee casino west siloam springs free downlaod of pornographic video games jouer au casino mega millions calif lotto play video poker myscene.com game triple red hot sevens slot game online casino map windsor free no download slot machine game roulette system uk casino verite blackjack pro v3.0.36 royal wedding1981 charles &dianna speciallly brewed beers game multiplayer online poker mohigan son casino play three card poker poker million live poker fun chat rooms for kids browser games playable through phone marango casino cabazon play shooter games online superenalotto online usa dv lottory casino club gaming uk grand suites hotel wpt online poker play 3d airplane games yoruichi porn game ming the master poker real birth games online bonus.com casino ch link play poker.e poker world bonus lottery number generator rain forest sloths lottozahlen generator the mill casino coos bay oregon free yaho pool games dejope casino wisconsin pokerstars radio casino royale mad props to the universe mountain rancheria casino lotto.com philippine grand theft auto 2 cheats :airplane psp instruccion juego poker 1 1.html casino online slot virtual room design game galaxi casino lotto 6 45 result betting horse racing spread who wrote poker face crown casino australia melbourne violent video game effects on children and adolescents the florida lotto can device in installed pci slot type support pokerroom.com free line blackjack monique vegas video gratis poker tips texas holdem frank wallace poker bonus code for partypoker.com online gaming cafe casino best bets game home poker tightpoker history of video poker machines top casino affiliate program game of horseshoes roots game theory casino conn foxwoods book casino gambling internet sport book gambling gambling live live.com sport probability poker book georgia guest lotto poker star 2 pro line lotto gambling losses on taxes poker gold indio casino lotto risultati superenalotto super slot casino to play ozlotto skateboard games online casino net pay top rules strip poker bratz mash game bus casino stone turning boston poker club online role playig games chating games pestana casino park hotel reviews
How strangely prophetic your article is, Dr. No.
Seems to be a description of the entire NHS ‘end of life care pathway’. This : http://www.ncpc.org.uk/sites/default/files/AandE.pdf advice was issued by the Coalition to the new nhs commissioners who take over in April 2013…it advises them to code up more or less the same patient groups for’end of life care’ your scheme does…and is specifically to avoid hospital admissions.
Page 4 reads:
“For the purposes of this guidance people are‘approaching the end of life’ when they are likely to die within the next 12 months.
This includes people whose death is imminent (expected within a few hours or days) and
those with:
(a) advanced, progressive, incurable conditions
(b) general frailty and co-existing conditions that mean
they are expected to die within 12 months
(c) existing conditions if they are at risk of dying from a
sudden acute crisis in their condition
(d) life-threatening acute conditions caused by
sudden catastrophic events.
http://www.ncpc.org.uk/sites/default/files/AandE.pdf
…that covers just about EVERYONE in the known frigging universe.
The profits have spoken. By the way, there’s a secret gematric code in the Book of Revelations, and what God meant is that on the Day of Judgement, everyone who can’t afford BUPA cover should be palliatively sedated in a Marie Curie (TM) unit – ‘have a good death’ as they say in Hospiceland!
Gematria Consultant: Julia ‘Rabbi’ Neuberger
..oh, and I guess the fact the Coalition planned their ENTIRE NHS budget savings (£20 million by 2014) on the ‘bed days saved’ with the End of Life Care pathway, which leaves people to die of treatable conditions at home (whether they want to or not!) is entirely coincidental to Messrs. Hunt & Lamb refusing to hold a Public Inquiry into the Liverpool Care Pathway ??? The tastefully named ‘Omega Report’ proves the wheels are coming off that one …
What annoys me is, I have seen almost everything mentioned here happen at Leighton hospital, yet despite these venal efforts, they STILL come bottom in the mortality figures.
Best maternity unit in the North West my arse! They “cook” those figures by showboating the unit during parent education classes and give you a survey to fill in there and then. They don’t wait until after patients have actually USED or endured those so-called “facilities” to carry one out!
It’s bloody dangerous in there, I was lucky to get relatives out alive, and violent staff-patient conflict is higher than average. I wonder why?