Category: Bad Stats

There has been much ado about hospital death rates lately, much of it focused on the Mid Staffs hospitals, where consistently high apparent death rates were repeatedly brushed aside and ignored. The issue at stake was the validity – or not – of certain statistics produced by Sir Jar and his operatives at HI5, the Dr Foster Intelligence Unit, and one statistic in particular, the HSMR.

The HSMR, or Hospital Standardised Mortality Ratio to give it its full name, is said by Sir Jar to offer a useful marker of a hospital’s performance, by providing a single figure that summarises how many patients leave the hospital feet-first. High value HSMRs suggest more stiffs than expected, low HSMRs indicate less stiffs than expected, compared to national figures. Unfortunately for Sir Jar, the method – quite apart from a myriad of other factors that might compromise validity – he uses to determine HSMRs suffers from a flaw that severely restricts its application. While an isolated HMSR can be compared to the ‘big picture’ – in other words, the standard population to which it is being compared – comparisons between hospitals, or even the same hospital over time, are prone to errors, which can render the results at best meaningless, at worst misleading.

Dr No is hopeless with numbers. Just being in the presence of statisticians causes a pressure of incomprehension to build up in his head. When they start to talk numbers, it is as if they are speaking in tongues; and when their chalk squeaks on the blackboard, all he sees is so many hieroglyphics.

But numbers are part of the fabric of medicine, and their understanding is necessary to the practice of medicine; and so Dr No has over the years developed a habit of translating the hieroglyphics into words, and the formulae into verbal instructions. Σx becomes the sum total of all the values of x; and μ = Σx/n becomes find the average value of x by adding them together, and dividing by the number in the sample. The dark impenetrable pool of numeracy is side-stepped on the well-worn plank of words.

One of the most bizarre facts about modern medicine is that most of the time, for most people, most drugs don’t work. Naturally, this is something that Big Pharma is keen to keep hidden. Even most doctors are only dimly aware that most of the drugs they peddle might not always be what they are cracked up to be. At a time when drug companies and doctors are pushing ever more pills onto ever more patients, we should perhaps be a little more savvy about the pharmaceutical pact we enter into when we agree to pop pills.

For hundreds of years, doctors had few effective drugs at their disposal. Those that they did have were either herbal toxins used in small doses – opium, digitalis, quinine and the like, which most certainly did and do work – or so-called “tonics” – dubious placebos that nonetheless pleased the doctor and his patient.

What two things do sex and statistics have in common – apart, that is, from both starting with the letter S? The answer, of course, is firstly that in both, bigger is usually butt (sic) not always better, and secondly that both can be massaged to make things, err, stand out more.

We are drowning in number soup again.

The UK medical blogosphere is getting worked up about some numbers released last week by the Patients Association. The consensus seems to be that the numbers are a classic example of foul play by statistics. But perhaps they are not.

One of the more striking changes in medicine in recent years has been the increasing use of – and reliance on – numbers. Now, cosying up to numbers is all very well, as long as you understand them. Most doctors do not.