We are drowning in number soup again.
The UK medical blogosphere is getting worked up about some numbers released last week by the Patients Association. The consensus seems to be that the numbers are a classic example of foul play by statistics. But perhaps they are not.
The Patients Association claims to be a pressure group standing up for patients. The fact that it appears to be funded by big Pharma and the private health industry (including Sir Richard Blagsome’s Virgin Healthcare) rather takes the heat out of their mustard. It is more likely that they wish to show that the NHS is one big Richard and quite frankly my dear we’ll all be a lot better off when the private sector runs things.
Last week they released a dossier of despair containing harrowing accounts of old biddies being minced up by the NHS. On page five of the report they note that the national inpatient survey shows that “the percentage of patients that rated their care as poor had not changed from 2002 to 2008, staying at 2%. Over the years of the survey this represents approximately 10,000 patients who personally rated their care as poor”.
Two percent, even 10,000 patients, may not sound impressive, but by a back of the envelope calculation it can be seriously sexed up. The two percent quickly became “over 1 million patients” who rated their care as poor between 2002 and 2008. You can read the press release here.
Naturally enough, “over 1 million patients” rating their care as poor got the press sitting up. The Torygraph led the way, wailing "‘Cruel and neglectful’ care of one million NHS patients exposed", adding "hundreds of thousands have suffered from poor standards of nursing, often with ‘neglectful, demeaning, painful and sometimes downright cruel’ treatment".
What has upset everyone, including Channel 4’s Factbuster website, is the extrapolation from two percent to “over 1 million”.
The Patients Association’s says it is quite straight forward. Using government figures, they note that there are some 10 million hospital admissions a year. Two percent of 10 million is 200,000. Multiply that by six years (2002 to 2008) and bingo – 1.2 million patients.
Putting aside the dubious practice of sexing up the numbers (in this case by converting small relative numbers into big absolute numbers), the calculation is itself quite reasonable. It is very simple (0.02 x 10,000,000 x 6), and none of the figures are in major doubt. If the two percent figure can be extrapolated to all admissions, then it is indeed very likely that over 1 million patients rated their care as poor between 2002 and 2008.
The key word, of course, is “if”. It means that the 1.2 million is not a fact but a conjecture. All the Patients Association have said is if the two percent can be extrapolated, then the total number is 1.2 million.
And that is a perfectly reasonable thing to say. In fact, extrapolation from research to real life is what doctors do every time they apply research findings to real patients. They say if the findings from this study sample can be applied to the population at large, then it is reasonable for me to provide treatment on the basis of the study findings.
Extrapolation from sample to population is inevitable and acceptable – without it, there would be no way to apply research findings to real life. But – most importantly – before we extrapolate, we must ask a crucial question: is this study sample representative of the population we intend apply the results to? If it is not representative, then we should not extrapolate, but if it is, then it is very reasonable to extrapolate the findings.
The importance of representativesness, and whether sampling errors have occurred (thereby rendering the sample unrepresentative), cannot be over-stated. It is, of course why the once infamous "8 out of 10 Cats prefer Whiskas" was later changed to the less snappy but more accurate "8 out of 10 owners who expressed a preference said that their cats preferred Whiskas", so reminding us that it was 8 out of 10 of a sample – and a proxy sample at that.
And – to complicate things even more – we do not know how many of the sample were repeats. The Patients Association would have done better to say 1.2 admissions (which may have included the same patient being admitted more than once) rather than 1.2 patients.
Unfortunately, on this occasion, the Torygraph hacks practised some hackery. They hacked off the “if”, thereby changing the Patients Association’s acceptable conjecture into apparent fact – and that is what is not acceptable. The mutation happened at the Torygraph, and it is the Torygraph, not the Patients Association, who have indulged in foul play on this occasion.
There will of course be other occasions.