Paul Corrigan, whose posts show a worrying trend towards titles so long they stand as posts in their own right, has declared himself a friend of FFT, the punter-friendly friends and family test based on asking patients at or soon after discharge whether they would recommend the unit they have just left to friends and family. The test is popular with government for its apparent simplicity, resented by managers for the real extra burden it imposes, and derided by front-line staff, for whom the test might be better known as the Flying F*ck Test: the punters don’t give a FF about responding, and we don’t give a FF about the results. Although first announced last year, FF testing was back in the news last week after friendly we’re all in this together Dave announced plans to extend FF testing to general practice. The news got a cool response from senior GPs; others went further. One called the test ‘meaningless’; another dubbed it ‘trite’.
Dr No approached this post expecting to do a routine shaft of a daft idea. Despite the superficial appeal of a neat single global measure of patient satisfaction, the fact is that if it is to do with the health service, and we’re all in this together Dave is in favour of it, then there is usually something wrong with it. It didn’t take Dr No long to find the idea’s Achilles Heel. Like most customer surveys, the response rate for FF testing is abysmal. The governments own FFT guidance ‘expects’ rates of ‘around 15%’, adding, as another porker roars by on after-burn, ‘for the majority, this figure could be much higher’. The overall (but with wide variation between trusts, with privately run Hinchingbrooke being notably high, but that’s another story for another day) response rate in a pilot study was in fact slightly better (current latest figure 18.99% for Nov 2012), but that doesn’t alter the fact that more than eight out of ten punters didn’t give a FF. Dr No raised the shaft of non-response bias, and was about to impale the daft idea, when he suddenly thought: careful.
Careful, because the last paragraph contains two assumptions, and assumptions are as we know the mother of all. The first assumption is that Tory health service ideas are routinely bad – an easy enough habit to get into given recent performance, but not necessarily the case. The second is that low response rates will result in non-response bias: the two out of ten who did respond are by definition exceptions, and therefore somehow freaks. Even if the gross demographics of responders and non-responders are shown to be the same, the discrepancy between two and eight out of ten is so striking as to give rise to a presumption of divergence of opinion on the matter of substantive interest between the two groups. How could a feeble two out of ten minority possibly represent an whopping eight out of ten majority? The responders, we feel intuitively, are bound to be unrepresentative; and the bigger the non-response rate, the bigger the bias. Low response rates, prima facie, are bad.
Well, not necessarily so. It all depends on whether there is a material – that is, relevant to the question at hand – reason governing whether a person responds or not. Such material reasons are very plausible: non-responders may not respond because they are disgruntled, and fear a black mark if they express their disgrunt. Responders may be toadies, eager to please. All manner of imaginary reasons can be imagined. But what is the evidence?
The evidence, somewhat counter-intuitively, supports ‘not necessarily so’. As humans, we naturally ascribe reasons to behaviour, but what if response/non-response behaviour is not reasoned behaviour, but random? What evidence there is suggests that the hand that guides response can indeed be the hand that rolls dice rather than the hand of reason. Although the setting, which may or may not matter, is different, eye-popping research by Keeter, Curtin and others on telephone surveys shows response rates appear to have little meaningful impact on results. One typical study, comparing ‘standard’ (25%) with ‘rigorous’ (50%) rates, found that, for ‘77 out of 84 comparable items, the two surveys yielded results that were statistically indistinguishable’. Perhaps low response rates aren’t an Achilles heel after all.
So Dr No has decided to put the shaft back in its box, and consider that FF testing may not be so daft after all. In fact, so long as the natural constraints of a single punter-based quirks-mode assessment are always borne in mind, Dr No finds the idea of FF testing appealing, precisely because it is a single comprehensible to all global assessment. Whether it would have turned on the red light at Mid Staffs faster than Sir Jar’s FosterKit is a moot point: its appeal, in contrast to Foster, is in its simplicity. Rest assured that, should it ever get ideas above its station and claim to measure complex detail beyond its capabilities, then Dr No will have that shaft back out of its box before you can say daft.
Witches and Black Cats never answer questionnaires. They make us squirm. Especially this kind.
This is what we would say about them if asked:
“ Apart from being a waste of public money in that the benefits are likely to be miniscule compared to the effort and costs involved, you are not measuring what you think you might be measuring.
If I am nice, (and especially if you have been nice to me and smiled at me and gave me pain relief, food, fluid and bedpans when required) then my answers on a scale of 1 to 6 will be positive (1 to 2). Even if you have smiled and done all of these things, and I’m quite nice but reserved and indecisive yet feel I should express an opinion, I’ll sit on the fence and answer 3 or 4. If I’m a cranky old curmudgeon then I’ll answer in the negative.
So what you are really measuring is how nice or how belligerent I am and that’s a useless measurement for the purposes required.”
However, perhaps the results will generally look good whether the response rate be 15% or 90% because most people are “nice” whether or not they are in the responder or non-responder group. There are probably few curmudgeons in the patient population since even those with this trait are so grateful to get out of hospital in one piece that they don’t answer true to form.
On the other hand if you don’t smile at me, feed me, water me, relieve my pain and fetch a timely bed-pan…..
But it should not require costly questionnaires up and down the land to ensure these basics are in place.
WD – DN doesn’t have in mind complicated expensive questionnaires. How about every patient gets a token (bit like Waitrose). At the ward/unit/whatever exit there are two ballot boxes, one marked ‘yes’ and one marked ‘no’ (plus perhaps a third marked ‘I’m a witch, don’t even think about asking me’) under a question: would you recommend this ward/unit to friends and family? Simple, but effective.
As we all know satisfaction is directly linked to expectations. A questionnaire in a hotel setting, a restaurant or a retail experience is pretty straightforward. I expect a bed, I get a bed ergo I am satisfied. If the service EXCEEDS my expectation for the money I have spent then I am happy and vice versa. In a health situation this direct correlation does not exist and is, therefore, too simplified a method to “police” the service provided.
Anything to do with health is emotionally charged (something rarely encountered whilst buying a six-pack of sausages from Sainsbury’s).
The moment I become a “Patient” I lose all objectivity (not that I had any to begin with since I am a human being and thus am physiologically incapable of being objective).
So, as a patient I have no objectivity and I also have no knowledge. I am in that most fearful of states: unwell/loss of independance and in the hands of total strangers.
I very much doubt I am the best judge of the service provided, all I can do is base my opinion on how nice you all were to me (as WD said) and how quickly you got me out of there alive.
Having said that goverments are chosen by the opinions of millions of people who have absolutely NO knowledge whatsoever of politics, economics, sociology etc.
We’re all in the hands of people we “like the look of”…
Suzanna
Dear Dr No,
The Government, as we both know, have failed to tackle the real problems that we have.
One of these is a totally hypocritical GMC. The website claims that probity is at the heart of good medical practice yet when I complained about Dr Mike Hobbs, clincal director of the Warneford being a liar, harassing me, failing to investigate and withholding paperwork they wrote back and said they could not see anything impairing his fitness to practice.
Then, of course, there was my complaint about Dr Deenesh Khoosal assaulting me on 25/11/82 in the Warneford.
Dr Thorne, the college medical officer who has recently written me a nice letter, described me as rather “upset” and “vulnerable” in his referral of 15/11/82 and I was only just 17.
As you know, I attempted suicide in my study on 6/12/82 after horrific bullying by all the adults involved.
When the internal notes came to light in 2008 there was a suicide diary written while listening to the new Dire Straits album “Love over Gold”.
That was my motivation to going to the Authorities; love not money. Khoosal is practicing as a child psychiatrist and undoubtedly drugging and harming children.
He built his entire career on trying to take mine away. It is recorded in the notes that I planned a career in the sailing world. He made completely false allegations of mental illness and I am still trying to clear my name as that puts up all kind of barriers.
I went to the police during the GMC hearing of Russell Reid who saved my life twice; in January 1988 and in 1991 with a free appointment and a referral to Fay Presto.
Reid told me at the GMC that Khooters is gay and everything dropped into place.
Nobody expects her doctor to bellow in her face, “are you interested in anal sex?” at the first appointment.
Reid was tried in The Guardian by David Batty. He wrote completely false article on Paula Rowe called Trapped in Gender Limbo in 2004.
It became increasingly obvious during the hearing in 2007 that Rowe was lying and so I left my job and went to see Reid at the hearing.
He suggested that it was nice of me to stand up for him and reminded him of the free appointment. “I knew you didn’t have any money”
Thats how a friendly relationship develops; when you do something excellent for a patient.
My current GP, Dr John Gallagher, wrote an excellent letter to Alan Whitehead MP saying that I had come to see him to express my disatisfaction with the criminal justice system and regulatory body governing the medical profession(GMC)
I had an excellent meeting with Dr G the other week when I expressed my appreciation of a doctor who can write beautiful English.
I was telling him about my adventures with Dr No and Dr Helen Bright the Yugoslav psychiatrist and we were roaring with laughter over sharks nuns and toilets.
Dr G said, “they can be very dangerous you know; especially the low flush ones”!
He asked me to say something nice about him on NHS choice which of couse I will.
Dr No’s excellent Medical Commentary was equally appreciated and will undoubtedly save lives.
David Batty has been trying to get the popular Dr Curtis, Reid sucessor struck off and all hell has broken loose at the Observer today with a demo.
And the GMC are being attacked on Twitter
On the sailing front Ripping Yarns won Concours D’elegance at Bosham Classic revival – there’s a film on the Yacht and Yachting website – and I have half a page in Lusimus…the Radley magazine for the sailing project
My Grandpa, Dr JTS Hoey Croix de Guerre would be proud of you. Have a look at the website
Paul Corrigan is exactly what is wrong with the NHS, his words are so ignorant and naive.
The FF test is unvalidated meaningless tripe, anyone who claims it is not is either corrupt or an utter moron.
Corrigan would do well to learn the basics of validity before spouting his wretched nonsense.
A recent NEJM article summarised neatly how trying to capture patient satisfaction from overall vague questions is not only pointless but completely meaningless. One needs detailed specific questions that focus on particular areas of the care received.