You are hereBlogs / dr-no's blog / Making Friends and Family of Us All

Making Friends and Family of Us All


Posted by Dr No on 10 January 2013

recommend.jpgPaul Corrigan, whose posts show a worrying trend towards titles so long they stand as posts in their own right, has declared himself a friend of FFT, the punter-friendly friends and family test based on asking patients at or soon after discharge whether they would recommend the unit they have just left to friends and family. The test is popular with government for its apparent simplicity, resented by managers for the real extra burden it imposes, and derided by front-line staff, for whom the test might be better known as the Flying F*ck Test: the punters don’t give a FF about responding, and we don’t give a FF about the results. Although first announced last year, FF testing was back in the news last week after friendly we’re all in this together Dave announced plans to extend FF testing to general practice. The news got a cool response from senior GPs; others went further. One called the test ‘meaningless’; another dubbed it ‘trite’.

Dr No approached this post expecting to do a routine shaft of a daft idea. Despite the superficial appeal of a neat single global measure of patient satisfaction, the fact is that if it is to do with the health service, and we’re all in this together Dave is in favour of it, then there is usually something wrong with it. It didn’t take Dr No long to find the idea’s Achilles Heel. Like most customer surveys, the response rate for FF testing is abysmal. The governments own FFT guidance ‘expects’ rates of ‘around 15%’, adding, as another porker roars by on after-burn, ‘for the majority, this figure could be much higher’. The overall (but with wide variation between trusts, with privately run Hinchingbrooke being notably high, but that’s another story for another day) response rate in a pilot study was in fact slightly better (current latest figure 18.99% for Nov 2012), but that doesn’t alter the fact that more than eight out of ten punters didn’t give a FF. Dr No raised the shaft of non-response bias, and was about to impale the daft idea, when he suddenly thought: careful.

Careful, because the last paragraph contains two assumptions, and assumptions are as we know the mother of all. The first assumption is that Tory health service ideas are routinely bad – an easy enough habit to get into given recent performance, but not necessarily the case. The second is that low response rates will result in non-response bias: the two out of ten who did respond are by definition exceptions, and therefore somehow freaks. Even if the gross demographics of responders and non-responders are shown to be the same, the discrepancy between two and eight out of ten is so striking as to give rise to a presumption of divergence of opinion on the matter of substantive interest between the two groups. How could a feeble two out of ten minority possibly represent an whopping eight out of ten majority? The responders, we feel intuitively, are bound to be unrepresentative; and the bigger the non-response rate, the bigger the bias. Low response rates, prima facie, are bad.

Well, not necessarily so. It all depends on whether there is a material – that is, relevant to the question at hand - reason governing whether a person responds or not. Such material reasons are very plausible: non-responders may not respond because they are disgruntled, and fear a black mark if they express their disgrunt. Responders may be toadies, eager to please. All manner of imaginary reasons can be imagined. But what is the evidence?

The evidence, somewhat counter-intuitively, supports ‘not necessarily so’. As humans, we naturally ascribe reasons to behaviour, but what if response/non-response behaviour is not reasoned behaviour, but random? What evidence there is suggests that the hand that guides response can indeed be the hand that rolls dice rather than the hand of reason. Although the setting, which may or may not matter, is different, eye-popping research by Keeter, Curtin and others on telephone surveys shows response rates appear to have little meaningful impact on results. One typical study, comparing ‘standard’ (25%) with ‘rigorous’ (50%) rates, found that, for ‘77 out of 84 comparable items, the two surveys yielded results that were statistically indistinguishable’. Perhaps low response rates aren’t an Achilles heel after all.

So Dr No has decided to put the shaft back in its box, and consider that FF testing may not be so daft after all. In fact, so long as the natural constraints of a single punter-based quirks-mode assessment are always borne in mind, Dr No finds the idea of FF testing appealing, precisely because it is a single comprehensible to all global assessment. Whether it would have turned on the red light at Mid Staffs faster than Sir Jar’s FosterKit is a moot point: its appeal, in contrast to Foster, is in its simplicity. Rest assured that, should it ever get ideas above its station and claim to measure complex detail beyond its capabilities, then Dr No will have that shaft back out of its box before you can say daft.

4 comments:

Witches and Black Cats never answer questionnaires. They make us squirm. Especially this kind.

This is what we would say about them if asked:

“ Apart from being a waste of public money in that the benefits are likely to be miniscule compared to the effort and costs involved, you are not measuring what you think you might be measuring.

If I am nice, (and especially if you have been nice to me and smiled at me and gave me pain relief, food, fluid and bedpans when required) then my answers on a scale of 1 to 6 will be positive (1 to 2). Even if you have smiled and done all of these things, and I'm quite nice but reserved and indecisive yet feel I should express an opinion, I’ll sit on the fence and answer 3 or 4. If I’m a cranky old curmudgeon then I’ll answer in the negative.

So what you are really measuring is how nice or how belligerent I am and that’s a useless measurement for the purposes required.”

However, perhaps the results will generally look good whether the response rate be 15% or 90% because most people are “nice” whether or not they are in the responder or non-responder group. There are probably few curmudgeons in the patient population since even those with this trait are so grateful to get out of hospital in one piece that they don't answer true to form.

On the other hand if you don’t smile at me, feed me, water me, relieve my pain and fetch a timely bed-pan…..

But it should not require costly questionnaires up and down the land to ensure these basics are in place.

WD - DN doesn't have in mind complicated expensive questionnaires. How about every patient gets a token (bit like Waitrose). At the ward/unit/whatever exit there are two ballot boxes, one marked 'yes' and one marked 'no' (plus perhaps a third marked 'I'm a witch, don't even think about asking me') under a question: would you recommend this ward/unit to friends and family? Simple, but effective.

As we all know satisfaction is directly linked to expectations. A questionnaire in a hotel setting, a restaurant or a retail experience is pretty straightforward. I expect a bed, I get a bed ergo I am satisfied. If the service EXCEEDS my expectation for the money I have spent then I am happy and vice versa. In a health situation this direct correlation does not exist and is, therefore, too simplified a method to "police" the service provided.
Anything to do with health is emotionally charged (something rarely encountered whilst buying a six-pack of sausages from Sainsbury's).
The moment I become a "Patient" I lose all objectivity (not that I had any to begin with since I am a human being and thus am physiologically incapable of being objective).
So, as a patient I have no objectivity and I also have no knowledge. I am in that most fearful of states: unwell/loss of independance and in the hands of total strangers.
I very much doubt I am the best judge of the service provided, all I can do is base my opinion on how nice you all were to me (as WD said) and how quickly you got me out of there alive.
Having said that goverments are chosen by the opinions of millions of people who have absolutely NO knowledge whatsoever of politics, economics, sociology etc.
We're all in the hands of people we "like the look of"...

Suzanna

Paul Corrigan is exactly what is wrong with the NHS, his words are so ignorant and naive.

The FF test is unvalidated meaningless tripe, anyone who claims it is not is either corrupt or an utter moron.

Corrigan would do well to learn the basics of validity before spouting his wretched nonsense.

A recent NEJM article summarised neatly how trying to capture patient satisfaction from overall vague questions is not only pointless but completely meaningless. One needs detailed specific questions that focus on particular areas of the care received.


Add a comment...

Will show as anonymous if no name added

If added, your name will be a link to the address you enter

If left blank, first few words of comment will be used

• Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li>
• Web page addresses and e-mail addresses turn into links automatically
• Lines and paragraphs break automatically
 

NOTE: Dr No's spam filter can be somewhat overzealous. If your comment has been wrongly rejected, Dr No apologises, and asks that you let him know (via Contact Form in side-bar). Many thanks.