One of the most bizarre facts about modern medicine is that most of the time, for most people, most drugs don’t work. Naturally, this is something that Big Pharma is keen to keep hidden. Even most doctors are only dimly aware that most of the drugs they peddle might not always be what they are cracked up to be. At a time when drug companies and doctors are pushing ever more pills onto ever more patients, we should perhaps be a little more savvy about the pharmaceutical pact we enter into when we agree to pop pills.
For hundreds of years, doctors had few effective drugs at their disposal. Those that they did have were either herbal toxins used in small doses – opium, digitalis, quinine and the like, which most certainly did and do work – or so-called “tonics” – dubious placebos that nonetheless pleased the doctor and his patient.
Over the last half century, all that has changed. Drug companies, armed with the latest technologies, investigate millions of compounds. Improvements in clinical research – notably the randomised controlled trial – have allowed us to measure even the smallest drug effect. We are able to detect statistical significance – which is a fancy way of saying the achieved result is unlikely to have arisen by chance; and in our zeal to chase statistical significance we have somewhat lost sight of another equal if not more important significance: clinical significance.
Statistical significance unfortunately tells us nothing about clinical significance. A trivial benefit that assists one in a million patients can have the same statistical significance as a useful treatment that benefits a million patients.
Recently, sharp medical minds have turned their attention to this problem, and come up with a number that better summarises the real world usefulness of a drug. That number is called the ‘number needed to treat’ – the NNT – and it is in the NNT that we find exposed the awkward fact that for most people, most drugs don’t work.
The NNT describes the number of patients that need to be treated with a drug for one to benefit. An NNT of one would mean that every patient benefits; an NNT of ten would mean that ten would need treatment for one to benefit. In a single figure, it gives us a real world measure of the drug’s effectiveness. And it also reveals something else. If the NNT is ten, then that means nine out of ten patients did not benefit from taking the drug.
But – I hear you say – lots of people benefit from drugs. But association is not causation: Some people will get better with or without the drug. Some just happen to be taking the drug, when they would have recovered anyway, just as some will take the drug, and not recover.
Let us imagine a drug used to treat migraine. We wish to assess the effectiveness of this drug, and so conduct a randomised (patients are randomly allocated to take either the drug or a placebo) double-blind (neither patient nor doctor know whether a particular patient is receiving drug or placebo) controlled (there is a comparison group who do not take the active drug) trial.
So we have two groups: the active treatment group, and the no active treatment (control) group. Let us say (for the sake of clarity – in real life the numbers would be larger) that there are five patients in each group; and the outcome we are interested in is migraine headache gone in one hour.
In this hypothetical study we find that, in our control group (patients 1 to 5), two patients recover, while three continue to have their headache, while in the treatment group (patients 6 – 10), three recover, and two continue to have headaches. We can summarise this as:
• Patient 1: Still has headache
• Patient 2: Still has headache
• Patient 3: Still has headache
• Patient 4: Headache settles
• Patient 5: Headache settles
Treatment group:
• Patient 6: Still has headache
• Patient 7: Still has headache
• Patient 8: Headache settles
• Patient 9: Headache settles
• Patient 10: Headache settles
What has happened is that active treatment has caused one patient (patient 8 in this example) to flip from still having a headache to headache gone. The NNT is five, meaning that five patients must be treated for one to benefit. The other four (out of the five) remain unaffected by the drug. And thus it is that we can say that, for most patients – four out of the five – the drug did not work, because it had no effect on their outcome
An NNT of five for an active treatment would, in a clinical setting be considered adequate, even encouraging. But even with single figure NNTs, we are forced to accept that for most patients, most drugs don’t work.
When it comes to preventative treatments – treatments such as aspirin and statins to prevent heart attacks – the NNTs start to head off into the stratosphere. Aspirin, for example, typically has an NNT around 200, while statins have NNTs ranging between 10 and 100, depending on baseline risk. And when hundreds of patients need to take a drug for one to benefit, truly can it be said that, for most patients, most drugs don’t work.
But – and this is the dilemma – for that one unknown (unknown because we cannot predict which patient will benefit) patient to benefit, tens if not hundreds of patients must take the drug without any prospect of benefit, to allow that one patient to benefit. That is the nature of the pharmaceutical pact: many must pop pills knowing they will not benefit, so that one might benefit.
I serve as a lay representative on the HTA Pharmaceutical Panel which undertakes an initial prioritisation of NHS funded pharmaceutical research and have recently become aware of the potential for DNA specific prescribing, which in 10 to 15 years time might mark a major step change for prescribing habits.
Comments made by more intelligent and knowledgable professional members have resulted in the following line of thought. (1) We know that an effective drug will not work in all cases (lets assume the number needed to treat – NNT is 1.3. (2) If we could identify common factors among the roughly 25% of patients for whom the drug was not effective we could spare them the potential side effects. (3) A likely candidate for explaining why 25% do not respond may be genetic in origin. (4) By extending our knowledge of these people’s DNA we might be able to identify which group of patients would not benefit. (5) Armed with that knowledge we could screen out such patients and reduce the NNT even nearer to 1. (6) Future research among this identifiable sub group of patients might also help the discovery of an appropriate response tailored better to their own chances of success.
In my more “visionary” moments I can see – given a very fair wind – how such a future might play out for the benefit of patients, and the clinicians who care for them. However Big Pharma would need to be won over to the idea of individually tailored prescribing if it is not going to dig in and delay the process. Given the vested interest that you have described – where many patients will not benefit whenever specific medication is prescribed and consumed – I do wonder how we could achieve the change of heart that would be needed.
I am of course aware that my line of thinking is shot through with broad assumptions but do think it is worth considering as one possible future scenario whenever (and if ever) the NHS returns to considering long term health policy and planning, instead of relying on short term market forces to distribute health care resources, which seems to be ineffective, uneconomic and inefficient.
PoH – where would we be without your erudition!
I don’t think what you describe is medically that far fetched at all. The problem is going to be Big Pharma – they will see better profit in Model T Fords (any colour you want, as long as it is black) than in custom Aston Martins – all the more so when the state is paying.
If genetic profiling did decrease the amount of drugs wasted on those we know it would not work on this would reduce the amount of drugs they sell quite possibly. Would this not lead to them increasing prices, using the large research bills involved in getting a drug to market as justification? Nonetheless the benefit of not prescribing drugs, which at best do nothing, while at worst could subtlety play havoc with a patients body I guess is a legitimate enough reason. Is big Parma a play on words, or just a rather nice hard cheese of a typo?
M – you win the Parma Ham Prize for knowing a Big Cheese when you see one. It was a typo pure and simple. Thanks for pointing it out. I have taken the liberty of correcting it.
On your point about Big Cheeses selling fewer drugs and so increasing prices – I think that almost inevitable – but the total cost might remain the same (five patients x £10 charge might become one patient x £50 charge to stick with the original numbers of patients). The real benefit would be as you suggest – avoiding indiscriminate scatter-gun prescribing, with its associated risks of harm to those who stood to gain nothing.
Government Funding / Research Scandal
(**Updated March 15th** – Participants)
Visit the website that the Canadian House of Commons and many Universities across North America have as well.
———————————————
It’s an ingenious form of white collar crime:
PHD credentials / contacts, an expendable family, participation of a dubious core of established professionals, Government agency funding (identity protected by Privacy Commissioner Office), unlimited funding (under the guise of research grants), PHD individuals linked with the patient (deter liability issues), patient diagnosed with mental illness (hospital committed events = no legal lawyer access/rights), cooperation of local University and police (resources and security); note the Director of Brock Campus Security.
This all adds up to a personal ATM; at the expense of Canadian Taxpayers!
——————-
Google
Medicine Gone Bad
Dr-no, for being right I do not suppose you could get Edinburgh Medical school to reverse my unsuccessful application to them? Failing that I need a lookout for a kidnap plan. Also I had one questions, is there any book or place where information on NNT’s is collated, I was thinking about my own medical treatment of growth hormone replacement and it got me wondering what the figures would be on such a thing. I assume genuine replacement therapy (as opposed to that ‘HRT’)for endocrinal deficiencies would be something like 1.00001?
Also, how would NICE go about adjusting for this change? Say cancer treatment x cost £10,000 for a years treatment and worked for 1/5 people. Treatment Y costs £20,000 but with genetic profiling you get a 50% ‘hit rate’. Arguably treatment y is in one way cheaper, treating one patient successfuly only costs £40,000 yet, however treatment x is more expensive, but more equitable as it can be prescribed with no distinction between patients.
M – going by what you have written here, you are Edinburgh’s loss. Maybe try again? Or try elsewhere? If at first you don’t succeed…
There’s a lot of adequate stuff on NNT’s on the web (Bandolier is a good sound place to start), but I haven’t seen a book that brings it all together. The Witch Doctor was kind enough to suggest recently that I collate my ‘bad stats’ posts into a book – maybe I should give it a go!
The NNT for replacing something that is missing (eg a hormone or insulin in diabetes) should be 1 (unless the diagnosis is wrong!).
Your lottery/equity question (another very good question – Edinburgh’s loss again!): assuming all other factors are the same (and often they wont be and might thereby force the answer one way or the other) I think the key to the answer might be that if our supposition is right (that response is down to genetics) then, regardless of cost, case two is better, because in case one only one of the five will respond, which means four out of the five are (a) exposed to the risk of harm without any possible chance of benefit (the most important reason) and (b) are given ‘hope’ when there is absolutely none – which seems perhaps somewhat cynical to Dr No’s jaundiced eye.
Having said that, ‘hope’ is very important, and going with case two means that the five people in case one will be denied the chance to hope for improvement…but on balance case two wins for Dr No (because of the risk of harm in patients who can’t possibly benefit).
Whether the general public would agree is another matter altogether. Given their approach to the National Lottery, one might find they favoured case one because more people get to play the game, even is the end result (one patient benefits) is the same in each case!
DR No – This is my third application cycle, so if I strike out this time (Durham is my last hope and I should hear from them today) I really could not face another application cycle as an undergraduate. I Already I have spent two years doing manual labour, which while good for my health and understanding the reality of the world some live in, pales in comparison with the degree I could have been getting instead. Its frustrating as well because its not A levels, admissions tests or understanding of the career which puts me behind but rather my inability to act normally around interviewers for fifteen minutes when it is something I want so much.
The bandolier website is very interesting, though collation is tricky and much of it goes over my head sadly. If you did write a book I assume you would go with conventional publishing? I suspect it would be tricky, given frankly, the lack of sex appeal the intricacies intolerable parts of the English medical health system has. Afterall, the only blogger to writer (whose name I forget) I know was that post graduate who funded herself in a very atypical nature.
I agree with your summation that case two is certainly a better option and likewise I suspect everyone would want a lottery ticket, not realising that this ‘ticket’ is presented to them before they possibly develop an illness but at birth, thus they have already benefitted from an equal chance (though I cannot seem to explain that idea lucidly). I definitely fear the rise of populism in governance of this nation and especially the NHS, though I guess that if this was at interview I guess I would have led myself into the classic ‘are you not being a bit paternal assuming you know better than what the public wants’ type of argument which is almost impossible to come out of looking good, almost as disastrous as when they place a nurse or other health professional and ask you to justify why not their career, all the time being wary of making any actual comparison which obviously requires you to put one career above the other for your personal interests, which is terribly awkward.
M – I did realise after I had posted my last comment that you may already have made a number of applications. I hope my “if at first you don’t succeed…” wasn’t too patronising.
The book, if I do it, will be conventionally published. But you are right about the inherent lack of ‘sex appeal’. I tried to get round this once but Rita Pal accused me of being a shameless hussey…
I do hope the news from Durham is good.
M: If you havenot yet come across it you might find “Reckoning with Risk: Learning to Live with Uncertainty” by Gerd Giggerenzer (Penguin £9.99 – but less at Amazon) of interest in your quest. Ben Gold’s very cheap book “Bad Science” also includes references to NNT estimates at various points and is a very good read.
For something a little “out of left field” readers of Bad Medicine might also like to look at “Understanding Variation : The Key to Managing Chaos” by Donald J Wheeler – SPC Press.
SPC Press try to spread the word among managers generally about the value of statistical process control and measurement to better “understand the voice of the system” before contemplating short term panic interventions such as the imposition of top down targets! BTW I came across this book when developing software solutions for – among other matters – continuing process improvement at GlaxoSmithKline for their I.T. programme. The senior GSK I.T. staff worldwide were expected to have read Wheeler’s book and I understand it has had a similar evangelical impact on other large corporations (but not the NHS – sad to say).
Good luck with the interview process and – just in case you do not succeed in the medical career – have you considered the Computer Science Dept at Edinburgh University? Jeff Tansley – who I have worked with on a number of occassions in the past – still has links there, and I’m sure continues to look out for people who understand both clinical processes and statistics to contribute to research and help to develop decision support systems for use in health care. If that sort of alternative career has an appeal he might be able to point you in the right direction.
Dr No: I’m looking forward to your eventual publication and ready with my credit card.
-Dr No, I thought I would let you know, I got my offer today, I am going to Durham. I do not think I really cannot remember being this happy in the last five or so years. Obviously now I have climbed the molehill of getting in, it is the mountain of being a good student and working on being able to coax a smile out of my face so I can develop a better bedside manner. Was there meant to be a link on once? I would very much like to see you being accused of being a ‘shameless Hussey’
PrisonerOfHope Both books look fascinating, I have stuck them on my wishlist. I am currently working out what I will reward myself with for finally getting in. If I were not doing medicine I would probably consider something in computers, I did a little bit of programming when I was younger and enjoyed the problem solving aspects.
M – how kind of you to think of letting us know on a day when you must be overwhelmed with relief and excitement.
You will know from what you have read here and elsewhere that medicine is in a state of turmoil, and practising medicine in the 21st Century is going to be more of a challenge than it has ever been (but as always ludicrously worthwhile). I have no doubt from your comments that I have read that you will be more than able to meet that challenge.
I wish you all the very best.