EFTA01100068.pdf

Dr. Ben Goldacre Dr. Ben Goldacre is a best-selling author, broadcaster, medical doctor and academic who specialises in unpicldng dodgy scientific claims from drug companies, newspapers, government reports, PR people and quacks. As of 2012, he is a Wellcome research fellow in epidemiology at the London School of Hygiene and Tropical Medicine. His first book, Bad Science, sold over 400,000 copies, is published in eighteen countries, and reached #1 in the UK paperback non-fiction charts. His new book, Bad Pharrna, which exposes bad behavior in the pharmaceutical industry, was published at the end of 2012. For this interview, we discuss the extent of the problems that face the medical establishment—the ramifications of an unscrupulous alliance between the pharmaceutical industry and regulators, the suppression of negative studies revealing many drugs to be either ineffective or less effective than those they seek to replace—and what we can do to solve these problems, including the new petition AllTrials.net, for the publication of clinical trial results, at www.alltrials.net. Dr. Ben Goldacre: I think it would be wrong to imply that it is melodramatic or controversial to say that there are flaws in evidence-based medicine. The book Bad Pharma really is about the aspirations of evidence-based medicine and how we have failed to meet those. It about how trials work, but it's also about how trials can be flawed by certain design quirks. It's about how we hope that doctors make decisions based on the results of good summaries of the evidence that's been collected—but in reality we've got very flawed mechanisms for disseminating such evidence to doctors and patients, which means it's very vulnerable to being gamed by marketing departments at drug companies. In addition, half of all clinical trials for the drugs that we use have never been published. Its not really that there was some sudden, big, dramatic moment that led me to write the book; it was more that there were all of these very well documented problems throughout the whole of medicine that we teach students about when they do epidemiology Masters courses or to medical students when they're doing their evidence-based medicine module. I thought it would be useful and interesting to share those shortcomings with the public. Partly because they're just interesting! I'm a nerd, obviously. I like stats and evidence and I study design, but I think everybody does, really. It's the answer to the question, "How do you know if something works or doesn't work?" I'm actually talking about the flaws in trials; that is the best way, really, to talk about how trials are supposed to be designed. That's how you teach things. When you're talking about how to design a proper trial, you do so by using flawed trials in the context of a journal club or whatever and talking about their flaws. Also, I think, something that's worth talking to the public about because I think we've been insufficiently ambitious in medicine about fixing these problems. The clearest example of that is Page I 1 of S EFTA01100068  the problem of publication bias. Publication bias comes about because half of all trials that are conducted and completed have never been published, so the evidence that we do have is completely distorted and biased. I think that we've failed to fix all of those problems behind dosed doors in medicine. We've failed to fix those problems quietly and discretely in professional discussions amongst the senior medical academic bodies and industry regulators. I think the time has come to discuss these more openly with the public. It's not a melodramatic book It just gives a very straight description of all of these problems. Kylie Sturgess: The findings that you uncovered are quite shocking—trial studies being withheld, the accounts you write about Roche and the European Medicines Agency withholding clinical data trials from Cochrane. At certain points, I thought this is incredible, it's almost bordering on the absurd, let alone tragic... Ben: I agree. First of all, it's important to be clear that a lot of it isn't new. The finding that half of all studies have never been published; that's not a revelation that I've made, that comes from the NHS, health technology appraisal people who use very good monographs and they've produced a systematic review just two years ago that looked at all of the evidence on missing data. That's where the finding that half of all trials have never been published comes from. It's not something that I made up out of thin air and it's not a revelation! I can't remember who says it, but there was an investigative journalist who said that sometimes a clearer explanation of a well-documented technical problem, that's protected from public scrutiny by a wall of modest complexity, is valid as a piece of investigative journalism. That's what I've done. I'm not sure I'd necessarily call it investigative journalism, I think it's just...it's pop science, just a clear explanation of these problems. When I put it all together in one place I was surprised by how bad it looks, because you can be aware of these atomic problems individually. You can be aware that half of all trials have never been published, and trials with positive results are twice as likely to get published as negative ones. You can be aware that sometimes there are these design flaws, you can be aware that there are problems in the way that evidence is disseminated to doctors, you can be aware that there are shortcomings around access to data that regulators have. But actually, when you put it all together in one place, you start to see how they all reinforce each other in this interlocking ecosystem of problems that add up and create quite a worrying overall picture. I'm surprised that people haven't made more of a fuss about the overall picture. All of things we talk about individually, they've all been discussed mostly behind dosed doors, in the sense of academic journals, and in quite technical language. But I think putting it all together in one place makes it feel more unnerving. I was surprised, I think, by how worried I felt at the end. Kylie: How difficult was it to get Bad Pharma written? What was the publisher's original response and then going through the process of writing and realizing the eventual size of the book—how did it make you feel? Ben: Initially, I thought it was going to be a very short book! Kylie: Yes, big surprise there! Page I 2 of S EFTA01100069  Ben: My intention was: "There's all these problems. They're longstanding. Lots of people know about them. I've got a bit of a platform now because Bad Science has sold half a million copies, so it's a good opportunity to just take these well-documented problems, put them all in one place, give a good dear explanation of them, and then crack on and write about something else." It was only when I started to write it that I realized, first, you get that obsessive thing of making sure you've covered all the bases. Recently, I went out with Nassim Taleb, who wrote The Black Swan, and he said, "I read your book, and it reads like it's written by the most paranoid man in the world." I was quite surprised: "You mean it's like a loony conspiracy theory?" He said, "No, no. It's written paranoid, because every single step of the way you try to anticipate anything that anyone will say in response to it and head it off. You do that so obceccively to make it watertight, you look paranoid about wanting to stop people from being able to dismiss what it is you're saying." I think that's partly why it ended up being 400 pages, because you want to leave no stone unturned, and you want to close off all the exits. People will try and say, "Oh, well, trial results don't get published but it's okay, because regulators will get them." So then you have to talk about what the problems are with regulators. Then at the same time, I have a day job to consider. I'm working on a project to see if we can find new ways of getting clinical trials to run seamlessly and unobtrusively in routine clinical care using an HS electronic health records with the idea that we can make trials cheaper and get better at collecting good quality data about which drugs work best from representative patients; that takes up a good deal of my time. So then because I was obsessing over all of that, I found myself wanting more and more to weave in stuff about not just the shortcomings in medicine's failure to achieve its own current stated objectives, but also how we could better achieve what we want to achieve. Kylie: Although you said that it may come across as paranoid in terms of checking every base, I thought that was really useful, because it demonstrated that this isn't conspiratorial thinking— you are demonstrating evidence for all of the flaws and the issues. Ben: Yes, and it's not a loony quack book! It's another reason why I wanted to write it; it's very disappointing to me how in mainstream popular culture, especially in the UK, critiques of the pharmaceutical industry are very unsophisticated. First, it's always about individual drugs rather than about systems failures. People want there to be a killer drug and a specific person who is killed by said drug. It's much more difficult to get people in mainstream media to talk about the idea that we've got flawed information for evidence-based medicine, which means that we end up prescribing drugs which will save six out of twenty lives when we could save eight out of twenty lives. There are two deaths more than you should have, but you can't identify who they were, and you can't say that a drug killed them in the sense that it did them more harm than good. It's more that the flawed processes of evidence-based medicine means that people were deprived of a better treatment at the same cost, and so two people have died unnecessarily, and that's quite a hard sell. But those are two very real deaths—and I think part of the real challenge of talldng about flaws in evidence-based medicine is that if you kill someone with your own hands, if you strangle them for instance, it's very obvious that you've killed them. If you killed them being an Page I 3 of S EFTA01100070  incompetent doctor in terms of your management of that patient right there and then by the standards of consensus or evidence, then it's very obvious that you've just killed somebody. But if you are a thoughtless participant in a wider system that produces and disseminates flawed evidence in a way that somebody dies unnecessarily, it's harder to pin that on someone. Columbo doesn't come around wearing his trench coat and turn around and go, "Just one more thing," before he delivers the killing blow that explains why it's all your fault. Also, it's not one person's fault. It's the fault of a whole system, and also I think that is part of the reason why people feel okay about doing these things—which when you read the book seems completely morally reprehensible, stuff like withholding data, running flawed trials, or disseminating distorted pictures of the evidence. One of the reasons why people feel okay with doing that stuff is pretty much everything we do in medicine actually does more good than harm. The problem is if you've got one treatment that can save six lives and one treatment that can save eight lives, and you end up getting people to use the one that saves six lives, well, you could go, "Hey, I've just saved six lives. Look at me. I'm fantastic. Look at my fantastic drug. Our marketing department has successfully got lots of people to adopt this drug which saves six lives out of too," and fails to recognize in doing so you've deprived people of the opportunity to use the drug that would have saved eight lives out of too. You did that by distorting the evidence. You've cost society two lives. You've killed two real people, but at the same time you've saved six people. I think that's how people let themselves off the hook. And so, that all sounds very "people in glass houses shouldn't throw stones"—but I teach evidence-based medicine and epidemiology to students. I feel like I have the same battle with them that I do with the readers of this book, which is to try and communicate to people that forest plots and meta-analysis are not abstract, academic stuff. They relate real world stories of flesh and blood and suffering and pain and bereavement and tears and loss and someone's mum or dad or daughter dying. You have to keep reminding yourself that there is this connection between the abstract world of evidence and the very concrete world of flesh and blood when you're teaching epidemiology but also when you're talking about flaws in evidence-based medicine to a general audience. Kylie: Are the systems and incentives so ingrained that essentially the industries are irreversibly flawed? Is there any way to change things? Ben: Yes, the book is absolutely stuffed full of very simple, very straightforward policy changes that would fix many of the problems that I describe. Actually, the real tragedy of this whole story is how easy this stuff would be tofix. There's absolutely no reason why you should be able to hide the results of clinical trials for drugs that are currently used. There's absolutely no reason why you should be allowed to switch their primary outcome in a trial between the protocol being registered and the paper being published. There's no excuse for any of this stuff and it would be very easy to fix. Of course, an exciting thing that's happened recently is the www.AllTrials.net campaign, which is amazing, really. So you know I feel much more optimistic now. Page I 4 of 5 EFTA01100071  Kylie: It looks like a great step in the right direction when it comes to informing stakeholders about clinical trials. What's been the response? Ben: Well, it's been very positive so far. We've certainly collected a lot of signatures given that we've really only existed on Twitter. We've got thousands and thousands of signatures. People have very kindly donated to cover some of our lobbying costs. Our first big story is that we've collected a number of patients who've participated in trials, a huge list of signatories, saying, "We are patients who participated in clinical trials and are doing something to help inform medical practice. We're appalled to discover that half of the trials for treatments currently in use have never been published, and many of them are completely, deliberately withheld from doctors and patients who need that information to make decisions. We want to know what you're going to do about it." I think that's incredibly powerful. Because it is a ludicrous betrayal, really, of what patients expect when they participate in a clinical trial. To discover, actually, if the results are unflattering to a company they might just bury the results and hide them. To discover that if the results are unattractive to an academic, that they're just going to go, "Oh, those don't support my favorite hypothesis, I'm just going to bury them." That's truly appalling. Just as appalling, really, as the idea that if a researcher says, "I'm just not very interested in that anymore, I'm going to go and do another job now. I'm not going to bother writing it up." I think that's also a serious betrayal of patients. Kylie: Anyone in the world can sign onto AllTrials.net and lend their support, can't they? Ben: Yeah, absolutely, at www.AllTrials.net. Also, if you're an academic or a doctor, you should give evidence to the Select Committee. The House of Commons Parliamentary Science and Technology Select Committee in the UK Parliament announced a couple of weeks ago—it's up on www.badscience.net —that they're going to hold a special inquiry into drug companies and researchers withholding the results of clinical trials from doctors and patients. If you are an academic or a doctor and you're affected by this, or if you're a patient, then it would be really valuable to present evidence to that. If you're a patient organization, anyone who's involved in or affected by this issue, it's really important people give evidence to that Select Committee inquiry, because that's how the government knows that people are taking this seriously. Page I S of S EFTA01100072