Can We Trust Drug Interaction Research?

Over the past several years, many publications in the medical literature and the lay press have lamented the large percentage of study results that cannot be replicated in subsequent studies.^1-5 One of the pioneers in this field is John Ioannidis, a professor at Stanford University School of Medicine, who created quite a storm with his article “Why Most Published Research Findings Are False.” Does this apply to published drug interaction research, as well?

Problems with Published Literature

Before looking specifically at drug interaction research, what about the general claim that most medical research findings are false? It is generally agreed that Dr. Ioannidis and the many others who have made similar claims have compelling evidence on their side. Indeed, many leaders in academic medicine and editors of medical and research journals almost seemed relieved that someone finally said the “emperor has no clothes.”

The reasons for the publication of false and misleading results are many, including obvious factors, such as errors in statistics or study design, financial bias in favor of a drug or product, and, rarely, cases of actual fraud. However, many other factors are not as obvious, such as (1) selective publication of studies showing positive results and “burying” of the more prevalent negative results, (2) the tendency of journals to publish only positive results, (3) the need for researchers to produce positive results to promote their careers, and (4) the problem of false-negative and false-positive results inherent in scientific research, even if a study is done correctly.

It might be thought that these problems with submitted papers would be screened out by reviewers; however, it has become clear that the peer-review process is also deeply flawed. This was shown to be worse than anyone thought when a paper in which 8 major errors were intentionally inserted was sent to more than 200 reviewers of the British Medical Journal. Not a single reviewer caught all 8 errors; on average, they caught fewer than 2 errors. Only about 1 in 3 reviewers felt this highly flawed paper should be rejected. Since this information was revealed, many journal editors have tried to improve their review processes; however, it seems unlikely that the problems will be easily or quickly corrected.

Drug Interaction Studies

There is no reason to believe that articles published on drug interactions are exempt from the problems discussed here. However, there are some differences between the typical drug interaction paper and the types of papers already described. Many drug interaction papers are case reports, and after reviewing thousands of such reports over the years, we have found that they regularly fail to establish a causal relationship between the drug interaction and the adverse outcome. This is why we developed the Drug Interaction Probability Scale (DIPS) to help individuals who publish drug interaction case reports to estimate the likelihood of a causal relationship.⁶

On the other hand, studies showing that one drug affects the pharmacokinetics of another are probably less likely than average to have flaws. These studies are often straightforward. For example, suppose a CYP3A4 substrate is given with and without pretreatment with a CYP3A4 inhibitor to healthy subjects in a randomized, double-blind, crossover study. We know a priori that an interaction is highly likely, as well as that the study is done mostly to determine the magnitude of the interaction, the time course, and other features useful to the clinical evaluation of the interaction. Nonetheless, it is certainly true that some pharmacokinetic studies are poorly designed and produce erroneous results, so vigilance is still required.

Epidemiologic drug interaction studies looking for adverse outcomes, however, suffer from all the difficulties outlined by Ioannidis and others, as shown by years of debates over the interaction between tamoxifen plus CYP2D6 inhibitors or clopidogrel plus proton-pump inhibitors. Epidemiologic drug interaction studies are needed and useful, but great care must be taken in evaluating their results.

End Note

Serious problems with the published medical literature have been identified over the past few years. The drug interaction literature, especially case reports and epidemiologic studies, also has problems with reliability; however, pharmacokinetic drug interaction studies are probably somewhat less susceptible to the same errors.

Drs. Horn and Hansten are both professors of pharmacy at the University of Washington School of Pharmacy. For an electronic version of this article, including references, if any, visit www.hanstenandhorn.com.

References

• Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
• Freedman DH. Lies, damned lies, and medical science. The Atlantic website. theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/. Accessed October 3, 2016.
• Unreliable research: trouble at the lab. The Economist website. economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble. Published October 19, 2013. Accessed October 3, 2016.
• Wilson WA. Scientific regress. First Things website. firstthings.com/article/2016/05/scientific-regress. Published May 2016. Accessed October 3, 2016.
• Gobry P-E. Big science is broken. The Week website. theweek.com/articles/618141/big-science-broken. Published April 18, 2016. Accessed October 3, 2016.
• Horn JR, Hansten PD, Chan LN. Proposal for a new tool to evaluate drug interaction cases. Ann Pharmacother. 2007;41(4):674-680.