The Other Side of Publication Bias

As a result of the 2007 FDA Revitalization Act, all clinical trials must be registered in a registry database. Sadly, neither this law nor any laws since have mandated the reporting of trial results to a results database. Therefore, it is possible that trials are registered, but their results are unreported or selectively reported.

For example, the following is an entry from ClinicalTrials.gov:

And therein lies the problem.

The following is an excerpt from the linked journal publication:

“Aripiprazole was added, initially at 15 mg/day then increased to 30 mg/day. Within a month, Ms. V noted her mood and concentration were improved, and she was no longer anhedonic...After 3 months on venlafaxine extended-release, mirtazapine, and aripiprazole, Ms. V noted that her appetitite remained good, she was sleeping 7 hours per night, her mood was much better, and she had begun to seek a new job, sending resumes, and phoning prospective employers.”

Everyone can be happy to read about Ms. V, but what about the other 14 patients in the study? And what about the outcomes that the investigators set out to measure?

A recent meta analysis in the New England Journal of Medicine identified phase 2 and phase 3 studies on 12 FDA-approved antidepressants from 1987 to 2004 with a total of 12,564 adult patients and then compared that study data with matched data from journal articles, if published. Unsurprisingly, roughly 30% of clinical trial data went unpublished, translating to 3449 patients with data that never officially existed, while another 15% were selectively reported. For all 11 of these publications representing 1843 patients, the primary outcome was nonsignificant, but each highlighted a positive result as if it were the primary outcome.

In a statement, Matthew Bennett, senior vice president of PhRMA, summarized the opposing argument to releasing detailed study data: “It could not only lead to misinterpretation of the risks and benefits of medicines, and potentially interfere with patient confidentiality, but would also deter future medical innovation if would-be competitors could access confidential commercial information.”

Fortunately, the tides seem to be turning in spite of this notion. The British Medical Journal announced in 2012 that it will no longer publish results of clinical trials unless drug companies and researchers agree to provide full study data upon request. Soon after, the journal’s editor, Dr. Fiona Godlee, said she expected “that eventually this will become the norm.”

Recently, the US Department of Health & Human Services proposed new rules for academic researchers to post findings through ClinicalTrials.gov. The Institute of Medicine (IOM) also released a 249-page report that called for a complete data package made publicly available no later than 18 months after a study is completed, asserting that “all stakeholders, from drug and device manufactures, to medical journals and regulatory bodies, have the responsibility to the public and to the voluntary participants in the trials.”

Notably, the IOM report lacked actionable metrics and has been criticized for not going far enough. Moreover, a recommendation is no mandate, and to what extent such a recommendation will be adopted remains to be seen.

The only reason a mandated trial results database does not exist is because it has not yet been demanded. I think it is time that we did so.