Sponsors will have to plan carefully — and show their work — if they want to use meta-analyses to prove drugs are safe, the FDA says in new draft guidance.
Randomized clinical trials sometimes have trouble creating big enough sample sizes to detect adverse events and to determine whether they’re linked to the meds being tested. In those cases, sponsors can combine the results of multiple studies and project out safety measurements — if they carefully detail how they chose and analyzed the data, say the proposed guidelines issued last week.
Meta-analyses should focus on the quality rather than the quantity of data and avoid bias at all costs, the guidance says, noting that the FDA is more likely to accept safety data from them if sponsors can provide:
“Background information available at the time of the protocol development that motivated the meta-analysis”;
Analysis design features, including target patient population, how outcomes were defined and measured, the length of time the data covered, and safety data from similar drugs;
Sources of pooled trial data —and the criteria researchers used to accept or reject it for the meta-analysis; and
Details on how researchers analyzed the pooled info — and reached their conclusions — including any subgroup and sensitivity analyses that played a role.
“Our goal is to encourage more pooled studies to evaluate important safety events,” FDA Commissioner Scott Gottlieb said in announcing the draft guidance. But because the meta-analyses can influence regulatory decisions, “rigorous principles should be applied to these studies,” he added.
Bias appears to be one of the FDA’s top concerns. “In general, reducing bias in a meta-analysis should be given greater weight than increasing precision and power,” the draft guidance states.
The agency also cautions researchers against relying too heavily on published articles.
“Information taken from published articles about the component trials may be incomplete or lack specificity,” the draft guidance warns. “Publications may not report on the safety outcome of interest, and, even when the outcome is reported, important details may be lacking. ... Protocols, study reports and subject-level data for the component trials are often important to determine whether the trial outcomes are adequate for supporting a high quality meta-analysis.”
Joshua Wallach, an expert on meta-analyses at Yale University’s School of Public Health, called the document an important first step. He said he’s encouraged by its emphasis on transparency but hopes the agency will include more direction on how to handle the complexities of data in meta-analyses in its final guidance.
“Standard meta-analysis methods often focus on efficacy — does an intervention work? But when it comes to safety, things are a little more complicated,” says Wallach, an assistant professor.
In particular, he would like more FDA input on so-called zero-event trials, noting that “they require more complicated techniques, but can provide insight about the frequency of outcomes, add to the total amount of information and they can change the perspective of the analysis.”
