The unplanned costs and delays associated with protocol amendments have prompted many sponsor companies to identify new approaches to simplify protocol designs and reduce the frequency of protocol amendments over the course of the past few years. Yet a new Tufts Center for the Study of Drug Development (CSDD) analysis found that the majority of protocols still require substantial amendments, which led to significantly longer clinical trial cycle times and higher costs.

The new analysis builds on a 2010 Tufts CSDD study that, for the first time, quantified the prevalence and causes of protocol amendments. It found that 57% of protocols had at least one substantial amendment and nearly half (45%) of these amendments could have been avoided, compared to 33% in 2010. About one in four (23%) amendments were implemented before the first patient was dosed.

On average, clinical trials with at least one substantial protocol amendment took three months longer to complete than those without an amendment. Overall, the Tufts CSDD estimates protocol amendments cost the industry a total of $20 billion a year in direct and indirect costs.

“It’s a call to action,” said Rob DiCicco, Pharm.D., vice president of Clinical Innovation and Digital Platforms at GlaxoSmithKline (GSK). “It may be that different initiatives that companies started a few years ago aren’t reflected in the data or that the problem is getting worse because of a variety of factors, including protocol complexity. Either way, there is a massive opportunity for improvement.”

The peer-reviewed study findings, published in the journal Therapeutic Innovation & Regulatory Science, link protocol amendments to performance measures for the first time and offer opportunities for companies to better understand the impact of major changes to finalized protocols.

In the following, CenterWatch looks at highlights from the new Tufts CSDD study and initiatives at forward-looking companies—including Amgen, Pfizer, GSK, Eli Lilly and Parexel—that are designed to improve the quality of study design, reduce the frequency of protocol amendments and better inform the decision-making processes.

Amendments impact performance and cost

The 2015 Tufts CSDD study, which was based on data from 836 protocols provided by 15 large and midsized pharmaceutical and biotechnology companies and CROs, found small signs of improvement in reducing protocol amendments compared to the 2010 study, but the frequency of substantial amendments remained high. The study defined “substantial amendment” as any change to a protocol on a global level requiring approval both internally and from a review board or regulatory authority.

The incidence of amendments in the 2015 study (57%) was below that observed in the Tufts CSDD 2010 study (69%), which might reflect early results from new industry practices designed to reduce protocol amendments. The study authors believe the difference largely could be explained by changes in the 2015 study methodology. The previous survey counted country-specific amendments and those from ongoing studies, while in 2015, only completed protocols and global amendments were included.

Two-thirds of phase III protocols were amended, with an average of 2.3 amendments per protocol. The median direct cost to implement a substantial amendment for phase III protocols, which are typically larger and costlier than earlier phases studies, was $535,000, a higher amount than originally expected. The Tufts CSDD report estimated that total indirect costs for a phase III protocol amendment could be three-to-four times larger. Meanwhile, phase II trials had the highest proportion of substantial amendments (77%), averaging 2.2 amendments per protocol, with a median cost of $141,000 to implement.

GSK’s DiCicco said the study results suggest sponsor companies should focus efforts to reduce protocol amendments on phase III, where the programs are more expensive, take longer to deliver and the risk around making protocol changes carry an element of regulatory risk. “You know more when you go into phase III and ought to be in a better position to get it right,” he said.

While the frequency of protocol amendments decreased between 2010 and 2015, the Tufts CSDD observed that the number of changes per amendment has increased, suggesting that sponsor companies are using new strategies to reduce the number and expense of protocol amendments. Instead of making numerous amendments, some companies have begun to hold off on non-urgent changes and bundle them into the next major amendment that arises. Since the largest costs associated with amendments are for institutional review board (IRB) fees and change orders to vendor contracts, the approach can result in cost savings for companies.

“We have learned that we should be waiting for a significant amount of changes to be required prior to implementing an amendment. So hopefully we are doing something right,” said Derek Dunn, associate director of Global Clinical Operations at Alexion Pharmaceuticals. “Each time you submit an amendment to a regulator and an ethics committee, you get charged for review. There are many internal and external costs, so if you could wait and batch things together and just do an amendment once, it just makes more sense.”

Sponsor companies have also become better at evaluating their own protocols and determining whether amendments could have been avoided. While the proportion of avoidable amendments increased by 12 percentage points since 2010, the Tufts CSDD study noted that the finding might be due more to the study’s classification system than an observed trend. Study participants had a better understanding of how to classify amendments in the 2015 study and were less conservative in what they considered an “avoidable” amendment.

“Clinical trial sponsors are spending an increasing amount of time and effort to ensure that they are correctly classifying the cause of amendments,” said Mike Capone, chief operating officer at Medidata Solutions. “For example, the increased availability and visibility into historic enrollment performance shows us very clearly that certain amendments related to participant enrollment and retention—such as eligibility criteria and demographics—are avoidable.”

Nevertheless, nearly half of all substantial amendments could have been prevented. While amendments are implemented for a variety of reasons, including the availability of new safety data and regulatory agency requests, the top reason for amending a protocol is to change study volunteer eligibility criteria because of changes in study design strategy and difficulties in recruiting patients. More than half (62%) of substantial amendments were implemented during the study enrollment period.

“A high proportion of these amendments are viewed by companies to be avoidable. It really calls us to action. If we can prevent these avoidable amendments, then we can really effect the cost of drug development and the speed with which we get life-changing medicines to patients,” said Jules Desmond, Ph.D., development design director of Amgen’s newly instituted Development Design Center.

The Tufts CSDD report also found that protocols with even one amendment had a substantially lower number of patients screened and enrolled compared to the original plan. Although the amendments had the desired effect of lowering the number of patients needed to complete the study, which can lower study costs, Tufts CSDD said those savings must be offset by the longer cycle times and direct costs of implementing an amendment. In addition, reducing the number of patients in a study could weaken the statistical quality of the data and which signals will be detected.

“If they can’t do their power calculations or even their descriptive statistics based on a smaller number of patients, it’s not going to fly,” said Alexion’s Dunn.

Redesigning the study development processes

Partly in response to the 2010 Tufts study, a growing number of research sponsors have begun to keep metrics on the frequency of protocol amendments, evaluate their protocol design practices and implement new governance mechanisms and processes to improve protocol designs and reduce complexity. In addition, half of the companies that participated in the Tufts CSDD research have begun setting aside funds to assess the cost of amendments and manage unplanned increases in study budgets.

“Drug companies are very cognizant of the impact of amendments to trials and their businesses. Sponsors are increasingly leveraging new technologies in protocol design optimization, investigative site and patient feedback panels and protocol review committees to help mitigate the impact of protocol changes,” said Medidata’s Capone.

Amgen dramatically changed the way it designs programs and individual studies by initiating a new design process last year that incorporates cross-functional, data-driven and real-time development principles. A new clinical development capability, called the Development Design Center, partners with teams to design better program-based studies. The center brings data sources; predictive analytics; local expertise resources, including feasibility managers in countries worldwide; and specialist clinical development expertise. The center packages these elements into a framework for teams that helps facilitate decision-making and better understanding of the impacts of various design trade-off decisions. Once data has been collected and options mapped out for teams, the process ensures all decision-makers have the chance to discuss the information in a collaborative manner.

“We believe the greatest opportunity to affect cost and cycle times exists at the time you design programs for studies. This Tufts study reaffirms it,” said Desmond. “The ability to reduce substantial amendments is or will be a key contributing factor to savings. But rather than address the proximate cause of amendments on a case by case basis, we like to frame amendment reduction as being a byproduct of good design. So we asked ourselves, how can we improve our design process at Amgen?”

As part of the changes, study design was separated from protocol authoring in order to realize time savings and focus individuals on study design in the initial stages. During the design portion, teams develop all of the design elements for the study along with an operational plan to ensure the design can be executed. The study design then gets reviewed and approved in a first stage of governance. The study design moves to a protocol-authoring step, where the design is translated into a protocol. At this point, the design is locked-down and cannot be challenged. While the protocol is being produced, the rest of the organization can begin study startup activities based on the approved study design.

“That is where we save time. We are not waiting around for a final polished protocol before we begin study startup work internally,” Desmond said. “Once the protocol is complete, it is reviewed/approved in a second stage of governance.”

At Lilly, a series of protocol improvement initiatives was adopted in 2013 to strengthen protocol quality and feasibility up-front in order to reduce the types of amendments caused by planning issues, such as recruitment or investigator requests, or those associated with errors or inconsistencies within the protocol. The initiatives were aimed at three specific objectives: simplify and focus the protocol design, incorporate patient-centered approaches and streamline the drug development process.

The initiatives included a redesign of the study development process for phase II-IV programs to better engage patients, study sites and investigators. One effort uses feedback from patients about their clinical trial experiences to inform future protocol design. Once the initial concept of the study has been determined, patients and study coordinators simulate the protocol while the study team watches in order to identify and address feasibility issues that could potentially trigger amendments before protocol approval.

Study teams learn about the new process through training workshops held just before the design phase begins for a clinical trial, which allows them to talk about feasibility issues within the context of a specific protocol and receive support in implementing changes identified through the program. The clinical research physicians and scientists who write protocols also participate in training sessions focused on improving the clarity and effectiveness of protocols along with minimizing non-essential protocol requirements.

“We want our study teams to recognize that the greatest cost of needing to amend a protocol for feasibility or planning issues is the extension of the clinical study duration,” said Mary Short, research advisor at Eli Lilly. “We have been committed to finding innovative ways to improve the speed and quality of our development processes so that we can actually get new medicines to the people who need them faster.”

Lilly measures the impact of its protocol improvement initiatives using a specific time frame: from protocol approval to within 100 days after the first patient visit. Since the program began in 2013, Lilly has seen a 50% reduction in amendments that were related to feasibility, recruitment, investigator requests and planning. The new processes also resulted in a significant reduction in the number of amendments due to internal errors, inconsistency or a need for clarification.

Protocol review processes

Both Pfizer and GSK have implemented extensive internal review processes to improve protocol quality and reduce unplanned and unbudgeted amendments.

At Pfizer, protocol review committees have been used for many years by some groups across the organization, depending on therapeutic area or phase of development. The company recently revised its organizational standard operating procedure (SOP) to require that all protocols go through a detailed protocol and amendment review prior to implementation. The protocol development has been distilled into a three-step process that will be applied universally to all protocols across the organization.

The first step of the process calls for a review by a senior management governance committee in the applicable therapeutic area. This review is intended to achieve consensus on the design elements of the study and ensure the protocol is consistent with the overall development plan for the asset. This review committee endorses design elements such as inclusion/exclusion criteria, objectives and endpoints and dose selection. The study team then writes a detailed protocol, which goes to an independent review committee made up of a diverse group of individuals, including clinicians, statisticians, clinical pharmacologists and operations experts. The group reviews the entire document to ensure consistency, readability, operational feasibility, clinical safety and scientific integrity.

The last step is a technical quality control review where a group of individuals familiar with the various protocol templates used at Pfizer further ensure consistency across sections of the document. This group also makes sure that study teams have used the correct template for their protocol and that all mandatory information has been included.

“The overall goal is to improve protocol quality, better ensure study success, reduce the number of protocol deviations and reduce the number of protocol amendments,” said Pfizer Executive Director David Kazierad, Pharm.D., who is the clinical lead in the cardiovascular metabolic research unit and business process owner (BPO) for protocol authoring.

GSK, which began one of the earliest facilitated clinical review processes in 2011, has reduced the average number of amendments per protocol by more than 20%. All phase II and III protocols are reviewed by a panel of experts, both from clinical and operational roles, late in the development stage. GSK considers the process a good return-on-investment because study teams don’t need to create new materials for the review; they are asked to give the committee whatever information they already have. The peer-review group uses regulatory correspondence, and the protocol and what they know about the space, to ask whether the inclusion/exclusion criteria are realistic, if all of the procedures are necessary and linked to important endpoints in the protocol, whether the burden on patients is reasonable and if the protocol aligns with feedback from regulators. The review team makes recommendations about the protocol, but the final decisions about changes are made by the study and project teams.

“If you marry the Tufts report to their prior work on protocol complexity, which looks at the amount of data collected, the number of procedures that are performed and whether or not they are actually related to a primary endpoint, the two things go hand-in-hand. For the teams that go through our facilitated review process, there is a clear reduction in amendments. The issue then becomes a question of whether 20% is enough? The answer is probably not,” said GSK’s DiCicco.

Other industry initiatives underway

Last year, Parexel launched its Clinical Development Optimization process and service offering, which includes a component aimed at reducing flaws in protocol design that can lead to amendments and study delays. Standard protocol elements—including endpoints, sample size, study design, study procedures and inclusion/exclusion criteria—are evaluated to determine whether the protocol makes sense from scientific, regulatory, operational and commercial perspectives. Protocol feasibility can be tested in silico, using modeling and simulation in virtual populations, and in practice through Parexel’s phase I units. As part of the evaluation, Parexel generates alternative designs or approaches to running the same study.

“If we spend more time up front, it will pay off handsomely later. In particular, we should spend more time compiling and reviewing our study protocols and conducting proper feasibility regarding the intended study population and the standard-of-care around the world. Doing this well, even if it takes a little bit longer, will almost inevitably end up saving time and money in the longer term,” said Sy Pretorius, M.D., chief scientific officer at Parexel.

TransCelerate BioPharma, a nonprofit organization comprised of about 20 of the largest pharmaceutical and biotechnology companies, has made protocol feasibility one of its top areas of focus, and recently released a common protocol template that can promote greater efficiency in protocol review processes. More than 90 entities, including member companies, government agencies, academic institutions and small biotech companies already have downloaded the template from TransCelerate’s website.

DiCicco, the workstream leader for TransCelerate’s Common Protocol Template project, said that by using a common structure and model language for protocols, it becomes more obvious when there is a misalignment between protocol objectives and endpoints in data collection. The common protocol template also makes automation possible. This allows companies to reuse libraries of information instead of recreating them manually, which could lead to human error.

Automating a common protocol template also sets the stage for using advanced analytics to inform protocol design and improve protocol performance.

DiCicco said clinical areas have been slow to adopt the type of quality by design processes used by the manufacturing, laboratory and preclinical areas. Yet he said various industry initiatives, including feasibility review committees, common protocol templates, investigative site and patient feedback panels, all have the potential to “review quality back into the protocol.”

DiCicco said, “There isn’t one magic potion or one magic solution. It has to be the quality by design element, harmonization and simplification of protocols through things like TransCelerate, and it has to be modernization of the clinical trials process where you can actually use analytics from the data that you have to inform your protocol.”
Looking forward

As sponsor companies face growing pressure to accelerate the development of new drugs while reducing costs, the Tufts CSDD study findings provide opportunities to better manage and reduce significant costs and delays associated with major changes to finalized protocol designs. Sponsor companies and CROs should continue to develop programs and mechanisms that challenge the executional feasibility of their study designs and help prevent avoidable amendments.

“The prevalence of substantial amendments remains high,” said Desmond. “It’s very powerful for an organization to actually see on paper what the effects are in terms of cycle-time increase and financial cost increase. We owe it to our patients to get our medicines to them quickly and minimize delays wherever we can. Studies like this really focus our attention on areas that we can improve.”

 

Karyn Korieth has been covering the clinical trials industry for CenterWatch since 2003. Her 30-year journalism career includes work in local news, the healthcare industry and national magazines. Karyn holds a Master of Science degree from the Columbia University Graduate School of Journalism. Email karyn.korieth@centerwatch.com.

This article was reprinted from Volume 23, Issue 04, of The CenterWatch Monthly, an industry leading publication providing hard-hitting, authoritative business and financial coverage of the clinical research space. Subscribe >>