Trials: Stochastic simulations of clinical-stage therapeutic programs
Probability of success and costs associated with clinical trials
The “Trials” series of articles discuss the process of drug development and clinical trials. These articles offer a rigorous, data-driven perspective that unifies science, medicine, and business.
Highlights and takeaways
Predicting the difficulty and associated costs for drug development can be a challenge. However, there are efforts to systematically examine the expenditure and probability of success of clinical trials, as related to the study phase and therapeutic area.
I simulated the likelihood of approval and associated expenses for hypothetical pipelines across different therapeutic areas. Emerging from this analysis were indication-related characteristic distributions, but the results could be refined by repeating the approach with a higher quality dataset.
Despite the risk and expenses, clinical trials can lead to better treatments and improved patient outcomes. Reducing costs could be done by, for example, forming corporate partnerships or narrowing patient selection.
Clinical trials are complex, expensive, and prone to failure
Clinical trials are conducted to test hypotheses, e.g. this drug candidate is superior to another drug (or placebo), and hopefully improve the standard of care. Since clinical research involves vulnerable patients (or healthy volunteers), it is highly regulated and requires considerable expertise and resources. Hence, there is a hefty financial burden with drug development. The expenses in conducting clinical trials are largely driven by clinical procedures (~20%), administrative staff (~15%), and site monitoring (~12%) (ref). Clinical procedures include physical exams, electrocardiograms, questionnaires, and clinical assessments, and expenses are highly variable in phase 1 studies. Increasing the number of clinical procedures will also elevate the costs associated with trial monitoring and sample storage. This corresponds with extra data collection, which may not be directly useful, but could enable investigators to explore additional hypotheses and make further observations. In contrast to clinical procedures, administrative staff expenses are highly variable in phase 3 studies. Site monitoring includes collection and review of case report forms, source data verification, maintenance of regulatory binders, drug accountability, and query resolutions. The remaining ~53% of expenses incurred from patient recruitment and retention, nurses and research associates, physicians, data collection and management, etc. Of course, each given study can have a completely different cost distribution, even when compared within the same therapeutic area and phase.
Source: Sertkaya et al. 2016
In addition to exhausting resources, clinical trials are also highly prone to failure. The probability of success can complicate decisions to enroll patients, as well as to finance the investigation. In a recent work, hundreds of thousands of clinical trial entries from Informa Pharma Intelligence databases were analyzed to ascertain probability of success across various attributes (ref). An algorithm was developed to trace the path of drug development and infer phase transitions, rather than immediately calculating probabilities from observed data. While previous studies have used the “phase-by-phase” method, this “path-by-path” method has enhanced accuracy in computing trial success rates. For example, consider the development of three assets, where the first program appears to be missing its phase 2 entry but has phase 3 entry and even regulatory approval. If based solely on the observed data, the phase-by-phase method would inaccurately compute the likelihood of transitioning due to the missing entry. Since the data shows that the program was able to advance, it can be inferred that the phase 2 trial did occur despite the missing entry.
Source: Wong et al. 2018; table was appended to aid visualization
Sertkaya et al., Wong et al., and others have calculated the typical probability of success and expenses associated with each phase and across therapeutic areas. Generally, phase 1 studies are found to be the most likely to succeed. In some indications, phase 2 studies could have a lower probability of success compared with phase 3 studies. Furthermore, success rates vary substantially by therapeutic area. For example, oncology drugs have the lowest likelihood of approval, while vaccines (not shown in the table; see the Wong et al. study) have a relatively high rate of approval. Notably, cardiovascular phase 1 or 2 studies have low expenditure relative to other indications, although phase 3 studies have quite high expenses. It could be related to, for example, the number of patients or clinical procedures for the different indications, as discussed earlier.
Simulating the clinical development of a therapeutic pipeline
Consider a hypothetical company with a portfolio of ten drug candidates, all of which are intended to treat diseases within the same therapeutic area. Perhaps this is due to streamlining clinical investigation and recruiting appropriate expertise. This company needs to adequately finance the development of these ten drug candidates, but capitalization really depends on the probability of success across the portfolio. That is, therapeutic programs only continue to incur costs if it advances to the next phase, and realistically many drug candidates will fail. This process can be simulated within a stochastic framework to assess the likelihood of getting a certain number of drugs approved, and correspondingly how much capital will be required. Each of the ten programs will progress through the trials based on the probability of success of phase-to-phase transitions, and the incurred expenses will be tracked. For every therapeutic area, 1000 trajectories were simulated to create a reasonable distribution.
The distribution is quite spread out on the approval axis, with the median outcome ranging between 0 and 5 approved drugs. Obviously, greater expenditure corresponded with drug approvals, as these programs progressed through more phases of clinical trials. Furthermore, there are noticeable differences based on the therapeutic area. Given the generally low probability of success to transition between each phase, oncology programs tend to have the lowest number of approved drugs, if any. Compared with other therapeutic areas, cardiovascular and ophthalmology programs have significantly more expensive phase 3 trials. Cardiovascular programs have a narrow financing distribution, likely due to the lower costs of running phase 1 and 2 trials. In contrast, CNS programs have a wide financing distribution. Ophthalmology programs have a shifted financing distribution, as there were very few simulations which had zero drug approvals.
Although there are interesting observations, this specific analysis has some limitations. First, the costs were not adjusted for inflation in the original study, and the results were not disaggregated. Second, the trials for estimating costs were not necessarily the same trials for estimating probability of success, as these were sourced from two different studies. However, the approach could be repeated with higher quality datasets, and more reliable conclusions could be drawn from the analysis.
Justification for and mitigation of clinical trial expenditure
Developing drugs is a risky endeavor, in regards to both exposing patients to potential harm and expending sizable resources. Clinical trials are complicated and fraught with challenges that could reduce the probability of success. Yet, there is an important justification for conducting trials and the resulting benefits could be huge. If successful, future patients would receive more efficacious treatments, or even the first treatment ever approved (e.g. those with a rare disease). While clinical research is very difficult and costly, there are increasingly more novel drugs being approved each year (ref). With each new treatment, patient outcomes and quality of life are improved.
Source: Brown and Wobst 2021
In addition to being difficult and costly, drug development takes a long time. Most drugs take 4 to 10 years in clinical development before achieving FDA approval. Even some could take two decades before getting approved. The patent life for novel drugs is typically constrained to 20 years, so the limited time for commercialization must allow for a serious return on investment. This necessitates a prediction on revenue generated by sales or an acquisition by big pharma (which is still dependent on the financial projections). The net income must be sufficiently sizable in order for a biotech startup and its investors to risk capital in developing a drug.
On the other hand, reducing expenditure is also critical for enabling drug development. Clinical trials are extraordinarily expensive, and require distinctly different expertise from the usual biotech startup founded by scientists. Thus, partnerships, or even selling assets/programs, may be necessary for balancing the costs and potential rewards in a maturing therapeutic pipeline. Expenditure can also be attenuated by having a better understanding of previously conducted trials, for example selecting fewer patients based on a conservative inclusion/exclusion criteria. This could also improve the probability of success for the program. However, it is generally difficult to collect and assess data on clinical trials. This is in part caused by the low compliance with reporting results on clinicaltrials.gov (ref). Dishearteningly, only ~13% of trials report summary results within a year of trial completion. Sponsors may be unwilling to expend resources and time for preparing the report summaries, and generally penalties for “failure to submit” are not enforced. Adjusting the regulatory expectations and alleviating the bureaucratic burdens related to reporting could facilitate better transparency, and thus lead to cost savings and derisk drug development as a whole.
Author information
Ergo Bio closely follows innovation in the biotechnology space and evaluates interesting drugs and deals. It is run by Vandon T Duong (LinkedIn), feel free to connect! I am a biotech enthusiast and a molecular engineer by training. I am also an avid consumer of news and research around precision medicine.
Ergo Bio pages
Disclaimer
This article serves informational purposes only and should be treated strictly as educational material, not as investment recommendation or legal advice. The information presented may be inaccurate or out-of-date. The contributing authors and editors disclaim liability for any errors or omissions. Any opinions expressed may change without notice. Ergo Bio LLC reserves all rights to the content generated through this resource hub (Ergo Bio Insights).