Here is a summary of an interesting article on how ALS clinical trials could be improved.
Introduction
Clinical trials typically proceed sequentially, through early, middle, and late phases, often referred to as Phases 1, 2, and 3. Phase 2 trials are generally designed to gather information about the safety, tolerability, and dosing of an experimental treatment, and to determine whether the therapeutic agent shows sufficient promise to warrant further study in a large, randomized, controlled Phase 3 study. In the field of amyotrophic lateral sclerosis (ALS), approximately several dozen drugs showed “promise” in phase 2 but failed in phase 3.
Much has been written about the many potential reasons why so many trials have led to so few effective treatments and how we might learn from this experience.
Duration
Historically, phase 2 trials in ALS have varied in duration, but a 6-month placebo-controlled phase followed by an open-label extension (OLE) seems to have become a common standard, for example the Healey Platform trial. The phase 3 VALOR study of tofersen in SOD1 ALS provides useful information on the length of time needed for a clinical trial in ALS. In this 6-month trial, significant reductions in CSF SOD1 protein levels (a marker of target engagement) and plasma neurofilament light chain (NfL) (a marker of axonal degeneration rate) were observed at 12 weeks. A meaningful clinical benefit did not emerge until 12 months in an integrated analysis of double-blind and open-label extension (OLE) data.
The long latency is notable given that the SOD1 ASO targets a biological mechanism that causes disease. This study showed that clinical benefits may take time to be detectable. Longer treatment periods may be required for clinical effects to become apparent for therapies that target more downstream biological mechanisms.
The VALOR study also showed that a significant reduction in NfL was detected over a shorter time period than meaningful clinical effects, providing an early confidence signal, later reflected in the FDA’s groundbreaking recognition of NfL as a “reasonable surrogate marker predicting clinical benefit in ALS” in its decision to grant accelerated approval to tofersen. The problem of a long latency to show clinical benefit is not unique to the ALSFRS-R.
A short study duration therefore risks missing potential delayed clinical effects, increasing the likelihood of a false negative result. This risk can be partially mitigated by implementing an OLE (non-randomized extended drug availability), which allows time for delayed effects to become evident during the extension phase. A long OLE, however, cannot fully compensate for a short placebo-controlled period.
But the longer the OLE period, the more similar the groups become, diluting a potentially effective treatment effect and possibly increasing the risk of statistical error. The results of these studies are confounded by incomplete follow-up, differences between completers and non-completers, and selection bias in those eligible for treatment.
Outcome measures
The ALSFRS-R measures the patient's functional independence with a set of activities that reflect bulbar, fine motor, gross motor, and respiratory muscle function. It is a relatively subjective measure of patient functioning, and a slower rate of decline, or a higher score, correlates with longer survival. The heterogeneity in the rate of disease progression among patients, manifested by enormous variation in the rate of change in the ALSFRS-R, generally makes phase 2 trials that rely on the ALSFRS-R or similar subjective measures of insignificance. The use of the ALSFRS-R is preferable in phase 3 trials that aim to answer questions of clinical efficacy.
In addition, the selection of the ALSFRS-R as the primary outcome measure for a phase 2 trial suggests a desire to demonstrate clinical efficacy. An overreliance on the ALSFRS-R as the primary outcome measure may also lead investigators to seek drug approval based on a single phase 2 trial of small size, as was the case for AMX0035.
If the ALSFRS-R is to be used in phase 2 trials, these results need to be supported by biomarkers, such as baseline blood NfL and the ENCALS prediction score. Although much is not yet known about NfL, it is undoubtedly one of the most promising candidates to date to help select participants for a phase 3 study.
Missing Data
For both trial duration and primary endpoint selection, the question arises as to how best to deal with missing data, whether due to death, treatment interruption, or patient “evaporation.”
Strategies such as the last observation carried forward (LOCF) used in the masitinib phase 2/3 trials, the edaravone phase 3 trials, and the post-hoc analyses of the CENTAUR AMX0035 trial data are subject to bias given the assumption that the outcome is constant after treatment is stopped. This is difficult to justify for diseases such as ALS, which are progressive over time, particularly when instruments such as the ALSFRS-R are used to measure the outcome.
This problem was well illustrated in the FDA’s analysis of the CENTAUR trial data, where differential treatment of deaths with accompanying missing data showed a loss of statistical significance.
The Dangers of Statistical Manipulation
Multiplicity is an important consideration in the analysis of clinical trial data. It occurs when multiple significance tests are performed, for example, opportunistic and dishonest selection of different outcomes such as measuring outcomes at multiple time points, using multiple doses, or different study populations and subgroups. Performing a statistical test at p < 0.05 has a 5% chance of finding a false positive on one type of measurement, but increases to 40% for performing ten statistical tests.
Interpretation and communication
The lack of biological and mechanistic information on many failed drugs significantly limits the lessons that can be learned from them.
The absence of primary and secondary outcomes in the main study population, but the discovery of a “hit” in one (or more) exploratory outcomes in a subpopulation, even if pre-specified, especially when financial considerations are taken into account, leads to a real risk of over-optimistic interpretation and unbalanced (and dishonest) communication of results. The release of the results of several recent trials provides a useful recent example.
Not only is there a risk that the discovery of false positives will encourage large, necessarily costly, and time-consuming phase 3 trials, but also a demand from patients for expanded access programs (EAPs) based on the promise of new and hypothetical therapies. Moreover, the appropriation of significant funds by political personnel for such purposes, motivated by “advocacy” by patient organizations and manipulation of opinion via social media, can divert valuable research funds from more promising therapeutic candidates and clinical trials. There is also a risk that the ALS community will be given false hopes about the potential clinical benefits of these investigational compounds based on minimal data.
Although "post-hoc" analysis of phase 2 trial data can sometimes lead to the generation of new hypotheses that could be tested in a future study, a distinction must be made between phase 3 trials that are based on a hypothesis that was tested in phase 2 and a hypothesis that was generated on the basis of phase 2 data and is, therefore, more fragile because it did not benefit from preclinical testing.
The risks associated with predicting phase 3 trials on the results of post-hoc exploratory analyses of phase 2 data are well illustrated by the experience with dexpramipexole, reldesemtiv, and NurOwn. Overreliance on clinical endpoints, short duration of placebo-controlled follow-up, and multiplicity, combined with overly optimistic reporting, have jeopardized the entry criteria and predictive value of phase 2 results for phase 3 outcomes. This problem was also clearly illustrated by the major negative phase 3 results for edaravone, AMX0035, and tauroursodeoxycholic acid. Each of these studies was preceded by positive clinical results in small, short trials, which were the primary drivers of the decision to proceed to phase 3.
Basing decisions on biological and mechanistic considerations could significantly reduce the risks of initiating Phase 3. While there is a risk that go-ahead criteria based on biological markers, such as NfL or other mechanistic markers, may not guarantee translation to clinically meaningful Phase 3 results, this approach could help to select better which drugs to test.
Premature Approval of Drugs
Regulatory approval of drugs based on Phase 2 data carries risks and benefits. It is certainly permissible for a company or group to advocate for early access to a drug with a favorable safety and tolerability profile while a confirmatory Phase 3 trial is underway, but approval of a drug based on limited evidence should not be confused with proof of efficacy.
My conclusion One development that seems likely to the author of this post is that future ALS clinical trials will address a smaller (and more homogeneous) population. This will ultimately make it even more difficult to conduct trials when the population with ALS available for trials is already very small.