Precision Medicine May Be Health Care’s Next Big Thing, But We Don’t Yet Know How to Evaluate Precision Drugs

Finally, there’s a drug available to treat Duchenne muscular dystrophy, a rare, fatal genetic muscle-wasting disease that grips roughly 10,000 people, mostly boys, in the U.S., who are unlikely to live into their 30s and typically lose the ability to walk in adolescence.

On Sept. 19, the Food and Drug Administration approved Exondys 51 (eteplirsen) after reviewing a clinical trial and hearing words of support from 50 advocates on behalf of the drug, including families with children living with the disease, experts on the illness at UCLA, and even a few politicians, like Florida senator and former presidential candidate Marco Rubio.

Duchenne families and advocates were ecstatic at the news. Sarepta, the Cambridge, Massachusetts-based biotech firm that developed the drug, saw its stock more than double. There was just one minor issue: The science suggests the drug doesn’t work.

An advisory committee impaneled by FDA in April to weigh the clinical data wasn’t convinced and voted against Exondys 51’s approval. But that panel was overruled by Janet Woodcock, director of FDA’s Center for Drug Evaluation and Research. Woodcock greenlit Exondys 51 immediately—even though it will only benefit the 13 percent of Duchenne patients who have the specific genetic mutation the therapy targets—in part because there are no other options for DMD patients.

Her decision kicked off a skirmish over the role of FDA in the era of precision medicine: When treatments target minuscule subsets of people with specific genetic illnesses and traits—can the agency do its job?

An op-ed published Oct. 24 in the Journal of the American Medical Association, co-authored by Aaron Kesselheim, an associate professor of medicine at Harvard Medical School and a member of FDA’s Exondys 51 advisory committee, said the agency’s decision “provided a worrisome model for the next generation of molecularly targeted therapies.”

Kesselheim’s concern was that more and more precision medicine treatments might get approved because they show positive effects in the body—like increasing protein levels or shrinking tumors—without actually making any measureable difference in patients’ health.

Traditionally, FDA’s standard for evaluating drugs has been to ensure they’re safe and that they extend and/or improve patients’ lives. But since Congress signed off on so-called accelerated approval in 1992, instead of examining patients’ overall health, FDA looks at “surrogate” measures—physiological indications that a drug is working.

For example, in the case of a therapy for heart disease, the agency might require it lower a patient’s cholesterol levels. A diabetes treatment might be evaluated on its ability to regulate blood sugar. Whether or not the patient experiences any improvement in symptoms after taking the drug doesn’t actually matter.

Sarepta submitted a study of 12 boys, all of whom took the drug—meaning the trial had no control group. The data showed little evidence of measurable improvement in participants gaining or retaining the ability to walk over time. So to illustrate Exondys 51’s efficacy, Sarepta pointed to the fact that three years after the trial ended, all of the boys had a protein called dystrophin in their muscle. DMD patients can’t make dystrophin themselves—and the trial participants, according to Sarepta’s data, had about 50 percent of the normal amount.

But FDA found flaws in its methods—and a re-analysis dropped the mean amount of dystrophin in the cohort to less than 1 percent of normal.

“It’s not just a surrogate; it’s just a whiff of a surrogate,” says Vinay Prasad, assistant professor of medicine at the Oregon Health and Sciences University, who has studied the approval of cancer drugs based on surrogate measure outcomes. “The primary reviewers”—FDA’s advisory panel members—“don’t think there was a whiff there at all.”

In addition to studying the clinical data, the panel heard emotional pleas from families whose children are living with the disease. Ultimately, seven of the panel’s 13 members found the change in dystrophin levels likely didn’t translate to patient improvement. Woodcock disagreed, writing in a July 14 memo the protein’s presence was “reasonably likely to predict clinical benefit.”

Woodcock’s decision was appealed by Ellis Unger, FDA’s director of the Office of Drug Evaluation, who sided with the advisory committee. But Robert Califf, who took over as head of FDA early this year, backed Woodcock. The approval was granted with a request that Sarepta provide evidence its drug works in a randomized clinical trial by the spring of 2021.

Shortly after it received the good news from FDA, Sarepta tagged Exondys 51 with a cost of $300,000 per year. On a call this past Thursday morning with investors, Sarepta CEO Edward Kaye reported the first patient was already receiving an infusion of the approved treatment.

“The FDA is supposed to operate on the black-and-white hard science of the data,” says David Gortler, an ex-FDA senior medical officer who is now a regulatory expert with FormerFDA.com. “They’re succumbing to public pressure and trying to make everyone happy. And sometimes, the answer in science is no.”

Another factor that probably influenced Woodcock’s decision: Exondys 51 probably won’t make any patients sicker. In several places in FDA’s 125-plus page evaluation of the compound, the drug is deemed to be safe.

That, says Darius Lakdawalla, a regulatory innovation expert at USC’s Schaeffer Center for Health Policy and Economics, allowed the agency to take a risk on a drug with uncertain effectiveness in an area where there was no treatment alternative. But, Lakdawalla, says, “I am very skeptical of regulatory agencies’ ability to weigh risks in the way patients’ would want them weighed.”

The Exondys 51 case follows a pattern set by FDA’s approval of cancer drugs over the past decade or so, many of which also proved their worth through surrogate measures. Oncology is at the forefront of the precision medicine movement, the theory being that cancerous tumors with certain genetics might respond more favorably to a certain type of therapy. But thanks to the surrogate system, many therapies that have been shown to simply decrease the size of tumors—without extending a patient’s life—have been approved.

A 2014 investigative report by the Milwaukee Journal Sentinel and MedPage Today found of 26 cancer drugs FDA approved between 2004 and 2011 based on surrogate effects, 22 of them still had not been proven to extend life when the report came out. Often, companies that get approval for drugs through surrogates must later provide studies that show their treatments improve or extend patients’ lives—if they don’t, FDA can fine them.

But as of late 2014, the agency had never levied a single fine. (FDA could not provide information on whether fines were levied since late 2014 by press time.) The Government Accountability Office released a report late last year finding the agency was behind on reviewing more than half of the 1,400 studies it requested from 2008 to 2013 for drugs already on the market. In a response to the report, FDA said it had identified and addressed issues causing the backlog, and by the end of the 2013/2014 fiscal year, only 13 percent of postmarket-study reviews were still delayed according to FDA.

At a summit for the National Organization for Rare Diseases a month after Exondys 51 was approved, FDA Office of New Drugs Director John Jenkins noted the path to approval taken by Sarepta was “not a good model for other development programs,” suggesting the case was a one-off. He also made clear that randomized, controlled clinical trials are still the “fastest way” to determine a drug’s effectiveness.

“In this era of precision medicine, the agency expects that the issues associated with developing therapies for rare diseases will be magnified,” an FDA spokesperson wrote in an email to Quartz. “We firmly believe that patients with rare diseases deserve therapies evaluated with the same rigor as more common diseases.”

But not everyone agrees that amount of rigor is always applied, especially in cases like Exondys 51. That level of scrutiny isn’t always going to be possible when you’re dealing with very rare diseases or when you’re developing targeted therapies for small subsets of people, says Art Caplan, a bioethicist at NYU’s Langone Medical Center.

FDA’s approval system was designed to generate enough data to make people feel confident drugs on the market are safe and effective. The precision medicine era means the agency will have to live with significantly more uncertainty—and that means elements like surrogate measures and the strength of a patient community are probably going to be a bigger part of the equation going forward.

“The good news is, we have precision medicine,” Caplan says. “The bad news is, we have a regulatory system that is designed for imprecise medicine.”