Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« A New NMR Probe Technology in the Making? | Main | A Sirtuin Activator Extends Lifespan in Normal Mice »

March 13, 2014

The Reasons for Failure at the FDA

Email This Entry

Posted by Derek

Here's a good retrospective in JAMA, from the FDA, about what's happened when the agency has rejected a new molecule entity (NME). The authors look over the data set from 2000-2012 to see what the most common reasons for trouble were, and what happened after that in each case.

Overall, 302 new molecules were submitted during those years. Half of them were rejected the first time through - a figure that does not jibe with most investors' ideas of the prospects for any given drug, for sure. But Tolstoy was right - approved drugs are all alike, but every rejected one is rejected in its own way. Looking over those, it appears that a bit under half of the rejects eventually did get approved (meaning that, ultimately, about 73% of all NMEs do eventually make it). But the median delay to approval, after that first rejection, was well over a year.

And that has to do with the reasons for rejection. They vary, and none of them are easily fixed:

Of the unsuccessful first-time applications, 24 (15.9%) included uncertainties related to dose selection, 20 (13.2%) choice of study end points that failed to adequately reflect a clinically meaningful effect, 20 (13.2%) inconsistent results when different end points were tested, 17 (11.3%) inconsistent results when different trials or study sites were compared, and 20 (13.2%) poor efficacy when compared with the standard of care.

But you'll note that while some of these are just nasty results in the clinic (like insufficient efficacy), some of them could also have presumably been mitigated a bit. You can see how crucial good trial design is when you note how many problems there are around endpoint selection and dosing. Now, it's also true that well-designed well-powered trials are invariably more expensive than most of the alternatives, and smaller companies are often running on fumes by the time they get to the end, but compare the pain of spending the time and money up front to the pain of a Complete Response Letter.

Taking a look at the ones that never did get approved shows a definite bias towards efficacy problems. Overall, safety concerns were similar between that group and delayed-approval group, but lack of efficacy was a problem with 76% of the never-approved drugs, almost double the rate of the delayed approvals. So the take-home is that you may be able to rescue a drug with safety concerns, given time and money, but there is likely to be no cure for poor efficacy.

And that's just what I think the effect of the FDA regulatory process should be. Once in a while, you hear calls for the agency to just let everything through as long as it's shown safety, to let the medical community and the marketplace sort it out. And although I'm a laissez-faire kind of guy, I think that would be a recipe for disaster. You'd have people selling repackaged barbecue sauce in gel caps as a cure for cancer. I'd rather have the agency focus on efficacy, and even let some drugs through that have some safety concerns, as long as they're known (as much as practical) up front and monitored thereafter. Just as long as they work against a disease.

Comments (30) + TrackBacks (0) | Category: Regulatory Affairs


COMMENTS

1. Pete on March 13, 2014 7:35 AM writes...

Any mention of lack of drug-likeness?

Permalink to Comment

2. post-doc on March 13, 2014 7:42 AM writes...

@Pete paywall paper, but what would be the link between drug likeness and efficacy (main problem) or the poor clinical trials if druglikeness was mentioned? How would make a correlation?

Permalink to Comment

3. Anonymous on March 13, 2014 7:51 AM writes...

I thought this was a very informative article. I read it when it was going around our company about 6 weeks ago (Reg Affairs sent it out to some folks). I'm a PK/Clin Pharm guy so I harped on the fact that the biggest reason for initial failure was dose-selection uncertainty (15.9%). The other thing I noticed from the paper that maybe hasn't been pointed out yet is that CMC deficiencies may be underreported in the paper; they only looked at CMC deficiencies if safety and efficacy looked good and those NDAs that got the boot for safety and/or efficacy may have also had unreported CMC problems.

Permalink to Comment

4. petros on March 13, 2014 8:07 AM writes...

It would be interesting to see the corollary of how many initially approved drugs have been subsequently pulled for safety reasons.

On the dosing issue-Pfizer's tofacitinib highlights some of the issues. The Phase III study in RA looked at 5 and 10 mg doses and Pfizer filed for approval of both doses. The FDA only gave approval for the lower dose. (In contrast the EMA has twice rejected approval).

Permalink to Comment

5. MTK on March 13, 2014 8:12 AM writes...

@3,

I guess you're technically right about CMC, but my view would be sort of a shrug. If the compound doesn't pass the clinical hurdles then why bother even worrying about CMC? If CMC was a problem and was fixed would it make the compound any more approvable?

As a taxpayer I'm glad to hear that the process wrt to CMC works as you described it.

Permalink to Comment

6. Pete on March 13, 2014 9:40 AM writes...

@2 post-doc, I can't see the paper either. I'm not a great fan of the drug-likeness concept which seems to be a very indirect way to ask questions like 'How predictive is logP of oral exposure?'. It would have been interesting if separate analyses has been done for intracellular and extracellular targets because we know a lot less about free intracellular drug concentrations.

Permalink to Comment

7. Slurpy on March 13, 2014 9:53 AM writes...

"You'd have people selling repackaged barbecue sauce in gel caps as a cure for cancer."

Clearly someone hasn't been to Whole Foods or the herbal supplement aisle in the local supermarket lately.

Permalink to Comment

8. CMCguy on March 13, 2014 10:26 AM writes...

I too do not have access yet the rejection categories in the extract above add up to just under 67% which leave another 33% that presumably is made up of multiple less frequent reasons that are rarer but wonder if any stand out as "inexperience based" issues? I agree good study design is critical, followed by execution and data collection/cleaning aspect and those often are were small biotech may be more naive even if just expense CROs. A hidden feature IMO is business/marketing input that might lead to certain end point misguidance as attempt to force to support specific targets rather than indications and designs that could be more scientifically justified.

As far as CMC as @5 MTK suggests these are largely a secondary consideration which if inadequate can only further be used to hurt poor clinical results but if clinical data is good CMC problems then probably might result is a hold/delay till resolved but do not recall any outright CMC only rejects (except maybe in generics).

Permalink to Comment

9. Anonymous on March 13, 2014 10:52 AM writes...

@7: And you have???

Permalink to Comment

10. annon too on March 13, 2014 11:26 AM writes...

Derek,
I have to disagree with your comment on approval with some safety issues, as long as they are known. The problem is that the safety issues are all to easy to "slip" down a slope, where then patients have issues whereby they did not expect, anticipate, or know how to respond. It's exactly the wrong approach to take in today's world.

Permalink to Comment

11. Miramon on March 13, 2014 12:31 PM writes...

I suppose the obvious question is why these NMEs were submitted with their various failing characteristics.

Surely the corps of people responsible for creating these submissions aren't incompetent idiots, or at least you'd think they'd learn from their mistakes. Is the mere fact of submission enough to satisfy investors? Is that the motivation? Or is the explanation inconsistent and whimsical behavior by the FDA?

Permalink to Comment

12. Am I Lloyd peptide on March 13, 2014 1:03 PM writes...

"You'd have people selling repackaged barbecue sauce in gel caps as a cure for cancer"

Nonsense. The only repackaged barbecue sauce in gel caps that I have seen being sold is as a cure for Alzheimer's disease. No way that stuff's going to cure cancer.

Permalink to Comment

13. watcher on March 13, 2014 1:15 PM writes...

GOOD barbecue sauce is too valuable for its TASTE to waste in a gel cap..........

Permalink to Comment

14. Brett on March 13, 2014 1:56 PM writes...

I don't think it would be "barbecue sauce in capsules", but it could be bad. More likely is that you'd see stuff hiding inside of dubious correlations between cancer recovery and taking it, to make it more difficult when either you or your survivors sue them over it.

Permalink to Comment

15. Anonymous on March 13, 2014 2:33 PM writes...

@5 MTK:

I pointed out the CMC thing not as something the FDA should be concerned with but as something CMC folks should be concerned with. They may be screwing up more often than the data in Table 5 would have them believe.

For those without access, Table 5 lists review failures as:

Efficacy deficiencies only: 31.8%
Safety and efficacy deficiencies: 27.2%
Safety deficiencies only: 25.8%
CMC alone: 11.3%
Labeling alone: 2.6%
CMC and labeling: 1.3%

According to this table CMC folks can pat themselves on the back because CMC is only a problem 12.6% of the time whereas safety problems occur 53.0% of the time and efficacy problems occur 59.0% of the time. But--as the authors note--CMC problems are underestimated based on the review priority process.

Permalink to Comment

16. Anonymous on March 13, 2014 2:38 PM writes...

@8 CMC Guy:

There's your answer for 'outright CMC only rejects': 11.3% (17 out of 151). Thirteen of them were subsequently approved after modifications and four of them were never approved (at least not in the time frame of the analysis).

Permalink to Comment

17. Lyle Langley on March 13, 2014 2:53 PM writes...

@1 Pete and @2 post-doc:

Not to be facetious here, but if a compound gets approved then it falls into the "drug-likeness" category, and if it does not, then it falls out of the "drug-likeness" category. C&EN ran a story a couple issues back with the 2013 approvals...how many on that list would one consider to have "drug-likeness"? Of course, they all do because they were approved, but just looking at the structures, there are some stinkers there that I'm sure will warrant some sort of review about how the LE, LLE, LLEP, ABCD, WXYZ - or whatever made up parameter was in vogue was not correct.

Permalink to Comment

18. Cellbio on March 13, 2014 4:05 PM writes...

When I look at how the categories are cut, I'd make efficacy a much larger slice of the pie. The text of the paper has this following the section above..."In these cases, efficacy was typically only seen in some studies or study sites and not in others or only in subpopulations that were not part of the original analytic plan." Post-hoc analysis slicing the population and hunting for signals of efficacy. Efficacy related categories, population too small, wrong population for intended use (=no proof of efficacy in intended population), lack of evidence for inferiority, and in some cases I am sure the question of proper dose, as this will not likely all be uncertainty from lack of testing but odd dose responses that most likely show study variation allowed one dose cohort to pop (not prior post of Derek's on La Jolla drug). In total, I would lump together these categories and top 70% related to efficacy.

Permalink to Comment

19. Cellbio on March 13, 2014 4:08 PM writes...

The above lump was for First-cycle failures. For never approved, the lump is more than 90%.

Permalink to Comment

20. Cellbio on March 13, 2014 4:31 PM writes...

Posted a little too quickly. These are all in efficacy categories by the authors, and deficiency counts exceed drugs so some have multiple deficiencies.

Permalink to Comment

21. CMCguy on March 13, 2014 5:32 PM writes...

#16 Anon thanks for the elaboration and some actual numbers. I am not that surprised at an initial 10-15% rate for having significant CMC problems (and might speculate for EU submission higher rate since can be greater CMC focus there it appears) however am enlightened that only 76% (13/17) were subsequently dealt with as would have expected close to 100%. Does this NME list include Biologics? For those type drugs often the process and control details is much more vital thus wonder if the 4 unapproved might fall in there. Of course with all the lay offs and outsourcing I also might suggest reflects solid CMC expertise is diminishing as older troops retire or go/sent elsewhere with lack of effective training of new generations.

Permalink to Comment

22. Marc on March 14, 2014 2:44 AM writes...

@CMC people. Hi, first time commenting. I've been reading for about 6 months now. I'm a pharmacy student planning to go into med chem once I pay off my debt... I would love to know what CMC stands for.

Permalink to Comment

23. google for Marc on March 14, 2014 3:56 AM writes...

Hey Marc,

I also don't know what CMC is, but a quick google of: "cmc drug problems" very quickly gets you "Chemistry, Manufacturing, and Controls (CMC)".
And from the same search you get a wikipedia article with a bit more description.
http://en.wikipedia.org/wiki/Drug_development

Permalink to Comment

24. Kath on March 14, 2014 3:59 AM writes...

CMC = chemistry, manufacturing & control, aka 'how we make the drug'. It can also be referred to as "quality".

Permalink to Comment

25. ex-FDA reviewer on March 14, 2014 6:53 AM writes...

Jives with what I saw when there. You need 3 things: safety, efficacy and quality and there is a risk benefit analysis that encompasses all. There is no risk acceptable if there is no benefit.

Permalink to Comment

26. Nony on March 14, 2014 8:46 AM writes...

I wonder if the poor experiment designs (dosing, enough clinical numbers) are more attempts to get drugs that don't do much through the system. So, not a simple trade off of expensive experiment. But actually trying to get marginal stuff through that would get exposed if better tested.

Permalink to Comment

27. johnnboy on March 14, 2014 9:01 AM writes...

@26: I would hazard that insufficient dosing issues happen where a drug has some known safety issues and the company is trying to get efficacy while minimizing toxicity. Choosing the right dose is a balancing act, especially when you're only at phase 2 and you don't have very solid efficacy or safety data yet. As far as insufficient numbers, I would think that's mainly a matter of money, or in the case of rare indications just low disease incidence.

Permalink to Comment

28. CMCguy on March 14, 2014 10:17 AM writes...

#24 Kath while I will grant you that quality does underpin majority of the CMC activities, and #25 xFDA summarizes a common simplified view on factors for drug approval, I am not sure who tags as just quality expect most MBA and certain MD types who can't seem to get their heads around what is done and required to develop a drug from bench discovery to pre-clinical/clinical studies to hopefully eventual marketing and commercial applications. Maybe I am just over sensitive to having to explain to business or management people why in development need to do certain things, often involving substantial costs, and delay/time compression only increased the risks of success. Could be QA & QC (Quality Assurance and Control) people involved might also tend to apply that condensed description however CMC entails groups of process chemists and engineers, formulators, pilot plant/manufacturing operators, analytical development and testing people plus a myriad of other support functions to enable the transitions along the path therefore even using CMC does not fully capture all requirements.

Permalink to Comment

29. Anonymous on March 14, 2014 10:52 AM writes...

@21 CMCguy: Regarding your question, "Does this NME list include Biologics?"

Their methods say they didn't include them. As you would expect, they also excluded generics, supplementary applications and nontherapeutic diagnostic applications (e.g. radiocontrast agents).

Permalink to Comment

30. CMCguy on March 14, 2014 2:14 PM writes...

#29 Anon thanks, again I assume, as would be interesting to learn what those particular 4 CMC issues are since likely have to be fairly egregious for a company not to fix then refile assuming the clinical data was decent and had the funding. Suspect will or could have come out in FDA presentations as example case reports or simply via regulatory grapevine because although Complete Response Letters are still confidential I believe such impactful info tends to leak out as hard lessons which people will ideally not repeat.

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
The Worst Seminar
Conference in Basel
Messed-Up Clinical Studies: A First-Hand Report
Pharma and Ebola
Lilly Steps In for AstraZeneca's Secretase Inhibitor
Update on Alnylam (And the Direction of Things to Come)
There Must Have Been Multiple Chances to Catch This
Weirdly, Tramadol Is Not a Natural Product After All