Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Too Big to Discover Anything | Main | Things I Won't Work With: Frisky Perchlorates »

May 29, 2006

Ask Not

Email This Entry

Posted by Derek

I was struck by a point that came up in the comments to the last post, about how since discovery organizations are going to have a certain percentage of failures, why not use that as a measurement of whether or not they're doing their job? Perhaps there should be a "failure quota" - if too many things have worked, perhaps it's because you're playing things too safe.

It's an intriguing idea, but I can see a few potential problems. For one thing, you'd need to be able to distinguish between playing it too safe and being really good (or really lucky). For another, there are quite a few organizations that are spending all their time trying to play it as safe as possible. If your research budget is running a bit lean because you don't have that many good products out there, then you may not feel like taking many extra risks. In that situation, the whole phrase "too many things have worked" just doesn't even parse.

It would be useful, though, for drug discovery organizations of any type to be a bit more realistic about how many of their efforts are going to fail. I mean, everyone knows the statistics, but everyone pretends that it's not going to be their own project that goes down. This is wishful thinking. Clearly, most of the time it is going to be your own project, because most project don't make it.

This isn't a license to give up. We should still do whatever we can think of to keep it from happening to our projects. But we shouldn't be amazed when our best efforts fail.

Comments (9) + TrackBacks (0) | Category: Who Discovers and Why


COMMENTS

1. Robin Goodfellow on May 30, 2006 3:14 AM writes...

This is interesting. I've been thinking recently about how our collective mindset is still stuck in the pre-industrial and pre-scientific past. Sometimes I really wonder if society and science will mesh cleanly or if it'll get all screwed up and there will be another dark ages.

Anyway, on a smaller horizon, I think this is a pretty classic problem in cutting edge development across industries. Aerospace and software have similar problems. You don't always know if something CAN be done, let alone how expensive it will be to produce, or how long it will take to develop. In software development these days one of the big catch phrases is "setting expectations". There has been a long history of unwarranted optimism on projects and an inability to meet projected delivery dates. Partly, of course, because sometimes the people that set schedules are not the same people that have to deliver on them, and everyone finds it hard to say "no". It's good to be optimistic, but it's bad to make concrete plans based only on fantasies.

It would be nice if people could become more mature and realistic (meaning: scientific) across the board. Of course, the real problem is, as you point out, how do you create a system which aligns incentives with the results you want without any of the easy traps (like researchers failing on purpose because it's easier than trying)?

Permalink to Comment

2. Jeff Bonwick on May 30, 2006 5:56 AM writes...

Oh, we're most definitely stuck in the pre-scientific past. We like to think that we're rational beings; but if that were true, it would be easy to extract square roots and hard to get angry. No, we're emotional beasts with just enough extra grey matter to see another possibility.

This is why science is such an immense achievement: because each advance is made not only against the odds, but against our nature.

Permalink to Comment

3. Paul Dietz on May 30, 2006 7:43 AM writes...

DOD's Defense Advanced Research Projects Agency (DARPA, formerly ARPA) has an expectation that only 1% of the projects it funds will lead to fielded systems -- but this 1% is more than enough to justify the entire effort. The Internet came from ARPA funded research.

The flip side of this willingness to fund highly speculative projects is a willingness to quickly kill them when they don't pan out. Researchers in this environment have to be able and willing to jump into new areas.

Permalink to Comment

4. FinanceMan on May 30, 2006 10:45 AM writes...

Why has Warren Buffett never invested in technology and drug companies? First, he doesn't have the technical background to assess true risk/reward profies. Second, the nature of both businesses is fraught with long-term risk. Both industries develop fantastic products, but both have certain life-spans. Coca-Cola is a brand that people have puchased over 100 years with little cap-ex required. Patents however continually force drug companies to find a new cure. Consequently they have to reinvest heavily in R&D. If they fail, massives amounts of investor capital can be lost. This risk should require a lower P/E multiple than widely recognized. While I believe in drug research and the above discussion, investors have to be cautious that management destroys previously earned wealth. A difficult task for the best of managment teams. Just remember, I have yet to see but a handful companies admit that they are mature and be satisfied being cash cows.

Permalink to Comment

5. TWAndrews on May 30, 2006 1:11 PM writes...

Either agroup has sufficient success, in which case the failures are going to be seen as just a hurdle along the way, or it doesn't, and I don't think that pointing to a good failure rate is going to do much to impress.

It might be worthwhile to assess the cost/benefit analyses for all the projects that a particular manager or set of managers has seen, to ensure that enough innovative, high risk/reward projects make it into the mix with the lower risk/reward projects.

Permalink to Comment

6. Jim on May 30, 2006 1:41 PM writes...

Derek, I understand what you mean about keeping our projects from being killed, but I think that attitude is part of the problem. I think at the discovery level more than at any other point in the drug development process, those people involved should be TRYING to kill their projects while putting forth their best scientific efforts. Of course, the problem with that is that upper management needs to see this as diligence rather than a lack of success, and that kind of goes against human nature.

Permalink to Comment

7. srp on May 30, 2006 3:29 PM writes...

Drug discovery is like oil exploration and movie making, in that the cause-and-effect mapping between what you do and what happens is not really known AND each full-fledged "experiment" is pretty expensive. (Contrast with direct mail or semiconductor manufacturing, where cause and effect is also badly understood but experiments are affordable.) In all these areas (oil, movies, drugs) there is a constant search for cheap early indicators that will improve hit rates on the full-fledged investments.

Unfortunately, despite advances, seismic modeling can't really take the uncertainty out of sinking wells. The movies are still in the realm of "nobody knows anything" (to quote screenwriter William Goldman), and all the focus groups and test audiences and "bankable" stars out there offer very little variance reduction (rigorous studies by Art deVany show this empirically). And it doesn't sound like any magic bullets for research productivity have been forthcoming for pharma.

Permalink to Comment

8. DLIB on May 30, 2006 4:56 PM writes...

The Semiconductor analogy only works so far. From my perspective ( I've got feet in both worlds ), there is one glaring difference. There is a point when the experiments in the Semiconductor industry become expensive enough that they cooperate to solve industry-wide problems ( differentiated mostly in design ). Mask sets in the industry are approaching 2 million dollars per stepper ( at the 45nm node )-- TSMC has 50 steppers. The new steppers run 40 million per. So when the industry decides to change from 8" to 12" Si. or when they go from Aluminum to Copper. There is a fair bit of cooperation ( IBM esentially solved the Copper problem and shared it with everyone else ). That year Semicon went from silver boothes to orange.
They have a group called SEMI which has a fab in texas that is a testbed for the member companies.

I do see analogous problems in the Pharmaceutical industry, but not the sharing and cooperation. Mega projects like the *omes are the closest thing but there are more problems ( toolsets )that have common utility that will remain unsolved.

DLIB

Permalink to Comment

9. Jonathan Quince on June 14, 2006 5:47 PM writes...

As a colleague of mine pointed out, assignment of brownie points for failure rates would create another interesting problem:

Anybody who sought funding for a project could advance the very persuasive argument that their proposal would have a 100% chance of either improving the organization’s success rate or its failure rate. In the pharma world, where researchers oftentimes must grasp at riskiest propositions, a 100% chance for every item in the pipeline seems like a darn good thing. For Big Corp industry research, a 100% chance is sure to impress the stockholders, too.

Of course, in cases where a partial success is possible—which it isn’t always—the argument becomes even better still: “What have we got to lose? It will improve our success rate, our failure rate, or both!�

I shall now proceed to buy lottery tickets, since I realize I have a 100.0% chance of winning on zero or more numbers I select.

(My colleague, as you may have guessed, is a mathematician. Our whimsical conversation about the odd virtues of failure rates quickly drifted into an orgy of Zorn’s Lemma jokes.)

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
One and Done
The Latest Protein-Protein Compounds
Professor Fukuyama's Solvent Peaks
Novartis Gets Out of RNAi
Total Synthesis in Flow
Sweet Reason Lands On Its Face
More on the Science Chemogenomic Signatures Paper
Biology Maybe Right, Chemistry Ridiculously Wrong