Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« A New HIV Therapy. Yawn? | Main | Time To Conduct Some Business »

April 26, 2007

Less Than Zero

Email This Entry

Posted by Derek

When I wrote about lousy animal models of disease a few days ago, there was a general principle at the back of my mind. (There generally is - my wife, over the years, has become accustomed to the sudden dolly-back panorama shots that appear unannounced in my conversation). It was: that a bad model system is much, much worse than no model system at all.

I've been convinced of that for a long time. When you have no model for what you're doing, you're forced to realize that you have no clear idea of what's going on. That's uncomfortable, to be sure, but you at least realize the situation. But when you have a poor model, the temptation to believe in it, at least partially, is hard to resist. Even if it's giving you the right answers at a rate worse than chance, you can still take (irrational) comfort in knowing that at least you're not flying blind - even as you do worse than the people who are.

There are many reasons to hold on to an underperforming model. Sometimes pride is the problem. I've seen groups that stuck with assays just because they'd invented them, even though the method was slowly wasting everyone's time. Never underestimate cluelessness, either. People will use worthless techniques for quite a while if they're not in the habit of checking to see if they're any good. But the biggest reason that useless procedures hang around, I'm convinced, is fear.

Fear, that is, of being left out in the middle of the field with no models, no insights, and no path forward at all. It's a bad feeling, rather scary, and rather difficult to explain to upper management if you're a project leader. Better, then, to hold on to the assays and models you have, to defend them even if you're not sure you trust them. With any luck, the project will end (although probably not happily) before the facts have to be faced. As Belloc advised children in other situations: "Always keep ahold of Nurse / For fear of finding something worse."

Comments (23) + TrackBacks (0) | Category: Animal Testing | Drug Assays | Drug Development


COMMENTS

1. Harold on April 26, 2007 9:01 PM writes...

I've been happily retired for almost a year now from the wonder drug factory, so I don't remember specifics, but another reason to retain a worthless model of efficacy or of toxicity was that the FDA expects to see certain assays in applications and it is often easier to do the work than to make the argument that the work is less than worthless and it shouldn't be required.

Permalink to Comment

2. milkshake on April 27, 2007 2:14 AM writes...

The lack of curiosity (about whether the used model has any predictive power) is rather typical of a pseudoscience.

I was making myself unpopular at the previous company - at the project meeting they were very seriously discussing how their candidate had to be above 30 min in human microsomes but their compound had only 16 min. They had these gorgeous Powerpoint slides summarizing (with bullets) that "we must improve the microsomal stability". I asked them why wouldn't they prefer to improve the rodent PK instead. They said the microsomes are like PK. I mentioned that I have seen plenty compounds with absolutely horrible PK but excelent microsomal stability, and vice versa. They said they had microsome assay set up and available every day, they were going to use it as a pre-screenbecause they cannot submit too many compounds for PK because there is a long waiting list.

And so it goes.

Permalink to Comment

3. TX Raven on April 27, 2007 4:13 AM writes...

Yet, there is the mother of all in vivo animal models (at least the CNS-ones), especially for those projects that don't have one that works: ex vivo receptor occupancy!
I have seen very well educated people, able to understand very complex scientific issues, come up with the most "sophisticated" reasons to perpetuate this type of ex vivo assays. In my med-chem biased view, these are just glorified in vitro binding assays, bearing very little relevance to the "real" in vivo receptor occupancy that drives behavior. Well, at least in theory :-)
I am talking about those studies where you dose the animal with your compound and you sac it, remove the brain, homogenize the region of interest and run a binding assay.
Equating this in vitro determination with an in vivo receptor occupancy determination and using this numbers to draw conclusions is done all the time in a number of industrial research organizations.
Wonder why our new drugs are taking so long to show up?

Permalink to Comment

4. NJBiologist on April 27, 2007 6:20 AM writes...

Maybe it's just me, but I've had a really tough time convincing some of my colleagues that there is a difference between a model's resemblance to clinical disease and a model's predictive capability.

Permalink to Comment

5. Kay on April 27, 2007 7:15 AM writes...

Only the special few can broadcast doubt, especially when paychecks are involved. Perhaps this is one of the root causes for our current difficulties.

Permalink to Comment

6. LNT on April 27, 2007 8:31 AM writes...

Milkshake raises a concern that I've had with my own "Wonder Drug Factory". How useful is microsome stability data? Generally, we only carry compounds forward that have good microsome stability. Obviously, some compounds with excellent microsome stability have horrible PK due to solubility, permiability, excretion, non-microsomal metabolism, etc. However, is the reverse also true? How many people have encountered a compound with poor microsomal stability and genuinely good PK? Since we never carry unstable compounds forward, I really can't answer that question based on experience at my company.

Permalink to Comment

7. MolecModeler on April 27, 2007 9:18 AM writes...

Excellent point Derek. This is something I learned from my manager and through experiences: sometimes something is worse than nothing. Many times some higher up is invested in a certain project/assay and so it has to be used, this is Kay's point in #5.

To give you a comp chem example, this often happens when using homology models of family A GPCRs. They're all based on bovine rhodopsin, no one really has any idea of their quality, and their predictive power is not statistically different from 0 imho. Yet they are used by many people who think they're "real".

Permalink to Comment

8. Hap on April 27, 2007 10:39 AM writes...

"The greatest obstacle to discovery is not ignorance - it is the illusion of knowledge." Daniel J. Boorstin

In this case, it's perhaps worse than an illusion of knowledge, because we know that some models are illusions, and at odds with reality, yet we keep them solely because the pretense that we actually know something useful comforts us. We would rather feel that we know something (even if at heart we know that we don't really) than know that we do not. The knowledge that we could be wrong doesn't help us if we ignore it.

Permalink to Comment

9. Cyan on April 27, 2007 10:43 AM writes...

This is a point that goes far beyond just animal models. Will Rogers put it, "It isn't what we don't know that gives us trouble, it's what we know that ain't so. "

Permalink to Comment

10. russ on April 27, 2007 11:41 AM writes...

"The measure of a man's greatness is how long his erroneous ideas retard progress in his field." Frank Boardwell

Permalink to Comment

11. Clark Kent on April 27, 2007 12:58 PM writes...

I have many examples on my current project of compounds which show low in vivo clearance (in multiple species) and high intrinsic clearance by S9 and microsomes. We were in the fortunate position of getting the in vivo data quicker than the in ivtro data, so we dont even send the compounds through the in vitro screen anymore because it was completely useless.

Permalink to Comment

12. Demosthenes By Day on April 27, 2007 1:26 PM writes...

The problem of whether liver microsomes predict real PK is the same problem that crops up with the cell permeability assays. CaCO2 and PAMPA are both tragically wrong in a number of cases.I've been on two projects where the clinical candidate failed both permeability assays but in at least one case it sure seems to be hitting the target. But these assays are part of a company's new automated high throughput way of anlyzing for pharmaceutical properties. What it really turns into is an almost random way of talking yourself out of making a compound because its quick not scientifically sound.

Permalink to Comment

13. SMIP on April 27, 2007 2:24 PM writes...

Good post. I would recommend the book "The Black Swan" by Nassim Taleb. You will appreciate his writing.
Good luck.

Permalink to Comment

14. Wavefunction on April 27, 2007 5:24 PM writes...

However (depending on the situation) sometimes even a bad model is better than no model because it can teach you a lot (about what not to do)

Permalink to Comment

15. Hap on April 27, 2007 6:13 PM writes...

I wouldn't figure that a bad model is better than no model unless you know what its weaknesses are. With simple models such as the ideal gas law and Bohr's model of the atom, you can use what you know and what doesn't work to make estimates for the things you don't know (like accounting for the volume of molecules and their attraction/repulsion), and even if you can't do so, you can at least know that the limits of the model. I don't know enough biology, but I figured that the biological models discussed aren't well determined enough to know their limitations, and so you don't really know anything other than what the model tells you, including what you actually want to know.

Permalink to Comment

16. srp on April 27, 2007 6:31 PM writes...

My history of science professor in college described the phenomenon Derek describes in the field of anatomy. Apparently the Roman physician Galen's writings on human anatomy, which was taken almost as sacred truth from the time of its rediscovery, was based on dissecting gibbons.

People knew that, but they kept assuming that Galen was right until Vesalius and company came along and said "Wait a minute--that's all based on gibbons. Maybe we should start dissecting human corpses." Obviously, dissecting human corpses raised a number of thorny cultural and religious issues, but it did get around the bad model problem.

Permalink to Comment

17. Chrispy on April 27, 2007 6:53 PM writes...


I hope all of you synthetic guys are familiar with the work of Alexander Shulgin. He tests all his stuff directly on humans -- himself first.

http://en.wikipedia.org/wiki/Alexander_Shulgin

Phenethylamines I have Known and Loved (PIHKAL) is an absolute classic (haven't read TiHKAL) for those in drug discovery.

He is very careful and tries very small doses initially. But I don't know anyone else doing drug discovery this way.

I'm surprised (and glad) that it is legal!

Permalink to Comment

18. BCP on April 27, 2007 10:09 PM writes...

Good call milkshake -- I've also seen plenty of examples where either in vivo or in vitro PK over predicated the other. Bottom line, it's very difficult to generalize - if you understand your SAR within a series there's at least some hope.

Nice Vonnegut reference by the way.

Permalink to Comment

19. DLIB on April 28, 2007 12:36 AM writes...

Hi Derek,

Off the subject, but I was just listening to the pfizer shareholder meeting and I think this blog gets a nod at about 9min 30 sec into it. Cancer patient.

Permalink to Comment

20. milkshake on April 28, 2007 12:38 AM writes...

It's like with the man who was looking for his keys under a lamp-post. When asked if he was sure that he had lost his keys right there he replied: "Not necessarily but at least I could see here"

Permalink to Comment

21. oldbio on April 29, 2007 8:42 PM writes...

There is no single assay or model which tells you everything you need to know about a compound. Which in vivo model is the most important, the one that predicts some ADME properties or the one that proves the mechanism of drug action or the one that models a disease state? Judging and prioritizing compounds based on a large amount of data, some of which not meaningful or accurate, is where the art comes into the science. If drug discovery was solely a rational exercise, all our pipelines would be full.

"To know what you do not know is the best. To pretend to know what you do not is a disease" - Lao Tzu

Permalink to Comment

22. Mark M on April 30, 2007 10:05 AM writes...

Chrispy:

That is how all respectable neuro-pharmacologists did their work back in the day.

I am not sure how legal that is today; I think Sasha had some special "license" to do his studies.

The point is that there is no substitute for whole animal work--problem is that is very expensive and not that amenable to high throughput.

-Mark

Permalink to Comment

23. Psychedelic Biologist on May 1, 2007 2:30 PM writes...

Shulgin does/did have some sort of licensure from the USA DEA for synthesis/possession. I'm not sure whether that entitled him to legally ingest likely-psychedelic analogs of schedule-one compounds like psilocybin or mescaline in the USA. The USA Analog Act (or something like that) is probably relevant. I haven't read it.

Personally I am very glad to see the new trials examining psychedelics such as MDMA for post-traumatic stress disorder. Many people may not know it, but prior to the prohibition of psychedelics, there was extensive experimentation by professional psychologists and psychiatrists interested in the medical use of psychedelics (not all of them necessarily as variably-flakey as Timothy Leary). Such research has been almost frozen until rather recently.

(Regardless of medical utility or non-utility, the religious, as opposed to recreational, value of psychedelics is demonstrated by their religious use in numerous long-lived cultures worldwide.)

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
A Last Summer Day Off
The Early FDA
Drug Repurposing
The Smallest Drugs
Life Is Too Short For Some Journal Feeds
A New Look at Phenotypic Screening
Small Molecules - Really, Really Small
InterMune Bought