About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Biotech Boom, Biotech Bust? | Main | AstraZeneca's Cambridge Move »

April 8, 2014

A Call For Better Mouse Studies

Email This Entry

Posted by Derek

Here's an article by Steve Perrin, at the ALS Therapy Development Institute, and you can tell that he's a pretty frustrated guy. With good reason.
That chart shows why. Those are attempted replicates of putative ALS drugs, and you can see that there's a bit of a discrepancy here and there. One problem is poorly run mouse studies, and the TDI has been trying to do something about that:

After nearly a decade of validation work, the ALS TDI introduced guidelines that should reduce the number of false positives in preclinical studies and so prevent unwarranted clinical trials. The recommendations, which pertain to other diseases too, include: rigorously assessing animals' physical and biochemical traits in terms of human disease; characterizing when disease symptoms and death occur and being alert to unexpected variation; and creating a mathematical model to aid experimental design, including how many mice must be included in a study. It is astonishing how often such straightforward steps are overlooked. It is hard to find a publication, for example, in which a preclinical animal study is backed by statistical models to minimize experimental noise.

All true, and we'd be a lot better off if such recommendations were followed more often. Crappy animal data is far worse than no animal data at all. But the other part of the problem is that the mouse models of ALS aren't very good:

. . .Mouse models expressing a mutant form of the RNA binding protein TDP43 show hallmark features of ALS: loss of motor neurons, protein aggregation and progressive muscle atrophy.

But further study of these mice revealed key differences. In patients (and in established mouse models), paralysis progresses over time. However, we did not observe this progression in TDP43-mutant mice. Measurements of gait and grip strength showed that their muscle deficits were in fact mild, and post-mortem examination found that the animals died not of progressive muscle atrophy, but of acute bowel obstruction caused by deterioration of smooth muscles in the gut. Although the existing TDP43-mutant mice may be useful for studying drugs' effects on certain disease mechanisms, a drug's ability to extend survival would most probably be irrelevant to people.

A big problem is that the recent emphasis on translational research in academia is going to land many labs right into these problems. As the rest of that Nature article shows, the ways for a mouse study to go wrong are many, various, and subtle. If you don't pay very close attention, and have people who know what to pay attention to, you could be wasting time, money, and animals to generate data that will go on to waste still more of all three. I'd strongly urge anyone doing rodent studies, and especially labs that haven't done or commissioned very many of them before, to read up on these issues in detail. It slows things down, true, and it costs money. But there are worse things.

Comments (19) + TrackBacks (0) | Category: Animal Testing | The Central Nervous System


1. annon too on April 8, 2014 10:46 AM writes...

Hey, it generates manuscripts.

Permalink to Comment

2. Kelvin Stott on April 8, 2014 10:51 AM writes...

Very concerning indeed, but on a positive note, this is a great example of how an institution can lay down the foundation of how to test drugs in animal models the right way, which others can adopt. First get the animal models right, then hopefully the drugs will follow, while avoiding so much wasted hope and resources on false positives.

Permalink to Comment

3. Anonymous on April 8, 2014 11:45 AM writes...

I've worked in pharmacology and in vivo modeling for years and I find this paper less than compelling. Some animal models are excellent predictors of clinical outcomes (e.g., the OVX rat for osteoporosis, the db/db mouse for diabetes). Apparently TDP43-mutant mice are a lousy predictor for ALS. Does this mean the studies were poorly conducted? I don't know - but the author doesn't provide particularly convincing evidence. Even the included graph (without error bars!) doesn't really address this issue. It just suggests the author couldn't replicate a few selected studies. How were the studies selected? Are they representative of all published studies or did he cherry-pick the publications with the biggest response? Ultimately I'm sure that ALS models are not good, but reducing the noise or studying tissue distribution of compounds isn't going to turn a bad model into a good one.

Permalink to Comment

4. Robb on April 8, 2014 11:46 AM writes...

A graph with no error bars in an article about scientific rigour. Sigh.

I was at a talk last spring where the presenter asked the audience (mostly basic immunologists) how they chose the number of animals to use for a study. Do a power calculation? (No hands) Ten animals? (most of the audience).

That might be okay for exploratory basic science studies, but any kind of translation should require replication with at least some of the rigor of a human study.

Permalink to Comment

5. MoMo on April 8, 2014 12:14 PM writes...

Many of the compounds, if not all, have already been through ALS human trials so what is the point of this study?

If you think neurodegeneration is due to one singlular cause or target (SOD1 anyone?) you are either naive or a biologist.

Permalink to Comment

6. luysii on April 8, 2014 12:57 PM writes...

The situation in stroke is even worse, and probably results, in part, from similar problems with animal models.

Almost 25 years ago, some 25 different compounds of 'proven efficacy' in treating focal and global brain ischemia in animal models had failed in human trials [ Stroke vol. 21 pp. 1 - 3 '90 ]. I stopped reading the experimental stroke literature in '95 for this reason. The last time I looked (at least 10 years ago) the number of failed human trials of treatments that had worked in animal stroke was up to 65.

I'm also far from convinced that TPA offers anything for stroke.

Permalink to Comment

7. exGlaxoid on April 8, 2014 2:59 PM writes...

I learned years ago that if you use a small sample size, no or poor controls, and noisy data source, the odds of getting a "positive" finding were at least 50%, much like the odds of picking red or black in roulette. So if you are trying to publish or sell your biotechnology company, those are the keys to success.

Permalink to Comment

8. Anton on April 8, 2014 6:34 PM writes...

I have a lot to say about this topic since it is my area of expertise. But why bother since Derek seems to delete all my comments. Censorship is bad DL

Permalink to Comment

9. Derek Lowe on April 8, 2014 7:10 PM writes...

#8 Anton - I delete the ones with gratuitous ad hominem comments (or should I say ad feminem). If your comments deal with your scientific expertise, they're always welcome.

Permalink to Comment

10. Kip Guy on April 8, 2014 7:31 PM writes...

Wowsa I thought we had it bad in oncology modeling. Underlying issues of power calculations, adherence to blinding strategies, careful attention to PK/PD correlations are all very real. Also a real problem in the primary driver for academia is the first paper on the model, not a careful study of predictive power. And then there's the whole curing mice thing. Still would be useful to focus this debate on how academia could realistically do a better job in developing and vetting models.

Permalink to Comment

11. Max on April 8, 2014 11:05 PM writes...

I agree with the anon comment #3 in that some mouse models are just plain bad, while others are decent. I've never worked with ALS, but I've worked with two neurodegenerative models along with other human disease mouse models.

As for the linked paper however I do agree with their suggestions, but having done mouse studies in four different med school/hospital settings, I would add a few confounding factors that would still make things not so straight forward even if those suggestions were all implemented:

1)Not all animal facilities are the same: some have different cage sizes, and as such the number of animals in a cage can vary. The bedding type and food access (distance from the bottom of the cage) can also be different. Differences in bedding, the number cage mates and what not may also lead to differences in mouse body temp. Some facilities have cages connected to air supply vents while others do not. Noise levels (construction/cleaning going on/people going in and out of rooms and so on) can be different. Despite attempts at control of room temp and light cycle, these can vary (doors opening to hallways, humidity in the building and so on). Some facilities like to have stimulation for the mice in the cages like the plastic igloos while other do not use them.

2) Survival studies are extremely difficult to run in every facility I've worked in (I don't know if this is different in industry animal facilities). Try getting IACUC approval with death as an end point; it is almost never going to happen. Deciding when to sacrifice a mouse can be difficult especially if there are multiple animal facility techs that also check on the mice, as all of them may have their own opinion on when a mouse is too sick and needs to be euthanized whether you agree or not. For models with relatively short lifespan this can be a big issue in maintaining consistency in survival assessment.

3) Drug administration can really change results. Even if you give the same concentration of drug, how the drug is suspended (vehicle type) and the means and timing by which it is given can all generate differences in results.

Permalink to Comment

12. SAR screener on April 9, 2014 7:07 AM writes...

@9 In which case, you might want to tidy up the comments in the recent GSK blog. There's a nice ad feminem post in the comments.

Permalink to Comment

13. Red Fiona on April 9, 2014 9:49 AM writes...

@11 - Some foolish questions since the nearest I get to animal work is E. coli, but are there not standard mice per cage or per square meter of space rules or something like that?

Permalink to Comment

14. Anonymous on April 9, 2014 10:35 AM writes...

@13 - There are some general guidelines that most IACUC follow, but there is a great deal of variability as @11 pointed out. And this is just a short list, many more could be added.

The general tone for many of the comments on this and other websites has been - 'wow, what shoddy research!' I don't see the evidence to support this belief. Looking at the graph couldn't one also conclude that ALS TDI did poor quality work? Perhaps the prior studies were thorough but the mice just don't respond the same as humans.

In the absence of a thorough statistical comparison of ALS TDI results with previous data it's impossible to conclude. Which is somewhat ironic given that topic of the article.

Permalink to Comment

15. Anonymous on April 9, 2014 12:10 PM writes...

I think the last paragraph of the article hits the mark.

"Public and private agencies should fund characterization studies as a specific project... This is unglamorous work that will never directly lead to a breakthrough or therapy, and is hard to mesh with the aims of a typical grant proposal or graduate student training programme."

To be able to perform better studies with well characterized animal models, someone will need to support the effort to develop and characterize such models. Congress seems to be doing an excellent job dropping the ball on this, so what recourse is left? Clearly I'm not an industry person, so I'm curious to know how much development and validation of animal models happens in the private sector.

Permalink to Comment

16. Immunoldoc on April 9, 2014 2:13 PM writes...

@3, I think you missed the point. The animal model, when done rigorously, perfectly predicted the clinical failures. This is not an article about poor mouse models of human disease per se, but rather they way they are used. Other readers who are commenting on lack of stats, I note that this is an opinion piece, not a peer-reviewed scietific report...

Permalink to Comment

17. Anonymous on April 9, 2014 6:54 PM writes...

@16, I think you missed @3's point. The author has essentially disparaged published studies claiming they didn't perform the experiment as rigorously. Very possibly true, but that wasn't really demonstrated.

Permalink to Comment

18. Max on April 9, 2014 10:32 PM writes...

@13 There are guidelines that institutional IACUCs will follow, which basically say you can have so many mice in a certain size cage. One institution I was at allowed a maximum of 3 mice per cage and another 5, as they used different cage types. These maximum numbers also assume the mice are all the same sex, as if there are mixed sexes you have to have less mice in the cage, which gets further complicated if there are pups being born. Additionally if male mice aren't litter mates they often can't be housed together as they will fight other male cagemates, whereas this isn't often a problem with housing non-littermate females. If cagemates constantly fight, or groom others they may get move to a separate cage, which could occur in the middle of an experiment. As researchers typically get charged per cage, per day, housing cost can play a role experimental design.

Permalink to Comment

19. Andrew on April 10, 2014 1:56 AM writes...

To the posters wondering about the lack of error bars, I noticed this too but I wonder, isn't "percent survival" pretty absolute? What would error bars show in this case?

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

The Last Post
The GSK Layoffs Continue, By Proxy
The Move is Nigh
Another Alzheimer's IPO
Cutbacks at C&E News
Sanofi Pays to Get Back Into Oncology
An Irresponsible Statement About Curing Cancer
Oliver Sacks on Turning Back to Chemistry