About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Vial Thirty-Three: Warp Drive | Main | Sell! (It's Not Just Me) »

November 17, 2006

Ah, the Heck With the Error Bars

Email This Entry

Posted by Derek

OK, here's a stumper: are there any anti-inflammatory medications that don't have cardiovascular side effects? Aspirin's GI bleeding effects have been known for decades, various NSAIDs have had warnings turn up over the years, the COX-2 drugs are somewhere over there in that huge cloud of legal and statistical dust, and now a study says that one of the last left standing, naproxen, may have cardiac effects as well.

Or does it? This is an attempt to get some useful data out of a large Celebrex trial, looking to see if it had protective effects against Alzheimer's disease, and the whole thing was stopped early when all the COX-2 cardiovascular risks became an issue. As this article makes clear, Steve Nissen isn't convinced, and he's not a person who keeps his worries about drug safety all bottled up inside. His point is that the trial's statistical validity was ruined by the early halt, and that larger epidemiological studies don't back up its conclusion. I should note that he's now running a massive trial addressing this issue as well.

Contrast that with this quote from one of the new paper's authors:

"Particularly for safety data, 'truth' may come in small doses. We firmly believe that results from trials should be published regardless of the direction, magnitude, or statistical significance of the observed results," said Barbara Martin of the John Hopkins University School of Public Health, who worked on the study.

Let me tell you, that kind of thing makes me very nervous. Regardless of the magnitude or statistical significance? That's not a way to arrive at the truth. That's a tornado passing over a pig farm

Comments (14) + TrackBacks (0) | Category: Cardiovascular Disease | Toxicology


1. Chris on November 17, 2006 10:20 AM writes...

I get worried when one has to decide what gets published and what gets buried. Should the first study that has significance get published, even though the other 9 that were attempted (and unpublished) were not even close to significant? Science should be about sharing information and including dead ends (that are peer reviewed and meet quality standards) is not a bad thing. Yes, it would produce a lot of bulk in journals, but IRBs should put pressure on researchers to see that results will be published so that in the end the trust that the research subjects gave is supported.

Permalink to Comment

2. KonradK on November 17, 2006 10:35 AM writes...

are there any anti-inflammatory medications that don't have cardiovascular side effects?

Glucocorticoids, but what I think you meant was non-steroidal anti-inflammatory drugs (NSAIDS), in which case I do not know of any off the top of my head which are free of potential CV side effects.

Permalink to Comment

3. Konradk on November 17, 2006 10:40 AM writes...

Opps, I read your blog too fast and didn't notice that you wrote:

various NSAIDs have had warnings turn up over the years

Sorry about that.

Permalink to Comment

4. david on November 17, 2006 12:07 PM writes...

Unfortunately, the pharmaceutical industry's record of disclosure isn't particularly palatable. If it were, one might be able to leave some discretion to the industry. In the absence of that, societal demand for publication of everything has resulted in the full disclosure approach suggested by the comment above. Oh, and I do work in the industry.

Permalink to Comment

5. John Johnson on November 17, 2006 12:09 PM writes...

Quite frankly, that quote doesn't bother me as much. However, I can see where the public, or a critic, might take a small study and overinflate it's importance (or distort the data) to support a conclusion (and advocates for the therapy under study as well).

The fact is (and maybe most of you are the choir on this) all these studies have to be studied in context of other evidence on a therapy. Putting those studies together is not always an easy task, even if you put it together through some standard strategy such as meta-analysis. I'd rather see a small trial and be able to discard it from the evidence pile (through, say, differences in inclusion criteria or problems in conduct of the study) than not see it at all. Then again, right now most of my time putting together articles involves finding them rather than trying to match them up.

Permalink to Comment

6. John Johnson on November 17, 2006 12:59 PM writes...

You know what, I may have to retract that last comment. It looks like the study didn't follow protocol, and was halted after "other research" indicated cardiovascular and neurovascular events for similar drugs. While people do seem to acknowledge that the data is next to useless for formal inference, I've seen several serious flaws already in how outlets are reporting the study.


Permalink to Comment

7. Cyan on November 17, 2006 1:32 PM writes...

There are two issues here:

(i) the general idea that data should be published without regard to the studied effect's magnitude and statistical significance (or lack thereof)

(ii) the idea that data from a poorly executed study is of little use for statistical inference

One can consistently object to the use of the data in a given study for inference because the study was poorly done, and claim that studies showing no significant effects should nevertheless be published.

Permalink to Comment

8. eugene on November 18, 2006 8:45 PM writes...

Hmmm... in my field, the biggest names don't do error bars. It's one run only and we try for good correlation between data points on that run. These runs take several days so there is a length issue, but I think it's also the fact that nobody really cares about reaction rates in 'model' systems. I'm certainly going to try to get out of doing triplicates if I can. One run can kill one day of NMR time and if it has to be high temp, will burn a lot of money as well.

Permalink to Comment

9. Still Scared of Dinosaurs on November 18, 2006 10:20 PM writes...

Of course the data should always be published. Maybe not in a journal that brags about its Distract Factor, but published.
I care a lot about whether dumb design and conduct decisions render clinical trial results invalid. I sometimes get a good laugh out of bad statistical applications or overblown interpretations. I like having some evidence to counter the objections that "THAT's never gonna happen." I like thinking about how some of the hard work of the good folks in Drug Discovery gets wasted by MD's who couldn't hack academic research or helping other human beings.
Actually the first half of the last sentence isn't true but I just wanted to see if Derek's regular audience was awake. >;)
In short, if the conduct of the trial means that the results tell you nothing about the science or the medecine I still learn something about the conduct of trials.

Permalink to Comment

10. Kent on November 20, 2006 1:04 PM writes...

"That's a tornado passing over a pig farm."

Which allows the lawyers to pork out.

Permalink to Comment

11. S Silverstein on November 20, 2006 2:25 PM writes...

Chris writes:

"Should the first study that has significance get published, even though the other 9 that were attempted (and unpublished) were not even close to significant?"

Yes, since the predictive value of the one "positive" study is likely diminished by the nine "negative" ones.

Permalink to Comment

12. S Silverstein on November 20, 2006 2:29 PM writes...

Re: # 11

Make that:

"All should be published, since the predictive value of the one "positive" study is likely diminished by the nine "negative" ones."

Permalink to Comment

13. Carl Feynman on November 21, 2006 5:34 PM writes...

5-aminosalisylic acid (mesalamine) is given for inflammation of the colonic epithelium, by enema or enteric-coated capsules. Negligible amounts enter the bloodstream, because the bowel doesn't find it very foodlike. It's hard to see how it could have any cardiovascular effects. How's that for a counterexample? If there are topical NSAID ointments, that would be another example.

Permalink to Comment

14. Derek Lowe on November 22, 2006 9:30 AM writes...

I guess I should have narrowed it down to "systemic NSAIDs". Mesalamine does seem to attain some blood exposure, though nothing like its cousin aspirin. And since it's presumably hitting the same targets, it probably has about the same liabilities (I noted that one of its more common side effects is GI bleeding, for example). Who knows what would turn up if someone ever did a study as huge and well-powered on it as has been done on the COX-2 drugs, looking specifically for CV effects?

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

The Last Post
The GSK Layoffs Continue, By Proxy
The Move is Nigh
Another Alzheimer's IPO
Cutbacks at C&E News
Sanofi Pays to Get Back Into Oncology
An Irresponsible Statement About Curing Cancer
Oliver Sacks on Turning Back to Chemistry