About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Bapineuzumab Does Not Work Against Alzheimer's | Main | A Diazirine That Will Knock You Out »

July 25, 2012

When Is A Company Shading the Truth About Its Clinical Trials?

Email This Entry

Posted by Derek

I wanted to point out this fine piece by Adam Feuerstein, "How to Tell When a Drug Company Fibs About Clinical Trial Results". The points he makes apply especially to small companies trying to stay afloat, but they can show up anywhere.

You need to look at when the trial started (and thus how long it took, relative to how long it should have taken), what the stated endpoints were before the trial, the time points at which these benefits (real and otherwise) occurred, and how the current trial results match up with previous ones. One general rule that I have, which Feuerstein also notes, is that when a company makes a big deal out of their investigational drug being safe/well-tolerated in a Phase II trial, that's a red flag. It's certainly a good thing that the drug was tolerated, but finding that out is not the point of Phase II.

But as the article details, clinical endpoints are where a lot of the hand-waving goes on. If a trial is designed well at all, it's run to look for the most clinically relevant signs that the investigators can think up - the ones that are going to make the patients, the physicians, the regulatory agencies, and the investors pay attention. And if a trial concludes and the company starts talking instead about various other benefits and trends that were seen in the data, while not making as big a deal out of the previously stated endpoints, well. . .there's a reason for that. It's not a good reason, and may not even be a very honorable one, but believe it, there is a reason.

Comments (15) + TrackBacks (0) | Category: Clinical Trials | The Dark Side


1. Virgil on July 25, 2012 9:28 AM writes...

While he does make some valid points, Fraudstein is a bit of a troll in some cases, including this one IMHO.

Specifically, he brings up the issue of end-points, which as anyone who's actually done a clinical trial knows, can be very difficult to nail down during trial design. In certain fields (e.g., cardiovascular) the criteria used to define health or disease status are constantly evolving, sometimes even within the time frame of the trial; think of the recent re-definition of whether HDL or "good cholesterol" is actually good nowadays compared to 2-3 years ago?

Say you do a trial, and the goal (end-point) is to lower blood pressure in patients with hypertension. The drug doesn't deliver the stated goal, but you discover during the trial that it does lower the risk of death from heart attack. Problem is, you didn't think to list heart attack as an end-point during trial design. The current system (FDA rules) essentially says you have to go back and run a whole new trial just to prove that this new end-point (heat attack) is real. You can't use the old trial because you didn't list heart attack back when you were designing it 5 years ago. That seems overly punitive IMHO, and the end result is we may miss out on new drugs due to these percieved "failures" in trial design. There needs to be a mechanism within FDA to get such un-anticipated positive end-points woven back into the trial process, without resorting to a whole 'nother trial.

On another note, Fraudstein is a smug SOB isn't he? The point of his article seems more about touting what an experienced journalist he is, rather than any substance. The bulk of it is "hey listen to me because I'm an expert".

Permalink to Comment

2. Kyle on July 25, 2012 10:05 AM writes...

I always have a hard time taking the word of financial investors when it comes to articles like this.

You can never know if there is some other agenda behind the type of advice that is given in these situations. People are always trying to fluff up a company or put it down to benefit their position in said company.

Is the authors friends shorting this company and need some help?

Permalink to Comment

3. Twelve on July 25, 2012 11:00 AM writes...

@1: Feuerstein is revealing the secret code by which the Players extract wealth from the beguiled masses - its up to the customer to have the self-awareness to avoid being taken in. Those whose money-making business depends on elevating the significance of the many weak relationships surrounding failed primary endpoints and a thickett of other feeble correlations with confident whispers to the elect about What It All Means. It is practically always a fraud, the only interesting feature of which is speculation on whether the messianic in charge ever really believed his own nonsense.

Permalink to Comment

4. Hap on July 25, 2012 11:09 AM writes...

1) How do you make a trial to tell if a drug works if you don't know what the endpoints are - how to tell if it works? Without fixed endpoints, it's hard for anyone outside the companies to tell when a result is "We think that modified endpoints are better measures of effectiveness than our original endpoints, and here they are" and not "Our drug didn't work, so we found some other (meaningless) endpoints that our drug did fulfill, and highlighted them instead". Of course, the latter is easier to distinguish from the former when you don't want to tell anyone what the endpoints were, or when you don't seem to want anyone to know that you changed them.

If you're using some sort of trial where you're collecting data as you go along to try to determine what the constraints and endpoints of your trial are (and not cherry-picking correlations that look good until you can bail), it would seem like you'd want to be clear about that with the FDA, and maybe (if honesty paid) with your investors.

2) When you won't say anything about your results other than the press release, and when the person who ran your study is mysteriously unavailable for comment, well, that also seems like a good sign that whatever you've got is not what anyone wants.

My default assumption is that if you have to hide your results from your investors or the public, you probably shouldn't have any investors.

Permalink to Comment

5. Imaging guy on July 25, 2012 11:41 AM writes...

I should think what Adam Feuerstein (AF) wrote is really valid. Primary outcomes of any therapeutic trials should be meaningful clinical outcomes such as prevention of death and disabilities if certain number of these events might happen within trial period. Although they seemed to have found some meaningful outcomes such as progression to heart failure and time taken to hospital re-admission, AF raised good counterpoints. The question here is why they do not release the information about the surrogate outcomes they were supposed to measure.

Permalink to Comment

6. bbooooooya on July 25, 2012 12:05 PM writes...

"There needs to be a mechanism within FDA to get such un-anticipated positive end-points woven back into the trial process, without resorting to a whole 'nother trial."

Darn statistics, that's the real problem. Maybe we need new math?

"On another note, Fraudstein is a smug SOB isn't he?"

No, I don't believe that he is. He's one of the few people in biotech that publicly tells how things really are in biotech. If there were more people like him you'd have fewer scamming biotechs diverting money from worthwhile projects that could actually help people.

Funny that no one ever complains about the shills that pump worthless biotechs that then, nearly inevitably, go kaput.

Permalink to Comment

7. HTSguy on July 25, 2012 12:31 PM writes...

@6 +1

Unless the effect of the new endpoint effect is so huge that it can withstand the correction for multiple testing, posthoc finding a new "significant" endpoint proves nothing.

Permalink to Comment

8. rab on July 25, 2012 2:55 PM writes...

@1; Good points about Feuerstein, but the issue is that trials are only powered to detect primary end points (by definition). Sifting through all the data to look for positive non-prespecified endpoints is a recipe for spurious results.
I thought that the article was interesting from several standpoints;
a) the use of off the shelf stem cells
b) stem cells for acute MI (no believable positive trial results published, but an ongoing story)
and c) what is going to happen to Osiris if/when the inevitable news has to be released.

I'd be interested to read the views of this community on these points!


Permalink to Comment

9. Clinical pharmacologist on July 25, 2012 4:49 PM writes...

One of the objectives of a Phase II study is absolutely to test the safety of a new medication. It is often the first chance you get to see what happens in patients with the pathology of interest rather than healthy subjects or small groups of intensively monitored patients and things can look very different, very quickly.

In the end the utility of a new medicine is the balance between the expected efficacy and the expected toxicity so you have to attend to both with the same level of attention in the development process. Sadly, we rarely do as "good" news builds careers and " bad" news doesn't.

Permalink to Comment

10. Validated Target on July 25, 2012 5:22 PM writes...

Feuerstein notes that the trial ended in Dec 2011 (according to clinicaltrials dot gov). Surely, if the results had been conclusively positive, the results would have been submitted in a Prochymal strengthened heartbeat. However, 7 months to publish isn't too bad if there was a lot of data or analyze and massage, if legal steps in to review the manuscript, and to just plain overcome any laziness to write a paper.

But there are plenty of cases of lousy clin trial outcomes being given the positive spin treatment. There seems to be a lot of 'It works as well or better than placebo. At least it wasn't WORSE than placebo.'

Permalink to Comment

11. bigpharmaliar on July 25, 2012 5:55 PM writes...

"small companies trying to stay afloat"

Does everyone REALLY believe it's only "small" companies trying to "stay afloat" that participate in this nonsense.

Permalink to Comment

12. bodele on July 26, 2012 1:48 AM writes...

Fine article and link to the Feuerstein piece.

I would just add to the comment regarding safety in Phase II trials with a point specific to oncology trials. I agree safety is a major part of Phase II (@9), but of course I suspect the point here is to see whether the goalposts have shifted during the trial. However, there are an increasing number of products (especially targeted agents) that are not (or cannot be) dosed to MTD because an MTD was not established in Phase I. In this case, my experience is that companies often have a problem keeping people focused on efficacy because the basic assumption in oncology is still that if you're not dosing at MTD, you're not dosing high enough.

In these (still) exceptional cases, I would say that stressing safety in a Phase II is doubly important, although of course efficacy still needs to be shown in some solid way (e.g. biological/biomarker endpoints ideally, in addition to PFS/OS).

Permalink to Comment

13. Sili on July 29, 2012 9:36 PM writes...

If a trial is designed well at all, it's run to look for the most clinically relevant signs that the investigators can think up
I don't know if this is a stupid question, but why are trials even allowed to begin, if they're not well-designed?

Isn't there some sort of government overview of human trials?

Permalink to Comment

14. Jonadab on July 31, 2012 3:59 AM writes...

> why are trials even allowed to begin,
> if they're not well-designed? Isn't
> there some sort of government overview
> of human trials?

Name one government that has ever existed in the history of the world that contained any agency sufficiently well-run to reliably, consistently determine whether something is well-designed before allowing it to be implemented.

Government is necessary, but it is not magic.

Permalink to Comment

15. Hap on July 31, 2012 2:11 PM writes...

You'd figure the companies themselves might want to design their trials well, though, at least if they're actually interested in bringing their drug to market and not just in pumping their (unfortunate) investors for more cash.

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

The Last Post
The GSK Layoffs Continue, By Proxy
The Move is Nigh
Another Alzheimer's IPO
Cutbacks at C&E News
Sanofi Pays to Get Back Into Oncology
An Irresponsible Statement About Curing Cancer
Oliver Sacks on Turning Back to Chemistry