About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Oh, And While You're At It. . . | Main | Weirdness: Montagnier Again, Teleporting DNA »

January 24, 2011

Not Enough Progress Against Cancer?

Email This Entry

Posted by Derek

Here's a topic that's come up here before: for a new cancer drug, how much benefit is worthwhile? As it stands, we approve things when they show a statistically meaningful difference versus standard of care (with consideration of toxicology and side effects). But should our standards be higher?

That's what this paper in the Journal of the National Cancer Institute is proposing. The authors look at a number of recent Phase III trials for metastatic solid tumors. It's a tricky business:

When designing a randomized phase III clinical trial, the investigators must specify in the protocol the difference (δ) in the primary endpoint between experimental and control groups that they aim to detect or exclude (24). The number of patients to be recruited and the duration of the study will depend on the value of δ; increasing the sample size will allow the detection or exclusion of smaller values of δ. Ideally, trials should be designed such that δ represents the minimum clinically important difference, taking into account the tolerability and toxicity of the new treatment, that would persuade oncologists to adopt the new treatment in place of the standard treatment. Of course, the opinions of oncologists as to what constitutes a minimal important value of δ will vary, but a reasonable consensus can be reached by seeking the opinions of oncologists who manage a given type of cancer. For example, an increase in median survival by less than 1 month for patients with advanced-stage cancer would not be regarded by most as clinically important, unless the new agent had less toxicity than standard treatment, whereas an improvement of median survival by greater than 3 months for a drug that was reasonably well tolerated would usually be accepted as clinically important.

And the problem is, given the costs of some of these drugs versus their benefits, you run the risk of, finally, paying too much for too little. I know that people say that you can't put a cost on a human life, but that's probably not true, when you're talking about an entire economy. As the article points out, the rough estimate is that the developed world can support expenditures of up to roughly US $100,000 per year of life gained, but past that, we're into arguable territory. (If someone wants to spend more out of their own pocket, that's another matter, naturally, but at these levels, we're usually talking public and private insurance).

The benefits can indeed be marginal, and you have to look at the statistics carefully so as not to be misled:

. . .several trials showed a statistically significant difference in a major outcome measure between the experimental and control groups, but the difference in outcome was of lower magnitude (eg, hazard ratio was closer to one) than that specified in the protocol. For example, the clinical trial that led to approval of erlotinib for treatment of pancreatic cancer was designed to detect a relative risk reduction of 25% (HR ≤ 0.75), but the best estimate of hazard ratio from the trial showed a relative risk reduction of 18% (HR = 0.82, 95% confidence interval = 0.69 to 0.99). The difference was statistically significant (P = .038), but the median survival differed by only 10 days.

What happens is that the trials are (understandably enough) designed to detect the minimum difference that regulatory authorities are likely to find convincing enough for approval of the drug. And the FDA has generally set the bar at "anything that's statistically significant for overall survival". These authors (and others) would like to see that raised. They're calling for trials not to go for a statistically significant P value, so much as to show some sort of meaningful clinical benefit - because it's become clear that you can have the first without really achieving the second.

I think that might be a good idea, whether or not you buy into that cost-per-year-of-life figure or not. At this point, I think it's fair to say that we can come up with drugs that provide some statistical measure of efficacy, given enough effort in the clinic, for many kinds of cancer (although certainly not all of them). But how many add-a-month-maybe therapies do we need? Not everyone's convinced, though:

Wyndham Wilson, a lymphoma researcher at the National Cancer Institute in Bethesda, Maryland, argues that the proposed clinical endpoints are somewhat arbitrary. “What constitutes a clinically meaningful difference? Six months is obvious, but where do you cut the line?” What's more, he adds, simply focusing on median responses often ignores important outlier effects that could merit approval for an experimental drug. “The difference in overall survival may not be great, but it may be driven by a great benefit to a small group,” he says.

Problem is, it's often quite difficult to figure out who that small group might be, and just treat them, instead of treating everyone and hoping for the best. And there's always the argument that these therapies are stepping stones to more significant improvements, but I wonder about that. My impression of oncology research has always been more like "OK, this looks reasonable. Lots of these tumors have UVW upregulated; let's make an UVW inhibitor. (Years later): Hmm, that's disappointing. Our UVW inhibitor doesn't seem to do as much as you'd think it should. But now it's been found that XYZ looks like it's necessary for tumor growth; let's see if we can inhibit it. (Years later): Hmm, that's not as big an effect as you would have thought, either, is it? Seems to help a few people, but it's hard to say who they'll be up front. How's the JKL antagonist coming along? No one's tried that yet; looks like a good cell-division target. . ."

It's just sort of one thing after another - that one didn't work so well, neither did that one, this other one and these three together seem to be a bit better, but not always, and so on. Would we learn as much, or nearly so, just from the earlier clinical work on such compounds as opposed to taking them to market? And although you can't deny that there's been incremental progress, I'm not sure what form it's taking. It's very likely that the answer isn't to keep turning over mechanistic ideas until we find The One That Really Truly Works - cancer is a tough enough (and varied enough) disease that there probably isn't going to be one of those.

My guess is that meaningful cancer success will come from combinations of therapies that we mostly don't even have yet. I think that we'll need to hit several different mechanisms at the same time, but that some of what we'll need to hit hasn't even been discovered. And on top of that, each patient presents a slightly different problem, and ideally would receive a more customized blend of therapies (not that we know how to do that, either, in most cases).

What I'm saying is that we'll probably need combinations of things that already work better than most of what we have already, and that these will stand out enough in clinical trials that we'll know that they're worth developing. As it stands, though, companies see hints here and there in the clinic, enough to run a Phase III trial, and if it's large enough and tightly controlled enough, they see enough efficacy to get things through the FDA and onto the market. Would we be better off to not proceed with the marginal stuff, and put the significant amounts of money into things that stand out more? Or would that choke off the market too much, since we mostly end up making marginal things anyway (damn it all), leaving no one able to keep going long enough to find the good stuff? It's a hard business.

Comments (32) + TrackBacks (0) | Category: Cancer | Clinical Trials | Regulatory Affairs


1. sigma147 on January 24, 2011 11:26 AM writes...

Likely we'll see improved returns when a more "personalized" approach becomes more widely used. Once the patient's specific cancer phenotype and genotype can be more closely tied together and screened so that specific therapeutic regiments are applied, the therapeutic effectiveness of specific drugs may be significantly bolstered. In other words, once oncologist learn how to screen for specific mutation patterns that respond well to specific therapeutics, the whole approach of cancer treatment is likely to become guided by applying the most effective medicine against the patient's particular form of cancer. After all, not all cancer is created equal.

Permalink to Comment

2. HelicalZz on January 24, 2011 11:41 AM writes...

"My guess is that meaningful cancer success will come from combinations of therapies that we mostly don't even have yet. "

Agreed. And we need to see some collaborative trials performed where more than one company contributes a therapeutic / cocktail piece to the tested set, and the companies have a shared interest in marketing the outcome, whatever it may be. Shared risk seems like a no-brainer for this industry, but it hasn't yet been embraced.


Permalink to Comment

3. RKN on January 24, 2011 12:28 PM writes...

Cancer is a systems biology problem: I think we will not see significant improvements in patient care until we more thoroughly understand how all these oncogenic gene products interact at the level of the proteome to produce the disease phenotype, and perhaps just as importantly, how they interact to sustain a normal phenotype. And that will mean a deeper understanding of molecular networks, going beyond the present state of knowledge of over-simplified "linear" pathways.

Permalink to Comment

4. Andy on January 24, 2011 12:47 PM writes...

I'd argue that the bar should be lower for first-in-class drugs with idea that follow-on real world treating data will benefit all successor therapeutics in the class, even if the initial health benefit is modest.

Permalink to Comment

5. Steven Walker on January 24, 2011 1:06 PM writes...

This is, I think an argument based in obsolescent thinking. Dr. Wilson almost makes the needed point, but as is usually the case with clinical researchers steeped in our 50 year old system, stops short of the reality that the comparative statistics approach discussed in this post can do little but measure the difference in outcomes between the median patient in the treatment arm and the median patient in the control arm for a very narrowly formulated hypothesis in a very narrowly (and usually artificially) controlled population. This is about as dumb a way to try to evaluate the safety and efectiveness of a new drug, or drug combination, that anyone could come up with. Understandable, since it was put in place in 1962 by Congress (almost 50 years ago), and never seriously evaluated or even questioned, since. We can do much better, but only if the debate turns to really new ideas about how to develop new drugs using real science, which any competent scientist outside the field of clinical research and FDA regulation will tell you allows, and in fact requires, the use of evaluative tools other than the incredibly limited capabilities of relative frequentist statistics. RF stats is the extremely narrow field of math that underlies our now very seriously tarnished and failing "gold standard" - the randomized clinical trial. Dr. Wilson is right to imply that it is essentially a useless tool for evaluating the new targeted medicines now emerging, and that alone should be turning the debate toward entirely new ways of conducting and evaluating clinical trials and making approval decisions. A determination of whether a benefit is clinically meaningful that is based on the almost meangless difference in outcomes between two population medians, takes us in a direction we should not go. It is a garbage in - garbage out scenario. New ideas please.

Permalink to Comment

6. barry on January 24, 2011 1:09 PM writes...

"My guess is that meaningful cancer success will come from combinations of therapies that we mostly don't even have yet. "

buried in this discussion are two big problems:
1-we don't know how many diseases cancer is
2-the clinical population in which we're testing new cancer therapeutics is overwhelmingly late-stage cases that have accumulated many mutations (i.e. they actually have multiple diseases)

Novartis tested Gleevec in early-stage CML and got dramatic life prolongation. "Cure" might not even be too strong a word. In more advanced cases, not so good.

Until the industry figures out how to test novel anti-cancer therapeutics in early-stage cancers that might have just one driving mutation, we will continue to see modest and ambiguous results, even with compounds that might have been game-changing in a cleaner genetic/disease setting.

Permalink to Comment

7. Rob on January 24, 2011 1:10 PM writes...

For many of the truly tough cancers, achieving the same lifetime extension with fewer side effects would be highly desirable. If you get a year extra with treatment for lung cancer (currently achievable), but it is a year of hell, what do you gain? Especially if the alternative were 9 months of reasonable quality living.

That said, I think that you and the posters are correct that the therapy that will work with the tough ones is not yet understood and will probably come from a systems approach. (one reason I'm working in systems biology).

Permalink to Comment

8. Lester Freamon on January 24, 2011 1:19 PM writes...

How we get to combination therapy is the big question.

I'd argue that, instead of betting it all on high-risk, $100m phase 3 trials, the best path for collaboration would be for the NIH to support a bunch of smaller (30-50 patient) trials choosing promising compounds that passed phase I with some signal of anti-cancer activity and put them in 2 to 3 drug cocktails (on top of standard of care). From there, we can go on to phase III. But it's going to be hard to convince drug companies to spend 8 figures on another trial of a compound that didn't do well enough the first time.

Permalink to Comment

9. IchDich on January 24, 2011 1:20 PM writes...

I am no medicinal chemist, so not really into the field, but what are the cancers that we do have a pure chemical "cure" for? One that works and is quite general, and not cis-platin...

Permalink to Comment

10. MTK on January 24, 2011 1:33 PM writes...



Excuse my ignorance on the topic, but what exactly would you suggest as an alternative to randomized clinical trials? How would one actually go about rigorously evaluating treatments in an ethical manner by other means?

I would also argue that current methods are not limited to simple median vs. median comparisons. For example, double-humped data as would be seen if in one treatment group a set of super-responders was seen vs. the other treatments group, would lead to efforts to try and identify factors leading to super-response.

As with other comments, the whole personalized medicine or other clinical approaches are great to talk about it, and research, but in the absence of real genotype/phenotype markers how in the world do you ethically run clinical trials other than comparing treatments added to currently accepted treatments?

Permalink to Comment

11. watcher on January 24, 2011 1:38 PM writes...

Have to agreee with #7. It is very hard to see how the current financial and clinical trial models which are used to manage current efforts in drug discovery & development can support novel approaches to combination treatments in oncology, and (even more unlikely) individualed therapies. Fr these to make efficient and effective progress, something different is needed in terms of the expectations for declaration of safety and efficacy, chosing patients, accepting of controls, etc.

Permalink to Comment

12. Bob on January 24, 2011 1:43 PM writes...

Good thread here!

Permalink to Comment

13. Keith Robison on January 24, 2011 1:48 PM writes...

A key challenge in cancer therapy is discovering the correct dose & schedule & combination. Post-approval optimization has made significant advances in patient care, and many of those came from initial weak success. Non-small cell lung cancer is one area I've heard experts state has benefitted from many incremental gains but few large leaps.

Even long before the current thinking on "personalized oncology medicine", patient subsetting was important for identifying valuable agents -- The Emperor of All Maladies gives the example of tamoxifen & estrogen receptor status. The key question is if a weak positive signal is seen for a drug that may be working in only a subset of the trial population, do you require that the subset be identified prior to approval, post-approval, or just let the market sort it out. Actually identifying these subsets is decidedly non-trivial.

Permalink to Comment

14. Matthew on January 24, 2011 1:53 PM writes...


The poster child is usually Imatinib for treatment of CML.

To Derek, I agree with the general sentiment, but I don't think there is a creative way around incremental changes. I think you're right about combos, and that is certainly the direction everyone is headed. But, until we have a lot of really good tool compounds to test the hypothesis pre-clinically, there's no other option than to keep passing favorite compounds to the clinic and cross fingers. What are you supposed to do when you develop a well tolerated XYZ inhibitor and you find out it kills most every something-anoma cell line you've tested? Most people would rush to design the smartest clinical trial possible, hope it works and then scramble like mad to find some combination to expand indications. It may be costly and incremental, but what's the alternative? Hold the drugs back?

Permalink to Comment

15. Matt on January 24, 2011 2:37 PM writes...

I don't see how setting the bar higher improves things. It does not seem to be the case that the pharmaceutical industry could be hitting home runs, but is content to crank out singles because the regulatory bar is set too low. The financial incentives all push for home runs, cancer's just a damn good pitcher on the mound. Sometimes the best you can do is foul off balls or take what you can get until the sweet pitch arrives.

As far as pricing goes, overpriced yet underperforming treatments survive because there are no better alternatives. Getting rid of the weak performers doesn't produce better alternatives, it just drops us back to no options at all.

With regard to trials aiming for meaningful life extension rather than significance testing, hasn't the FDA pushed rather strongly for this already? You can design your clinical trial however you want, but the FDA is going to strongly consider whether the candidate provides a meaningful benefit?

And if, as seems likely as sequencers drop in price by orders of magnitude, sequencing a cancerous biopsy becomes a common lab procedure, won't all those UVW, XYZ, and JKL inhibitors be invaluable tools in the arsenal?

Permalink to Comment

16. dearieme on January 24, 2011 3:35 PM writes...

"I know that people say that you can't put a cost on a human life.."

Humankind cannot bear very much reality.

Permalink to Comment

17. gippgig on January 24, 2011 10:38 PM writes...

This is one case where an ounce of prevention is clearly worth a pound of cure. How much effort is being made to develop drugs that prevent cancer (for example, by inducing DNA repair enzymes)?

Permalink to Comment

18. willia_A_Nelson on January 25, 2011 4:32 AM writes...

It's the environment stupid. Clean up the polluted waters, polluted air, contaminated foods, prohibit cigarette smoking etc. ad nauseum and cancer statistics will diminish dramatically. Common sense is no longer common!

Permalink to Comment

19. cliffintokyo on January 25, 2011 4:34 AM writes...

We are probably all in it together on this one, for the time being.
What I mean is:
1) The reality at present is that the single agents we are using have only limited efficacy;
2) Currently combinations are being explored with more frequent clinical success than with single agents;
3) We probably will not make significant new progress until we understand better why some agents work when used early and some do not
(Do they stop or delay metastasis? How?);
4) We would probably be more successful if we could use some drugs earlier before metastasis starts; but
5) It is very difficult to justify or conduct clinical trials with new agents given earlier in the disease process (Bioassays? For what?);
6) Anti-cancer drug treatment is usually indicated for patients whose disease has progressed;
7) This is the most difficult disease stage to be successful in; because
8) Most cancer treatment success is based on removing or zapping the solid tumor until there appears to be none left, and hoping that you got to it before metastasis started;
9) With increasing use of chemo, hopefully as insurance (only empirical clinical evidence, no scientific basis yet, that I am aware of).

See what I am getting at? Investigators in all disciplines involved need to communicate real good to make progress.
Communicate, then cooperate!

Permalink to Comment

20. cliffintokyo on January 25, 2011 4:41 AM writes...

(9) should read:
"With increasing use of post-surgery chemo....."

Permalink to Comment

21. dvizard on January 25, 2011 6:33 AM writes...

"See what I am getting at? Investigators in all disciplines involved need to communicate real good to make progress.
Communicate, then cooperate!"

Are you trolling here? Reading sentences like this makes me want to throw rotten fish at you.

Surely we should all sit together and talk and communicate. Once we do that, surely a definitive cure against cancer will have to appear, because you can't do wrong when you're oh-so-interdisciplinary.

Permalink to Comment

22. g on January 25, 2011 9:44 AM writes...

I agree with Matt (#15). It seems somewhat silly, that with approval rates near 10% for drugs tested in phase III, to ask drug companies to "do better". It is not as if people are not trying. They are trying very hard. Cancer is not one disease, but many. Showing efficacy is tremendously difficult. Setting the regulatory bar higher will stifle investment because people have a hard enough time getting over the current one.

The economics of collaborative trials and getting personalized medicine to actually pan out is immense. I do not see industry alone making it work.

Permalink to Comment

23. luispedro on January 25, 2011 12:30 PM writes...

A lot of these trials involve terminal or at least late-stage patients. Even small differences in survival in this population can (but does not always) indicate much better effects when the drug is then used earlier in the cancer progression.

Herceptin had this exact path. Some of these regulatory changes might have caused it to never be approved, while it is actually a great drug.

Permalink to Comment

24. metaphysician on January 25, 2011 5:18 PM writes...


Aside from the obvious ( cigarettes ), do you have any evidence for cancer rates being broadly determined by environmental factors? Note that a lack of cancer in the pre-industrial world isn't indicative, because of the difference in lifespan.

So, has anyone done studies comparing the population of two regions with similar average lifespan but different environmental quality indicators?

Permalink to Comment

25. Matthew on January 26, 2011 1:57 PM writes...


One possibility that comes to mind is that women in the San Francisco Bay area (particularly in the North Bay) have a significantly higher incidence of breast cancer. The epidemiology is unknown, so speculate away.

Permalink to Comment

26. TexUte on January 26, 2011 5:06 PM writes...

Prevention would indeed be the ideal, but as a practical matter it's a commercial nightmare. Extremely long trials involving very large numbers of patients, safety profile of any such drug would have to be extremely clean, couldn't be very expensive because it would require long-term chronic use, etc., etc. Only the government could afford to run a study like this, and they're not much good at drug development.

Permalink to Comment

27. cliffintokyo on January 27, 2011 3:10 AM writes...

Many of your comments also apply to clinical trials of treatments for early stage cancer (#19). As you say, prevention is not a realistic option for clinical testing in any terms (cost, ethics, risk, subject numbers, whatever).
Until the risk aversion climate changes (Ha!), we will be restricted to testing NMEs only in trials for patients with advanced cancer.
It is important to communicate this limitation to the discovery research community before they become too enthusiastic about potential new paradigms.
Despite the flying rotting fish, I stick to my guns about the critical need for better communication before collaboration.
In my experience, poor communication between silos is the usual reason for failure of the pharma R and D process.
(Don't tell us what to make, just test our new compounds....we cannot ethically run clinical trials on these new compounds even though they work in vitro/vivo/animal models; etc, etc)

Permalink to Comment

28. gippgig on January 27, 2011 3:43 AM writes...

Wouldn't a clinical trial of a cancer prevention drug be largely the same as a vaccine trial (which are done)?

Permalink to Comment

29. gippgig on January 27, 2011 3:45 AM writes...

Wouldn't a clinical trial of a cancer prevention drug be largely the same as a vaccine trial (which are done)?

Permalink to Comment

30. gippgig on January 27, 2011 3:59 AM writes...

If anyone (such as myself up until now) is wondering why duplicate entries such as the previous two appear so often, I hit Post, the site seemed to freeze, I hit refresh, and, oops! two copies...
Is there any way I can delete the duplicate?

Permalink to Comment

31. cliffintokyo on January 27, 2011 10:09 PM writes...

#28 Good question, but the answers are rather obvious.....
1. Vaccines are stupendously safe; would you dare to say the same about any drug to treat cancer?
2. How would you decide when to stop the trial, i.e. stop administering the treatment? Would it be ethical to stop treatment, for high-risk patients?
3. Follow-up would be over a lifetime for a cancer prevention drug. Who would pay?

Any further advice about how to design clinical trials will have to be paid for! Cliff

Permalink to Comment

32. soydegales on January 31, 2011 11:11 AM writes...

or, we're simply all working on the wrong targets with the wrong approach, driven by incorrect assumptions. "improved statisics" just sounds like a justification for snake oil and gets people very suspicious. especially for a drug that's going to cost >100K per year. more charitably, perhaps we need to consider a diversity of approaches rather than all piling onto the same bandwagon?

"personalized medicine" sounds great but how does this align with therapy-selected genotypes in tumors after 2, 3, 4 or 5 rounds of prior chemo? hard to prove that these are clonal and that pre-therapy "driver" mutations still drive in that setting. they might, but are you going to bet the farm on that? even if you go first line, for the personalized approach to hold merit, more biology is required to prove a given targeted therapy is still valid in combination with standard of care.

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

The Last Post
The GSK Layoffs Continue, By Proxy
The Move is Nigh
Another Alzheimer's IPO
Cutbacks at C&E News
Sanofi Pays to Get Back Into Oncology
An Irresponsible Statement About Curing Cancer
Oliver Sacks on Turning Back to Chemistry