I mentioned yesterday that my opinion of Merck and their handling of the Vioxx cases isn't very high these days. The reason for this is the press release that the company sent out a few days ago on follow-up data to the APPROVe study, which is the one that caused the company to withdraw Vioxx in the first place.
That study was looking at possible use of Vioxx for the prevention of precancerous colon polyps. That may sound slightly insane if you're not following the field, but there's some biochemical rationale that suggests a role for inhibition of COX-2 against colon cancer. (This would be another huge market, naturally, which is why Merck - and Pfizer - have both looked into it). As the world knows, the study also showed clear evidence of an increased cardiovascular risk after 18 months of Vioxx use., and that's what started us all on the bumpy road to where we are today.
The APPROVe study was designed to have a one-year follow-up period to evaluate how long any colon-related benefits persisted. Unfortunately, it wasn't really designed (or powered, as the clinicians say) to address cardiovascular safety, so everyone just has to take what they can from the data we have. Merck, naturally, takes the current data to mean that Vioxx is doing just fine. They point out that in the post-drug follow-up year, the cardiovascular risk for the group that was taking Vioxx doesn't seem to be statistically different from the group that had been taking placebo.
Which is fine, as far as it goes. A more objective look at the data, though, show that they didn't miss statistical significance by all that much. The numbers seem to be all against Vioxx, which is enough to make you wonder if the lights would have truly flashed red in a more statistically appropriate study. As it is, Merck is in the position of saying that a study which wasn't expected to show a statistical difference between Vioxx and placebo heart safety didn't show a difference - and that that's good news.
Even if the numbers had gone the company's way, statistical arguments are a notoriously hard sell for the defense in front of a jury. Having a bunch of muddy but trending-ugly data is one of the worst things that could have happened to Merck, actually. No one knows, from these numbers, just when the effect of Vioxx on cardiovascular risk might wear off. It's a playground for the lawyers - can't you just hear it? "Isn't it true that more patients had heart attacks on Vioxx? Even during the year after they'd stopped taking the drug? No, no, I didn't ask you for a lesson in statistics - just tell me if more people had heart attacks or not!"
No, no courtroom help there. I hope, for Merck's sake, that no one at the company believes there is, and that no one's charging them by the hour to try to convince them otherwise. At this point, they're going to need something better, and I'm not sure where they're going to get it. It's past the time when we can usefully argue about whether Vioxx should have been withdrawn, about what its risk-benefit ratio is, and whether Merck should be facing thousands of lawsuits or not. They are, and more than this latest batch of data will be needed to fight them.
(See also Jim Hu's comments).
Update: According to today's WSJ, things have gotten even muddier. Here's the subscriber link, and this is a Reuters summary.
1. SP on May 17, 2006 11:49 PM writes...
If something isn't statistically significant, it's not really valid to say, "Well, it's on the bad side, just not quite enough." That's the whole point of statistics, that those effects can arise by chance. I agree that a jury would never understand this, though.
Permalink to CommentAfter all, four out of three people don't understand statistics.
2. tgibbs on May 18, 2006 12:27 AM writes...
It's a common error to say, "It's not significant, so there is no effect." But absence of evidence is not evidence of absence, and a statistical criterion that is designed to determine whether an effect is present cannot be used to determine whether an effect is absent. A result may fail to be significant because there is no effect, or because the assay is not sensitive enough to detect an effect that exists. So it is more appropriate to look at the confidence limits, and ask, just how small have you proved the effect to be, with 95% confidence. And in this case, it is clear that the answer is that there could be a pretty large effect. So Merck can take little comfort in this result, except that it could have been worse. While they certainly have not shown that the risk goes away within a year, at least they haven't (quite) proved that it does not.
Permalink to Comment3. tom bartlett on May 18, 2006 9:32 AM writes...
It is unfortunate Merck didn't get to finish the study to (presumably) vindicate VIOXX.
I also hope Celebrex sales recover from their current doldrums. I suspect any cardiotoxic effects for VIOXX would be compound specific, since no credible evidence is out there implicating selective COX-2 inhibitors as problematic from a mechanistic perspective.
Permalink to Comment4. SP on May 18, 2006 9:43 AM writes...
Is that true? I thought that the problem with COX-2 is that it's involved in both prostaglandin formation (pain and swelling) but is also part of the platelet proliferation pathway.
Permalink to Comment5. tom bartlett on May 18, 2006 9:55 AM writes...
"but is also part of the platelet proliferation pathway." I haven't heard anything to that effect, but I have been out of inflammation a few years. I recall see graphs of edema as a function of time where VIOXX clearly caused more swelling than Celebrex or Naproxsyn.
Permalink to Comment6. tgibbs on May 18, 2006 11:11 AM writes...
Actually, Mukherjee, Nissen, & Topol raised mechanistic concerns about possible cardiovascular hazards of COX-2 inhibitors as early as 2001 (Mukherjee &al., JAMA 286:254). Based on this hypothesis, the more selective an agent is for COX-2 over COX-1, the greater the expected cardiovascular hazard. They also reanalyzed the VIGOR data and raised doubts as to whether the protective effect of naproxen was large enough to account for the increased cardiovascular risk of the Vioxx group, as Merck optimistically hypothesized. Topol has been loudly saying "I told you so" ever since the APPROVe results were published.
Permalink to Comment7. Chrispy on May 18, 2006 6:37 PM writes...
Actually, the RR of 1.2 doesn't seem too bad. Take it with a baby aspirin and call it good.
Permalink to Comment8. John Thacker on May 18, 2006 11:08 PM writes...
... just how small have you proved the effect to be, with 95% confidence.
Which is itself a common statistical error, at least if you're a frequentist. :)
Luckily, I'm a Bayesian, so I'm willing to put it your way. If I were a frequentist, then what you've really shown would be "an interval where if the actual value lies there then the probability of obtaining the result we got or anything closer to the actual value is 95% or less, and thus the probability of obtaining something further away from the actual value is at most 5%."
Permalink to Comment9. Richard on May 19, 2006 12:01 AM writes...
Ok this is my rather simplistic take on it, but isn't the reason to use a COX-2 inhibitor in the first place to reduce the incidence of bleeding? What is contraindicated when using a COX-2? any COX-1 and COX-2 inhibitor! (otherwise what's the point!).
What medication reduces cardiovascular events? aspirin! Seems to me that VIOXX isn't cardio-toxic it's just the lack of cardio-protective inhibition of COX-1.
Permalink to Comment10. tgibbs on May 19, 2006 5:35 PM writes...
Seems to me that VIOXX isn't cardio-toxic it's just the lack of cardio-protective inhibition of COX-1.
Which is exactly what Merck thought when they got the VIGOR results. The problem was that the apparent protective effect of naproxen was toward the upper end of what one could reasonably expect, based on other studies, but you know...different patient groups, not directly comparable...they brushed it aside.
The VIGOR study didn't accept anybody taking aspirin, and a number of the cardiovascular events were in patients who met the criteria for aspirin. I'm sure that Merck thought that the problem would go away in the APPROVe study, which they amended to allow patients taking low-dose aspirin (whether it actually makes sense to take a COX-2 inhibitor together with aspirin is another question that neither study addresses). But in that study, the people taking Vioxx had significantly more cardiovascular events than those taking placebo--and low-dose aspirin didn't seem to help. So it looks like there may in fact be a specific cardiovascular toxicity associated with COX-2 inhibition (which is what Topol and colleagues had been contending).
Permalink to Comment11. Jim Hu on May 19, 2006 7:45 PM writes...
The CMAJ paper noted in the update was cited in the post on my blog you linked to (thanks!) and was noted in the Nature news item from this week. I hope one of the more statistically literate commenters here can get access to it, since my reaction is...huh?
They find an elevated early risk, which makes the headlines. But they also find that the rate ratio (is that similar to relative risk?) decays to baseline with continued use (a silver lining for Merck?!!).
So...about 5-6% of the users have MIs in a given time interval, and are removed from the study population. The baseline for nonusers is about 4%. What fraction of the overall population is the susceptible subpopulation...and how much is the risk elevated for them? If the data are consistent with this idea, then is seems like they should be able to estimate these from the kinetics. Wouldn't it have to be a small, highly susceptible population? Permalink to Comment