Here's every outsourcing manager's nightmare: you contract out for research, and your CRO turns around the studies you want right on schedule. They send back the complete data package, and everything's in place. But they faked it.
Gives you the shivers, doesn't it? Well, unfortunately, it seems to be the case with an outfit called Cetero Research out of Houston. The FDA has been investigating them, and has found enough warning signs to believe that none of the data generated there can be relied on. The companies that used their services now have to decide if they need to re-run these studies themselves, which I'm sure excites them no end.
FDA is taking this action as a result of two inspections of Cetero's bioanalytical facility in Houston, Texas conducted in 2010, as well as the company's own investigation and third party audit. The inspections and audit identified significant instances of misconduct and violations of federal regulations, including falsification of documents and manipulation of samples.
The pattern of misconduct was serious enough to raise concerns about the integrity of the data Cetero generated during the five-year time frame. FDA concurs with the assessment of Cetero's independent auditor who stated, "This misconduct appears to be significant enough to cast doubt on the data generated...If the foundation of the laboratory is corrupt, then the data generated will be also."
The investigation appears to have started with an internal whistleblower, according to this letter from the FDA. The company conducted its own investigation, but the agency is slamming them both for the violations and for the inadequacy of that internal review:
According to your internal investigation, electronic records of key card building entry times demonstrated approximately 1900 instances of blood/plasma samples allegedly extracted on weekends and holidays between April 15, 2005, and June 30, 2009, where the arrival times of laboratory chemists were greater than one hour after the documented start time of the sample extraction. In addition, there were approximately 875 instances where the laboratory chemists were not present in the facility at the documented sample extraction date.
The company's original theory was that its scientists were falsifying the weekend hours in order to get paid more, but that the numbers themselves were OK. But as the FDA notes, they didn't go on to see if times were being faked during the working week (where there was no overtime incentive), and it turns out that they were. Falsus in uno, falsus in omnibus, as the lawyers say, and that appears to be the case here. A third-party investigation found many other irregularities - faked standard concentration curves and back-filling of internal standards, for example. One of the worst was the use of not-yet-analyzed samples as preliminary runs, apparently as a quick look to see if the numbers were going to come out looking good or not. That's not quite how you want to be conducting your research, and most especially if you're going to follow that up by "fixing" the numbers that you don't like. The whistleblower alleged just that. The FDA letter says that the company's conduct makes that a real possibility, and that the documention that's available sure isn't sufficient to rule it out.
You hear a lot of talk (in the comments on this blog and in other venues) about the alleged unreliability of offshore CRO work. But you don't have to go to China to get shaky data. Houston is far enough.
Update: Here's Cetero's response to the FDA.
1. johnnyboy on July 28, 2011 9:43 AM writes...
Indeed. Dishonesty is a human trait, not a cultural one, and 'financial incentives' work the same all over the planet.
What is striking to me is that this was uncovered only after a whistleblower complaint, not by the FDA's normal inspection process. I work for a CRO, we've been doing studies for FDA regulatory submissions for over 5 years, and we've never even been inspected by the FDA (we would love to be, by the way). In these circumstances it is a little bit scary how easily a rotten CRO can fly under the radar.
Permalink to Comment2. Student on July 28, 2011 10:00 AM writes...
I looked up their office in Houston and it is rather small. I know I'm out of the loop on these kinds of things, but would it have been possible for them to be performing all of these studies out of such a small leased out space?
Permalink to Comment3. CMCguy on July 28, 2011 10:16 AM writes...
Definitely causes fear and concern as dealing with CROs and CMOs has enough issues even in the best of situations. As a sponsor can audit, monitor, review and supervise to the extreme and still probably would not detect dishonest behavior whether individual or systematic, especially if well disguised. It always comes down to basic trust built up by development of long-term relationships which often can get short changed when mentality is simply lowest cost bidder that does not account for quality or reliability factors.
Permalink to Comment4. Hap on July 28, 2011 10:22 AM writes...
I think the assumption would have been that US CROs aren't inherently more honest than Chinese or Indian CROs, but that (through painful history) there is more regulation and oversight to keep domestic CROs from taking the cheap dishonest way out that might not be present elsewhere. If that is no longer true or was never so, well then we're severely hosed.
If someone tells you what you expect to hear, and all the i's are dotted and t's are crossed (there actually is experimental data), how do you determine if it is correct of not? If it's clumsy, pieces of data won't be consistent, but if the fraud is good, it would be impossible to detect unless you replicate the data yourself.
Permalink to Comment5. Hap on July 28, 2011 10:25 AM writes...
I think the assumption would have been that US CROs aren't inherently more honest than Chinese or Indian CROs, but that (through painful history) there is more regulation and oversight to keep domestic CROs from taking the cheap dishonest way out that might not be present elsewhere. If that is no longer true or was never so, well then we're severely hosed.
If someone tells you what you expect to hear, and all the i's are dotted and t's are crossed (there actually is experimental data), how do you determine if it is correct of not? If it's clumsy, pieces of data won't be consistent, but if the fraud is good, it would be impossible to detect unless you replicate the data yourself.
Permalink to Comment6. pete on July 28, 2011 10:34 AM writes...
So a typical workday for the laboratory chemists was to arrive > 1 hour after fictitious withdrawal of blood. Apparently blood assays and assay results were also fictitious.
So how did these lab chemists fill their day ? Lemme guess: 1) phone calls to project managers on the other side and 2) Powerpoint. (Aggressive timelines, y'know ?)
Permalink to Comment7. Anonymous on July 28, 2011 10:46 AM writes...
I agree with Hap, this isn't going to help domestic CROs make their case against their foreign counterparts.
But hey, look on the bright side; our entire economy is about to tank on August 2nd so maybe none of this matters.
Permalink to Comment8. Still Scared of Dinosaurs on July 28, 2011 11:06 AM writes...
Wow, yikes. I want to convince myself that this could never have happened on any trial I have worked on. For the most part I can because I've usually worked with good Clinical people and for companies with adequate resources to keep an eye on things, but...if it was going to happen it would be with a PK vendor that we basically decide to trust. Not only are PK studies the most likely clinical trials to have all the clinical activity outsourced but they are also the ones for which the bulk of the analysis is shipped out. Have the same vendor do both at the same time major Phase 3 trials are going on and maybe this could happen to me. Very scary.
Think about how bad this could get. I remember a drug that had good Phase 3 data but needed to change their formulation to get it up to commercial yields. They had a major setback because they failed a bioequivalence trial and had to repeat the Phase 3s. The drug eventually got on the market and turned out to have rare but very, very serious AEs.
Imagine where the legal liabilities would fall if the drug had been approved earlier because of faked PK data.
Permalink to Comment9. johnnyboy on July 28, 2011 11:10 AM writes...
@ 3 CMCguy: I would add that one complicating factor in the relationship between CRO and Sponsor is that a Sponsor is very unlikely to look for errors or fraud in the work as long as the data is good for them. It's weird, unexpected or unwanted results that will raise eyebrows. So the chemists at Cetero were probably making sure that everything in their results looked tickety-boo, so that business kept coming. But as any practicing researcher knows, weird and unexpected results are more often the norm than the exception. It's consistently getting just what you want or expect that should raise an eyebrow, not the inverse.
Permalink to Comment10. Jessup on July 28, 2011 11:38 AM writes...
With the analytical organizations I've used falsification of data is common. It's not as blatant as in the above article, but time plays a crucial factor in which results are to be ignored.
Say they screw up a machine calibration and that will take several hours to correct--well, they simply go ahead using the uncalibrated machine and data.
Say they need to work out new conditions for a class of compounds-- no way, too much work-- use the old conditions and assume certain peaks correlate to certain compounds. Etc...
Of course I had to clean up all these messes on my own and discard the supposed results.
It's why these analytical companies are largely staffed with h1-b visa and other temp foreigners.
They'll do anything you tell them to do without thinking about it.
They're paid to do, not to think.
Permalink to Comment11. Coffee on July 28, 2011 12:22 PM writes...
As a scientist whenever I look at a poster/paper/etc. I look at the reliability/consistency of the data (standard deviation/P values/etc.) before I consider the conclusion of the results. This begs the question if the MBAs are driven by results of the data and if a scientist making those decisions would be driven by the consistency of the results.
Permalink to CommentThe financial incentives for MBAs is certainly there, but not so much for scientists.
...Imagine a corp. culture where you could get a bonus for small P-values.
12. Coffee on July 28, 2011 12:24 PM writes...
As a scientist whenever I look at a poster/paper/etc. I look at the reliability/consistency of the data (standard deviation/P values/etc.) before I consider the conclusion of the results. This begs the question if the MBAs are driven by results of the data and if a scientist making those decisions would be driven by the consistency of the results.
Permalink to CommentThe financial incentives for MBAs is certainly there, but not so much for scientists.
...Imagine a corp. culture where you could get a bonus for small P-values.
13. Anonymous on July 28, 2011 12:38 PM writes...
"Trust is the coin of the realm."
-Sec. of State George Shultz before the Iran-Contra Committee.
Permalink to Comment14. TJMC on July 28, 2011 12:44 PM writes...
This is extremely disconcerting - and a huge expense to those honest sponsors that relied on that data to make decisions. Wonder what names are on that client list?
However, this matches up with a key concept I have been advising clients on. That is, past practices and tech have created the assumption that we need to trust initially, but always validate. Currently, this creates a big lag that can hide a lot of blemishes. Apparently it also has big loopholes that can be exploited. The new idea is that having access to realtime updates on data allows for sophisticated, ongoing checks for inconsistencies.
I helped one client create a solution along these lines. But for a different reason - delays in getting medical image data (from site thru CRO) often took weeks to months. They wanted greater speed and enough notice to have a patient re-assayed in time before the clinical protocol window closed. They also wanted earlier data to support adaptive trial decisions.
This approach (cloud, "virtual cloud" in a shared database owned by the client) would ALSO make this kind of deception far more difficult. Normal savings for the original approach was in the millions. Avoidance of this kind of fraud: priceless.
I touch on this somewhat here in terms of compressing time to quality decisions. No reason it cannot work for validation and QA decisions as well: http://www.randdreturns.com/grasping-the-information-explosion-to-advantage/
Permalink to Comment15. lt on July 28, 2011 12:49 PM writes...
If physically and chemically possible it might be a good habit for a sponsor to start systematically and routinely spiking some samples to make sure the CRO is not pulling data out of thin air and will spot all of the resulting outliers...
Permalink to Comment16. Fishy Fish on July 28, 2011 12:58 PM writes...
If you see one cockroach in a kitchen, you can be sure that you will find more.
I would say that this is not an isolated incident.
Permalink to Comment17. Hap on July 28, 2011 1:05 PM writes...
I am starting to read "Plastic Fantastic", and in the beginning the author argues with the self-correction capability of science. If someone tells you what you expect to hear, you probably don't question it as much - but even if you do, fraudulent data is likely to be pretty consistent (because if someone is telling a story someone else wants to hear, the teller doesn't want anything jarring to disrupt the story and make the listener start looking for problems). If the story is consistent, and you can't rerun the data yourself, it's likely to be believed. I don't know how to combat that.
#14 has a good idea, but I figure that if you're worried about whether your CRO is lying to you, you aren't already hosed.
Permalink to Comment18. TJMC on July 28, 2011 1:17 PM writes...
I agree with Hap that #14 has a good idea for checking on results by including known standards amid the samples. Hard to do when incoming samples are managed by the CRO. However, the lag in time still would cause a problem at huge costs if caught too late.
As per my prev. post, shared databases that the Sponsor owns and requires the CRO to use (uploading daily results), may solve several problems. First as a way to scan for over-consistencies, or masked standards. Second, as a disincentive for fraud. Other reasons include adaptive trials, data mining and getting rid of what six-sigmaites call the "hidden factory" of costs, time, errors and rework.
Permalink to Comment19. Dr. Dad, PhD on July 28, 2011 2:12 PM writes...
Wow. That's all I have to say. WoW.
Permalink to Comment20. CMCguy on July 28, 2011 5:17 PM writes...
#9 johnnyboy is too true that always must be cautious in self deception and "weird and unexpected results are more often the norm than the exception". Reminds me that at times I feel to have been an over participant in the "Chinese curse: May you live in interesting times" and crave for plans to remain more or less intact and results that don't wake me up at night in puzzlement. Similar to comment by #14 TJMC we approach as "trust but verify" which still doesn't mean can not be fooled since as in this case (and typically much criminal activity) it takes someone with insider information to bring light to things.
Permalink to Comment21. Mother Natoor on July 28, 2011 6:58 PM writes...
Well there it is M.B.A. Wharton-ites from my company. You dicated I utilize Cetero against our scientists wishes, as they were the cheapest bid. Thank you and when do I get my severance package?
Permalink to Comment22. Sili on July 28, 2011 9:55 PM writes...
To be fair, Texas is often considered a third world country. Permalink to Comment23. gcp expert on July 28, 2011 11:09 PM writes...
So it's about time FDA exposes this corrupt company and its conniving executives. The power of the media and the FDA won't help Cetero get off with ease. Full blame is on the VPs and Execs! This is their downfall just like the previous companies they corrupted with their mischievous system of lying and hiding. Their rebuttal letter is a joke! But then again they're already in a massive shithole so what else can they do except deny and publicly challenege the government? Real smart, real smart! Their reputation has gone into the negative digits. Bad news for sponsor so it's time to start the litigation process!!
Permalink to Comment24. the regulator on July 28, 2011 11:40 PM writes...
Want fast but corrupt research data? Just google Cetero's locations and contract out your research studies there. THE END.
Permalink to Comment25. Coffee on July 29, 2011 12:18 AM writes...
I like how they say it was all a mistake...but most of their job openings are for auditors.
Permalink to Comment26. Hap on July 29, 2011 1:03 PM writes...
And they're still sticking with the "it was scientists looking to chisel us out of overtime pay" line, even though the fraud was also done during normal working hours.
"Hi, stockholders and customers. Please don't read those stories about us. Our data is good. Trust us."
Sili: More like a 2.5 world country. Their education and justice systems are pretty poor, and they do have Ted Nugent, but they did manage to limit house financing to 80% (and thus limit exposure to the bubble) - of course, it's not like you can know what houses cost there, since Texas is one of the few states where housing sales are considered public record (so only realtors know the housing prices there). If you go to Galveston, the coffee...er, seawater, looks pretty good, though I don't know what the caffeine content is. I'm guessing you can get your RDA of hydrocarbons, though.
Permalink to Comment27. MoMo on July 29, 2011 2:08 PM writes...
Its slow on friday afternoons here so I looked up Cetero's leadership-LOL
They all look like used car salesmen photographed by the guy who took the picture of Jerry Lundergard in "Fargo".
To anybody who used them I ask seriously, Why?
Permalink to Comment28. MoMo on July 29, 2011 2:35 PM writes...
One other important point. How come the companies that used them never thoroughly checked their data? Standard curves? Errors of Internal standards and all the other parameters good analytical chemists use to validate methods.
They are just as negligent and get what they deserve.
Permalink to Comment29. Just Sayin' on July 29, 2011 7:11 PM writes...
I looked up their website too, did some research thru various channels and those Cetero execs have a long rotten trail of mischief and history of misconduct. Maybe the drug sponsors should have done the same... corruptio optimi pessima.
Now they're paying with a magnifying glass over them for many many years to come. Time to stay far far away from this company with huge pile of problems!
Permalink to Comment30. Anonymous on July 30, 2011 4:25 AM writes...
From the "Careers at Cetero Research" Facebook page. It starts " Extraordinary People Work At Cetero Research - Are You One? "
Permalink to Comment31. Bruce Hamilton on July 30, 2011 4:26 AM writes...
From the "Careers at Cetero Research" Facebook page. It starts " Extraordinary People Work At Cetero Research - Are You One? "
Permalink to Comment32. mad on July 30, 2011 9:21 PM writes...
Eventually pharma will realize the obvious...dont outsouce your core business....
Permalink to Comment33. research for all on August 1, 2011 4:25 AM writes...
More importantly pharma companies need to finally realize the obvious - never send your business to the most corrupt CRO, Cetero. Never ever again.
Permalink to Comment34. manbe on August 2, 2011 10:53 AM writes...
well we reach the tipping point where quality is been affected
Permalink to Commentwe want quick and inexpensive clinical/bioanalytical services
sponsor need to understand that at one point this is incompatible
we should never compromise quality for - there is a cost and time for this
35. Interviewed at a CRO on August 3, 2011 12:37 AM writes...
I once interviewed at an Indian CRO where they explicitly asked me how would I convince a client that their data was fine if something had gone wrong with the data generation. I told them I would tell the client there was a mistake on our end and the data needs to be regenerated. I wasn't invited for the followup interview :).
Permalink to Comment