Update: Richard Van Noorden at Nature runs the numbers on this paper, and comes to the same conclusions: you can't use it to say that US papers are more prone to fraud. . .
There's a new paper in the Journal of Medical Ethics looking at fraudulent publications. The author went back and studied the papers in the PubMed database that have been retracted during the last ten years (there are 788 of them), looking for trends. And so far, the press coverage that I've seen of his conclusions seems to be missing the point.
One thing that stands out is that retracted papers tend to come from higher-profile journals. That makes sense to me, because that's (a) where they're more likely to be noticed, and (b) where the editors are more likely to care. As you move down the list, people just seem to start shrugging their shoulders. Has a paper ever been retracted from Bioorganic and Medicinal Chemistry Letters, for example? I can't think of any, and readers are invited to try to find one themselves.
Another conclusion is that retracted papers tend to come from serial offenders. Over half the retracted papers had a first author who had retracted something else. I think that probably represents cases where someone had published a body of fraudulent work which all got exposed at once, but if that's not a serial offender, I don't know what is.
But a third conclusion is the one getting the headline writers going. The paper examines the retractions, looking for whether they were withdrawn for error or for fraud. The US is notable for having more retractions in the latter category, as compared to the rest of the world: one third of its 260 retracted papers are attributed to fraud, while the rest of the world comes in between 20% and 25%. So you get things like "US Scientists More Likely To Commit Fraud". But no, for that conclusion to be valid, you'd want to know how many fraudulent papers were published as a percentage of the whole output.
Even that wouldn't tell you the whole story. Remember, we're looking at papers that have actually been retracted. Most published fraudulent research never gets to that point. Lower-end journals have a terrible problem with plagiarized, derivative junk. Piles of it gets sent to them, then too much of it gets into print, and it just sits there, with no one ever paying any attention. Well, years later, some poor person might try to reproduce a prep, find that it doesn't work, sigh, and try something else, but otherwise. . .
No, in the same way that the prominent journals are over-represented, I think that the US might be a bit over-represented because slightly more fraud gets caught. More US papers appear in prominent journals than average, and fewer appear in the absolute bottom-rung journals. The United States accounts for at least one-third of the total scientific paper output, so those 260 papers out of 788 are exactly what you'd expect if all other things were equal. But other things aren't equal. We may have more prominent (and more harmful) frauds here, but I'd be willing to bet that as a proportion of the whole, we have fewer of them.
One last note: the figures in this paper seem to conflict with an earlier analysis, which seems to have found fewer retracted papers in PubMed, and a higher proportion of them tainted by fraud. Who's right?
1. Virgil on November 17, 2010 9:08 AM writes...
Muddying the waters even further is the fine line of the definition between error and fraud. I have seen several retractions with simple statements such as "we are unable to reproduce these findings" - does that mean there was an error, or fraud? Similarly, you see errata such as "we accidentally posted the wrong figure", which usually means we deliberately posted the wrong one, someone called us out on it, so we had to do a proper one - that's an "error" in name but is really more like fraud.
I'd like to see a similar analysis on errata - there are a surprising number of papers which should probably have also been retracted, but which got by with a simple erratum.
Permalink to Comment2. Iridium on November 17, 2010 10:03 AM writes...
That is nice...."half the retracted papers had a first author who had retracted something else"
In my opinion, if it is PROVED that a professor working for a University (where students are supposed to be educated) is CONTINUOSLY "inventing" substantial amounts of data or "copying" results from other research groups..... the professor should be fired.
Permalink to Comment3. darwin on November 17, 2010 10:28 AM writes...
What are people's viewpoint of purposeful misinterpretation of data in industry and its relationship to fraud?
Permalink to Comment4. Matt D on November 17, 2010 10:42 AM writes...
These headline writers are so cynical. Why not "US Scientists Less Likely To Commit Errors"?
Permalink to Comment5. anchor on November 17, 2010 11:06 AM writes...
The big news of the day is that Roche is bleeding jobs here in the US and is doing away with gene silencing research (aka-siRNA).
Permalink to Comment6. Pete on November 17, 2010 11:36 AM writes...
I hope that you're not picking on Bioorg Med Chem Lett as part of your duties as a member of the editorial advisory board of ACS Med Chem Lett which competes for the same niche. How often do articles get retracted from J Med Chem?
Permalink to Comment7. Anonymous on November 17, 2010 12:04 PM writes...
Come on Pete, let's be honest with ourselves. The whole house of El Sevier isn't in the same league as ACS journals. Granted, neither is perfect, but the quality and trustworthiness of the two journals couldn't be more different, and that would be the case whether or not Derek was an editor for one of them. You won't find many people to argue the flipside of his statement and there's good reason for that. The very idea that you would even run a journal that revolves around synthesis of new drug compounds and not require any type of supporting information is ludicrous.
Permalink to Comment8. Pete on November 17, 2010 12:20 PM writes...
Anonymous 8, You have completely missed the point of my commnent which was about potential conflict of interest and not about the quality of journals (or indeed publishers). Can you answer the question about how often articles are retracted from J Med Chem?
Permalink to Comment9. Dennis on November 17, 2010 12:54 PM writes...
Pete: No one needs to pay Derek to pick on BMCL. He's been doing it pro bono for years before ACS Med Chem Lett even existed.
Permalink to Comment10. Wagonwheel on November 17, 2010 1:07 PM writes...
BMCLbashing is common place around these parts and there is no need to go into that one again, but there are still some decent little articles in BMCL. I'm with Pete though, there are some conflict of interest issues...
Permalink to Comment11. anonymous2 on November 17, 2010 1:13 PM writes...
And the problem is that anon1 is correct in that his complaints don't have much refutation in fact. If your journal exists because med chemists don't like to write papers and SI, and have lots of interesting bio but little chance of actually being reproduced, well, you sort of earned your reputation. Having interesting structures and really interesting structures published in your journal without any indication that someone looked at them before publishing just adds the rotting cherry to the top of the fetid sundae.
Permalink to Comment12. Iridium II on November 17, 2010 1:28 PM writes...
"In my opinion, if it is PROVED that a professor working for a University (where students are supposed to be educated) is CONTINUOSLY "inventing" substantial amounts of data or "copying" results from other research groups..... the professor should be fired."
Either that or the university should take no action, impose no sanctions, attempt to persuade the research council to fund him/her, and offer the person in question large cash funds to continue "research".
I guess it's either or.
Permalink to Comment13. Iridium on November 17, 2010 1:50 PM writes...
"I guess it's either or."
I saw it happening..
:-(
It should be advised to students that are looking for an academic career:
"Steal" 20-30 publications so you get founding, you get contracts, you get students...
...so what if after a couple of years you have to retreat 3-5 papers?
You have already safely brought home a salary for many years and for more years to come.
Permalink to Comment14. Danelo on November 18, 2010 12:42 AM writes...
Re: Iridium 10:03 AM
While it might be true that sanctions for committing fraud aren't serious enough, I think what most often happens, as Derek pointed out, is that Prof. Fraudulent publishes, say, 2 papers on Topic X in 2005, 1 in 2006, and 3 in 2007, and then the fraud is discovered in 2010 and the papers are retracted more or less at once.
I think it's rarely if ever the case that Prof. F publishes fraudulent paper(s), gets caught, their institution slaps them on the wrist, they publish more fraudulent paper(s), gets caught, get wrist slap #2, publish more fraudulent papers, etc.
So even if the university did fire Prof. F after the first instance of fraud is uncovered, they would still be a "repeat offender". There's not administration can do up to that point, unless the Provost personally duplicates each experiment in publications from their instituiton. :)
Permalink to Comment15. gradslave on November 18, 2010 12:55 PM writes...
Just wanted to add here, there are no real mechanisms to fight fraud on the very basic lab level. Say if a grad student knows that papers published by his own PI are not exactly reproducible, what can he do? nothing! it will harm him just as much (if not more) than his PI.
Permalink to Comment16. Captain Stern on November 18, 2010 1:05 PM writes...
"US Scientists More Likely To Commit Fraud"
This headline would seem to slam Americans. I take offense.
From my perspective most fraud is perpetrated by the lead author acting alone. Even in the USA most first authors are not American.
This is just a gimmick headline designed to bait the 'ugly american'. Newsworthy? Yawn.
Permalink to Comment17. Tyrosine on November 22, 2010 11:19 PM writes...
"No, in the same way that the prominent journals are over-represented, I think that the US might be a bit over-represented because slightly more fraud gets caught. More US papers appear in prominent journals than average, and fewer appear in the absolute bottom-rung journals."
Interesting bit of tap-dancing there. Another interpretation might be that not only are more US researchers committing scientific fraud, but the US fraudsters are also too dim-witted to bury the work in lower-tier journals where it won't be discovered. :-)
Permalink to Comment