Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« The Economics of the Drug Industry: Big Can't Be Big Enough? | Main | Lundbeck Cutting R&D »

August 11, 2011

Scientific Retractions: A Growth Industry?

Email This Entry

Posted by Derek

The Wall Street Journal has an interesting article based on data from Thomson Reuters on the frequency of retracted papers. It seems to be increasing dramatically:

Since 2001, while the number of papers published in research journals has risen 44%, the number retracted has leapt more than 15-fold, data compiled for The Wall Street Journal by Thomson Reuters reveal.

Just 22 retraction notices appeared in 2001, but 139 in 2006 and 339 last year. Through seven months of this year, there have been 210, according to Thomson Reuters Web of Science, an index of 11,600 peer-reviewed journals world-wide.

They mention Retraction Watch, as well they should. But ten years ago, would there have been enough new material to keep that blog running? Pharmalot has some more from its founder about what might be going on. There are, of course, more journals than ever these days, and many of them are junk. But it's not the bottom-tier journals that are driving this trend, I'd think, since honestly, when does anyone ever retract a paper in one of them? Consistent with that view, this bar chart of PubMed retractions by journal is heavily weighted towards the big dogs. A lousy or nonreproducible paper in one of the top journals is more likely to be of enough interest to attract attention, but one in J. Whatever will just sit there.

No, when you look at this chart, it appears that retractions-per-papers-published have been climbing, so the answer must be some combination of more mistakes, more fraud, or better policing. Retractions due to fraud seem to be where most of the growth is, according to this study, so that takes us down to the latter two explanations.

Software has definitely made the lazier sorts of fraud easier to detect, automatically flagging copy-and-paste hack jobs. But those aren't the kinds of things that show up in the better journals, are they? We may be seeing a mix of greater incentive to commit fraud and a rise in skepticism among readers. There have been enough cases, enough highly-publicized retractions and scandals, that more people may be willing to wonder if some exciting new result is true at all.

That's not a bad thing. The rise in fraud is a bad thing, but a corresponding rise in scrutiny is the only thing that's going to cure it. There are always a few pathological types out there that kind of know that they're going to get caught and kind of don't care. Those we shall always have with us, and not much is going to discourage them. But as for the rest of the fraudsters, the thought that they have a better chance of being found out and punished should give them something to think about.

Comments (15) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature


COMMENTS

1. Student on August 11, 2011 8:45 AM writes...

It would be interesting to see how many reactions are of negative data...I also hope the readers here remember Derick quoting Bruce Booth that something like 50% of experiments data is not reproducible in biotech/startup/industry hands.
The messed up thing is that in this funding environment if I base my hypothesis (as an up and coming or even established academic) on unverified work, I could be chasing a ghost for months, I could be throwing away tens of thousands of dollars...and essentially my career, depending on my funds.

Permalink to Comment

2. PharmaHeretic on August 11, 2011 9:09 AM writes...

A significant minority of biomedical research has been irreproducible for almost two decades.

Permalink to Comment

3. johnnyboy on August 11, 2011 9:43 AM writes...

For the reason behind it, i'd vote for better policing, myself - and generally more awareness that papers actually can be retracted. I remember being at a university seminar about 15 years ago, where a researcher was talking about a paper he had published before, from which he now knew that the results were basically wrong, and about which he said, contritely, "unfortunately, you cannot un-publish articles". Guess what, turns out you can. Not that retractions necessarily involve papers that are wrong (as opposed to fraudulent), but maybe they should ?

Permalink to Comment

4. Cialisize Me on August 11, 2011 10:02 AM writes...

To Student #1:
No, don't worry about wasting 10s of thousands of dollars chasing ghosts (BTW, $100s of thousands or Millions would be the likely scenario).

Ruining your career? Hardly, in fact quite the opposite: Hand waving and esoteric explanations will suffice save your name, and you will have established what they call a "track record" and become a "known quantity". You will be the rare bird that actually received funding for an idea and managed the money. Results are secondary, and you will be on the short list to lead the next venture, since you know the ropes.

Seriously, biotech insiders can name many in this category!! Someone with three failed companies based on bad science and/or bad business models, and later heads up a new startup. So get the money, try your best to make it work, the rest is secondary.
Best regards, C.M.

Permalink to Comment

5. luysii on August 11, 2011 11:05 AM writes...

In biomedicine, the absence of a retraction may not matter. Back in the early 60's there was something seen on electron microscopy of cells (which was just getting started) called the unit membrane. It would now be called the plasma membrane or cell membrane. It showed up as two dark (electron dense) lines surrounding a light (electrons pass through easily) line. Much effort was expended to decide whether the light line was lipid surrounded by proteins, or whether it was protein surrounded by lipid. It was an artifact of staining which was never retracted. Now we know that the cell membrane is an asymmetric lipid bilayer containing proteins stuck to either side or passing through it. The only continuous structure surrounding the cell is the lipid bilayer. Unit membrane papers were never retracted, just ignored.

In medicine this sort of thing is dangerous. The late cardiac surgeon Michael DeBakey was a great man who made great strides in the field, but he was terribly wrong about one thing. He claimed that you could operate on a carotid artery (supplies 70% of the blood going to the brain) which was completely occluded to treat stroke. Everyone else who tried this wound up killing their patients -- as a full head of pressure of blood going into a dead vessel and dead brain resulted in hemorrhage in the brain and death. People stopped doing the procedure after killing a few patients in each institution. To my knowledge his papers were never retracted, and he never admitted he was wrong. Also no one every wrote a paper saying he was wrong as far as I know.

Permalink to Comment

6. NJBiologist on August 11, 2011 11:30 AM writes...

@5 Luysii--There are examples of this in virtually every field I've looked at. I believe science and medicine need to forget some things, and it makes me nervous to see how readily accessible all these old papers are by electronic retrieval.

Permalink to Comment

7. matt on August 11, 2011 12:20 PM writes...

@6 Luysii, @5 NJBiologist:

@NJBiologist specifically: better not to forget, but remember more completely the whole story.

What is missing from the scientific record is the follow-up, right? Some sort of consensus was reached, quietly and humbly in the case of DeBakey's critics, but without a trail.

Seems like there's a need for a blog/journal/forum of follow-up studies, where people chime in with a "yep," "nope," or "still scratching my head."

As a matter of fact, it seems like each journal ought to have a forum, with a sub-forum generated for each article. Comments on the article would get a thread, of course. Priority should be given to threads where researchers are trying to replicate the findings, and subsequently include all those additional tests people thought should have been done in the first place. True, as in the DeBakey example, there will be exposed a great variety of technical proficiency (the "talk radio" effect), but I think the back-and-forth on how to make it work or if it works would improve the science and improve understanding.

Now that I think about it, it seems so natural I'm surprised all the major journals aren't doing it--are they and I'm just ignorant of it?

Permalink to Comment

8. John David Galt on August 11, 2011 12:28 PM writes...

Punished? I wish. The biggest fraudsters have nothing to fear, because the supposedly unbiased institutions that employ them share their political reasons for committing the fraud.

Permalink to Comment

9. luysii on August 11, 2011 12:50 PM writes...

Well, it cuts both ways. In my first year of neurology residency, a paper was published (in a very reputable neurological / neurosurgical journal) saying a particular neurosurgical procedure didn't work. I'd seen some pretty impressive results with it. So I asked the chief about it. His response:

"Oh, I see old bananafingers has written another paper"

Permalink to Comment

10. Chris D. on August 11, 2011 3:54 PM writes...

Matt has a keen idea. I've actually created an independent forum that collects critical evaluations of scientific protocols:

http://www.sciencecheck.org

I'll be working to help it gain traction in future months (my own science keeps me very busy), but I'll definitely be recruiting contributors. I firmly believe that we need to better recognize and reward reproducible science, and flag science that may be problematic. Luysii, your great surgery example is the most egregious I've heard!

Permalink to Comment

11. Anonymous on August 12, 2011 2:19 PM writes...

@7 Matt, @10 Chris D.: My inner pessimist expects a forum like that to have more than its fair share of between-labs sniping... but I'd be happy to be wrong.

Permalink to Comment

12. S.Pelech-Kinexus on August 12, 2011 3:15 PM writes...

With over a million new scientific manuscripts published annually, the actual increase in retracted publications over the last decade is actually pretty inconsequential. It is unlikely that there has been any real increase in scientific fraud for several decades as the pressure to publish for promotion and tenure has not really changed. As pointed out by Derek Lowe, it is a lot easier to detect some kinds of fraud with the Internet and search engines. The best way to confront scientific fraud or deficiencies is for readers of journal articles to be able to add flags or comments at the end of scientific publications if there are issues with post-publication peer-review. Today, this is much easier with electronic rather than printed articles.

Permalink to Comment

13. Anonymous on August 12, 2011 5:16 PM writes...

@4: "Someone with three failed companies based on bad science and/or bad business models, and later heads up a new startup. So get the money, try your best to make it work, the rest is secondary."

How do you get to be a coach in the NBA (or manager in MLB)? Be a coach in the NBA (or manager in MLB). You can drag team after team into the outer darkness, season after season. You can ruin player after player. But you will always find another owner wanting to hire you at an even bigger salary to manage his team because of your experience.

Permalink to Comment

14. Jose Geraldo Brito on August 14, 2011 7:09 PM writes...

The problem is the way science careers are organized around the peer review system. This whole thing was idealized to rationalize paper use, in an era when the sheer cost of printing scientific journals was overwhelming. Now, in the internet age, most of the journals we read have gone electronic. Yet we stick to this system because it supposedly improves science quality.

Is this truly the case? Do educated scientists really need to have some editor/reviewer judge what he/she can or cannot read?

It clearly doesn't prevent bogus reports from reaching "high profile" journals, as we are all sick and tired of seeing. On the contrary, the more the quality of science becomes identified with the reputation of the journal in which it is published, the more we invite this kind of behavior. Sincerely, I don't see an end to it.

Permalink to Comment

15. Loon E Toon on August 15, 2011 5:25 PM writes...

In the past, most of the 'retractions' were done informally, in conferences, esp, at the bar after the presenters were done. People would complain about being unable to reproduce certain experiments and if enough people had problems, it would be obvious which ones were, um, a bit beyond the ability of current science. Probably still occurs, but I haven't been at that scene for awhile.

Also keep in mind the pressures the researchers are under. A friend of mine spent 5 years developing a new catalyst system that had the potential to improve an aspect of wastewater treatment. After a year of characterization, he presented his work in a thesis defence. Although he obscured it as much as possible, the new catalyst was less efficient than the base catalyst that he had modified. One of the professors on the committee wanted to deny him his PhD and have him start over. My friend got his PhD, but it involved some arm twisting and some bribery involving TA positions between his advisor and the other professor. After many years of work, how many graduate students or postdocs will just write off several years of work (or possibly their career in the field) if a few data points do not support their hypothesis? I continued the work of two other people (who had graduated) to form my thesis; for one of them one of the controls proved unreproducible, which may have been too unimportant to doublecheck. The other person's work proved to be completely irreproducible. I asked some undergrads I was working with to repeat the work; they got the same data I did.

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
One and Done
The Latest Protein-Protein Compounds
Professor Fukuyama's Solvent Peaks
Novartis Gets Out of RNAi
Total Synthesis in Flow
Sweet Reason Lands On Its Face
More on the Science Chemogenomic Signatures Paper
Biology Maybe Right, Chemistry Ridiculously Wrong