Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« SRT1720: Good (And Confusing) News for Obese Mice | Main | Disappearing Information, Courtesy of Aldrich Chemical »

August 24, 2011

What Are Impact Factors Doing to Chinese Science?

Email This Entry

Posted by Derek

Here's a useful (and rather brave) editorial from Chinese chemist Nai-Xing Wang in Nature. He's pointing out that the government's funding agencies are taking a crude and harmful approach to who gets research support: publish, and publish according to their schemes, or flippin' well perish:

The biggest problem remains the obsession with journal impact factors. Generally speaking, articles in journals with high impact factors are judged to appeal most to readers, but not every paper published in a high-impact-factor journal is high quality, and papers published in lower-ranked journals are never worthless. Yet some administrators in China take a very crude approach: high-impact-factor publications mean excellent work.

Research proposals are judged according to the impact factor of a scientist's previous publications. (And referees are usually selected on these criteria too.) Worse, the salaries of my chemistry colleagues go up or down depending on a complex mathematical formula based on the impact factor of the journals in which we publish our work — which we must supply in a detailed list.

Now, this sort of thing has been going on around the scientific world for a while, and it's going to be hard to put a complete stop to it. But the Chinese system that's described is about the most blatant that I've come across. The effects are pernicious:

If a high impact factor is the only goal of chemistry research, then chemistry is no longer science. It is changed to a field of fame and game. There are other effects too. Administrators in almost every university and research institute like to evaluate researchers by their papers at the end of each year. As a result, chemists often choose easy research topics that can be written up inside a year. . .

You get what you subsidize; I don't think that law is ever broken. And if the Chinese government wants people to crank out lots of papers in what they feel are high-end journals, well, that's what people will do. But if they want something useful to come out of all that effort, well, things might need to be adjusted a bit. But "useful" is a slippery word. For the people who are gainfully employed in keeping the current system running, it's just about as useful as it can be the way it is.

Comments (26) + TrackBacks (0) | Category: The Scientific Literature


COMMENTS

1. Another student on August 24, 2011 9:53 AM writes...

Would it be a stretch to compare this to teaching kids to take standardized tests? You don't learn the material but you learn the game. I can actually see this leading to some nasty scientific-politics and the formation of unspoken alliances to generate the maximum benefit.

Permalink to Comment

2. CMCguy on August 24, 2011 10:08 AM writes...

Gosh I only thought the Chinese had sent their Spies to the infiltrate the labs and technical areas but apparently there must have been a few in Pharma Management to learn the concepts of "research by the numbers" and "only count blockbusters". Although not really sure the NIH "old boys network" of funding distribution has done much to accelerate advancements either. Problems occur when want to avoid or over control risks when it's taking risks that are most likely provide true creative innovation.

Permalink to Comment

3. anchor on August 24, 2011 10:10 AM writes...


Once corrupt..always corrupt! Ms. Nai-Xing Wang's open expression is very brave and we all hope that when all is said and done, she still can function effectively and still be a productive chemist.

Permalink to Comment

4. MTK on August 24, 2011 10:15 AM writes...

I'm not sure how brave it is.

She got something published in Nature; a journal with a high impact score.

It'll look good for her next performance review.

Permalink to Comment

5. NoDrugsNoJobs on August 24, 2011 10:16 AM writes...

I reckon they are being explicit about what is already implicitly true here - Makes them more upfront about what the criteria are rather than obfuscating

Permalink to Comment

6. Ash on August 24, 2011 10:24 AM writes...

Seems to be consistent with the rumor that Chinese scientists are promised tens of thousands (and sometimes a hundred thousand) dollars if they publish in Nature or Science.

Permalink to Comment

7. Anatoly on August 24, 2011 10:34 AM writes...

Well, as long as Journal's Impact Number is based on citations by other scientists, and citations are honest, the system should work.

But I wonder if Nature only publishes articles that cite other articles from Nature. Or that perhaps the high impact journals form a cartel where they only publish those who cite from a pool of their own articles...

Permalink to Comment

8. Innovorich on August 24, 2011 10:42 AM writes...

Errrm - isn't this how it works in the US too??

Permalink to Comment

9. RB Woodweird on August 24, 2011 10:57 AM writes...

"You get what you subsidize; I don't think that law is ever broken."

It is a restatement of the First Law of Metridynamics: The observed metric will improve.

Permalink to Comment

10. MTK on August 24, 2011 11:39 AM writes...

Well, the whole thing is gaming isn't it.

The journals game the system to get as high an impact score as possible. We game the system to get our papers into high scoring journals which leads to better grant reviews, etc., etc.

I guess that's what happens anytime you have a bureaucracy. And by that I mean any organization over five people.

Permalink to Comment

11. David Formerly Known as a Chemist on August 24, 2011 12:00 PM writes...

This isn't at all shocking. When someone knows what factor is being measured, that factor improves. Remember 10 years ago, when pharma started measuring the productivity of chemists based on the number of new compounds they registered? We saw an explosion of new compounds (aided by combichem and parallel synthesis), but how much value was provided by those new compounds? Very little, if the flushing and pruning of corporate collections over the past 5 years are any indication.

Permalink to Comment

12. simpl on August 24, 2011 12:00 PM writes...

The people working at our chemical development and production plant in China have impressed me (I have no contact with the research team, alas). Maybe they see interesting and useful work as an viable alternative?

Permalink to Comment

13. Henning Makholm on August 24, 2011 12:01 PM writes...

@Innovorich, it's how it works everywhere. The remarkable thing is that it's being said _in China_, since the Chinese government is known for being somewhat more ... enthusiastic ... about stifling dissent than many other governments.

(Which is to be understood relatively: if you criticize a bad policy, those responsible for that policy are always going to find a way to get back at you, no matter where you do it. The difference is that in an authoritarian system there are going to be people in power who _despite_ not being personally threatened by your criticism but will join in the getting-back on general principles, because the system views any dissent as inherently suspect).

Permalink to Comment

14. RD on August 24, 2011 12:21 PM writes...

Welcome to America!
And you thought we would have to compete against the Chinese. Now even they can indulge in petty, research killing politics, corralling resources for their next publications and kissing the asses of the people two levels above them while stabbing the backs of their colleagues.
The values of the ultracompetitive business class trickle down once again. It's all hierarchy, status and conformity.
What we need is a good plague.

Permalink to Comment

15. MoMo on August 24, 2011 1:46 PM writes...

Impact factors of journals are useless tripe. They should be studying the citation analysis and number of times cited and whether the paper generates a scientific front via cluster analysis techniques- pioneered by Eugene Garfield at ISI now Thompson.

That is how you know science is valuable, and the smart people use this. This keeps the Chinese in the dumb-ass category relegated to others like those Pakistani publishers scientists love to get their names attached to.

China, if you are listening, now is the time to avoid mistakes aand create the uber-master scientific race!

But first, who wants to hear a story why you shouldnt outsource to China? Its a double knee-slapper!

Permalink to Comment

16. Anonymous on August 24, 2011 2:25 PM writes...

Henning Makholm, we should begin placing bets on whether Wang will "mysteriously" disappear.

Permalink to Comment

17. Pete on August 24, 2011 4:56 PM writes...

Could impact factor be a Trojan Horse? On the subject of useless metrics, was there any mention of h-indices? Perhaps the journals could find a way to take a percentage of the prize money.

Permalink to Comment

18. K. Marx on August 24, 2011 5:16 PM writes...

Pure Capitalism!!!

Permalink to Comment

19. Jimmy on August 24, 2011 8:10 PM writes...

Regarding the "useful" research comment, the number of patents coming out of China shows that that huge amount of research isn't resulting in commercial gain. That said it may be more of a priority to prove that China has "good" research, somewhat akin to the cold war Olympics.

Permalink to Comment

20. anonymous on August 24, 2011 10:08 PM writes...

this equal that.... how much time have you search for a publication and finding a dark chinese journal giving info on what you want to do...

you try it, thinking : hey it's been published' and finally another turd at the bottom of the line...

yeah! right! there's something fishy when there is too much vowels in the journal name :p

Permalink to Comment

21. Ich Dich on August 25, 2011 1:39 AM writes...

This reminds me of the expedition of the Nobel committee to China (I think it was 2006, but I could be wrong) where they had to explain how Chinese could win Nobel prizes.
What would the financial reward be if you would win a Nobel prize in China? Perhaps higher than the actual prize...

Permalink to Comment

22. Anonymous on August 25, 2011 7:27 AM writes...

So does anyone have an idea of a surrogate measure of scientific quality? The mentioned h-factor doesn't seem to get popular for some reason, does it?

Permalink to Comment

23. Artmisinin on August 25, 2011 8:41 AM writes...

Last year, two biologists criticized China's funding system in Science (Y Shi and Y Rao, Science, 2010, 329, 1128 (DOI: 10.1126/science.1196916). I totally agree with them that the problem is about the system and culture.

China does not have a working peer review system to ensure merit based funding. To make the funding mechanism relatively fair, there must be something that can measure applicants' performance by numbers. Otherwise, things will be more ugly.

Another similar situation is China's college entrance examination - all the universities pick students according to their exam scores, nothing else. Why? Just try to leave no room for foul plays.

You American scientists should feel good about your system. It is not perfect, but it is working in a relatively efficient way to promote innovation. We see so many Chinese talents achieved much more here in the States than their peers in China. It certainly is not because that they are smarter or they work harder...

Permalink to Comment

24. gyges on August 25, 2011 12:00 PM writes...

"the salaries of my chemistry colleagues go up or down depending on a complex mathematical formula based on the impact factor of the journals in which we publish our work "

I've seen post-doc positions advertised in the UK where one of the selection criteria to be used is the number of publications that each of the applicants have.

It looks as though we're going down this Road to Serfdom too.

Permalink to Comment

25. CR on August 25, 2011 12:07 PM writes...

@Anon, #20:
"yeah! right! there's something fishy when there is too much vowels in the journal name :p"

Then I guess Tetrahedron Letters has too many vowels.

Permalink to Comment

26. loupgarous on January 31, 2012 10:54 PM writes...

The Chinese are putting a rod in pickle for their own backs by making publication impact the overriding metric for evaluating the success or failure of individual researchers. They're going to have a whole culture built around prestige and success in publications, without the necessary freedom to question published results of prestigious authors - no real audit.

In "Carnage and Culture," historian Victor Davis Hanson observes that while ancient Chinese culture outstripped Western culture at the very beginning of the Enlightenment, the evolving process of audit caused the nations of the West to be able to prevail militarily and commercially over China and the other great Eastern cultures.

If Chinese scholarship comes to emulate Chinese crony capitalism, they can publish all they wish, with no audit process to give the scandals with which Western scholars are all too familiar - and at the end of the day truth will be the main casualty.

Finding the truth is st times a messy process - you get Baltimore scandals, Duke biomarker scandals, all sorts of unpleasantness. Perhaps the main determinant of a culture's success is its willingness to tolerate the mess which leads to the truth.

A face-saving culture seems to be handicapped in this way; if it can't tolerate Kuhnian paradigm changes and exposure of academic misconduct, then it will proceed with dignity along the path of error while more flexible cultures cleave closer to the truth.

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
One and Done
The Latest Protein-Protein Compounds
Professor Fukuyama's Solvent Peaks
Novartis Gets Out of RNAi
Total Synthesis in Flow
Sweet Reason Lands On Its Face
More on the Science Chemogenomic Signatures Paper
Biology Maybe Right, Chemistry Ridiculously Wrong