Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Crowded Proteins | Main | Sticky Containers, Vanishing Drugs »

August 28, 2008

PNAS: Read It, or Not?

Email This Entry

Posted by Derek

You know, every time I point out a paper from PNAS, there are always a few comments to the effect of "Why do you bother reading that garbage heap, anyway?" Since I keep citing papers from the journal, it's obvious that I disagree, but I suppose I should take a minute to explain why.

The reason people are down on PNAS is the way that members of the National Academy can, if they choose, sort of jam things into the journal through a side entrance. Here are all the details. The unusual thing about the journal is the existence of "Track I". Basically, a member of the NAS can publish up to four of their own papers per year. Each of these have to be submitted with the comments of two qualified referees, but the author gets to pick them. So a reasonable member should be able to get any sort of interesting or at least non-insane paper in there, by judicious choice of colleagues for review. Members can also pass along up to two papers a year by others in their field, with a similar review process (Track III). Some NAS members take full advantage of these privileges, and some hardly ever do, even (so I'm told, in some cases) for their own papers.

It's a lot less rigorous than the open (Track II) submissions, that's for sure. For those, you're supposed to name three editorial board members, three NAS members, and five external referees, and the editorial board can still do whatever it wants with your paper or with the lists you've sent. (To be sure, they can also reject those direct-submission papers from members, although no figures are available on how often that happens). Two thirds of the Track II submissions are rejected before being sent out for review at all.

But hold on: according to the journal, 80% of the submissions are via Track II, but those make up only 40% of the published contents. Doing the math, that means that the most of Track I and Track III submissions have to get in. Assume 80 Track II manuscripts and 20 of the others. Rejecting two-thirds of the first group will give you about 27 papers to send out for review. If you've accepted all 20 of the others, that means that about half of those 27 will have to get canned during the later review process, to make that 40/60 proportion come out right. So the overall acceptance rate for open submissions has to be, at most, 16%.

But if you ditch some of the 20 member-track papers, you have to come down even harder on the open submissions, of course. If you only (only!) take 75% of the member submissions, that gives you 15 manuscripts. Now you have to reject not half of the open submission papers that made the first cut, but 63% of them, to knock it down to ten published Track II submissions. So with an acceptance rate of 75% for member submissions, it has to be about 12% for everyone else. And so on.

So much for the numbers - it's clear that NAS members must put a lot of things of their own (or from their friends) into PNAS. The real question is: what does this do to the quality of the journal? As far as I can see, it's still a very interesting read, and definitely cannot be safely ignored. And the publication routes are out on the table: if you want to keep score and adjust your perceptions accordingly, the Track I papers are identified as "Contributed by" the member, and the Track IIIs are "Communicated by". I think, myself, that the advantage of letting members publish unusual or possibly controversial work outweighs the temptation to fill the journal with junk.

Comments (24) + TrackBacks (0) | Category: The Scientific Literature


COMMENTS

1. Hap on August 28, 2008 9:15 AM writes...

I would figure that if enough bad papers get through, PNAS's review policies would be modified to change that. If no one reads the journal or cares about it, the NAS is weakened a little, and one would think they would try to avoid that.

Knowing which papers are which is also helpful - supposedly stringent review policies don't help if they can be circumvented or ignored (ACIEE?). If the type of review a paper receives can be readily assessed by readers, they can also easily decide whether the paper is worth reading or following. The lack of ability to know the subsequent reproducibility of results (unless the paper is retracted or corrected) affects all journals, and makes it hard to determine if Type I or III review yields less reproducible papers than Type II review, or the review processes of other journals.

Permalink to Comment

2. Bob Hawkins on August 28, 2008 11:08 AM writes...

You have to figure that the "Proceedings of" a given body will reflect the prejudices of that body and its members. Otherwise it would be just another journal.

As long as the reader can identify which track a paper traveled, fine. Let a hundred flowers bloom.

Permalink to Comment

3. qetzal on August 28, 2008 11:36 AM writes...

You can tell which papers are which. Track I papers say "Contributed by ..." directly below the author affiliations on the first page. Track III papers say "Communicated by ..." instead. Track II papers say "This article is a PNAS Direct Submission" in the footnotes.

Of course, you have to have the full paper to know that. I don't think you can tell just by looking at PubMed or Medline.

I think PNAS is a worthwhile journal. Even if NAS members can get things published with much less rigorous review, they are still NAS members. In theory, they have a track record of top quality research in their field.

Besides, the normal peer review process doesn't always do such a stellar job screening out the crap.

Permalink to Comment

4. Alan Lehman on August 28, 2008 12:54 PM writes...

There ought to be a fast way to get hard numbers regarding the Track I, II and III submissions. Sadly my first attempt to do it seems to have produced ambiguous results. Using the online PNAS search engine, I limited my search to 2007 and then searched for the phrase

For Track I, I searched for "Contributed by" which got 811 hits.
"This article is a PNAS Direct Submission" got 1941 hits and
for Track III, I searched for "Communicated by" which got 746 hits for a total of 3498.

I am in no way implying that these numbers are valid especially since searching for "www.pnas.org" (which ought to be in every publication) only got 2596 hits and by my back of the hand calculations, PNAS publishes 85 papers a week or about 4500 papers per year. I am certain, however, that better minds than my own could come up with a more authoritative set of search strings and get the actual numbers.

Permalink to Comment

5. Chrispy on August 28, 2008 2:39 PM writes...


PNAS is one of my very favorites, just precisely because there is some crazy stuff which gets in there. Anyone see the article on cattle and deer aligning themselves with magnetic north in the last issue? I'm not saying that it is wrong, mind you, but it IS pretty crazy. And I certainly agree with qetzal that the normal review process doesn't do such a great job of screening out the junk. Working in early research at a large company has required frequent duplication of published results, and all I can say is just because you see it in a top flight journal: 'tain't necessarily so!

Permalink to Comment

6. Jose on August 28, 2008 5:01 PM writes...

Along the same lines, an Early View:

The Effectiveness of the Peer Review Process: Inter-Referee Agreement and Predictive Validity of Manuscript Refereeing at Angewandte Chemie

10.1002/anie.200800513

Permalink to Comment

7. German on August 28, 2008 10:23 PM writes...

Hi Derek, thank you for the blog - I am working on making a submission to PNAS, and this was very useful. However, I think you may have switched the numbers of the tracks: Track I is "Communicated by" and Track III is "Contributed by", not the other way around.


Permalink to Comment

8. Still Scared of Dinosaurs on August 29, 2008 6:18 AM writes...

Now that the intelligent posters have chimed in I feel less guilty about pointing out my complaint with the journal - PNAS has to be a leading candidate for worst journal initials ever.

Yes, it's pathetically sophomoric, but every PhD (almost) was once a sophomore.

Permalink to Comment

9. chris on August 29, 2008 6:23 AM writes...

re: TI/III rejections. I have only ever heard of one case of a "Contributed by" article being rejected from the journal, which caused a bit of chitter on the grapevine. I was highly amused.

Not that I hobnob with NAS members often enough to be an authority on rates.

Permalink to Comment

10. john.spevacek@aspenresearch.com on August 29, 2008 7:31 AM writes...

But also keep in mind that all the research in PNAS is freely available after 6 months. And some of course is also available immediately. I wish more journals would take this approach to open access.

Permalink to Comment

11. chris on August 29, 2008 11:41 AM writes...


Passed Over By Nature and Science (PNAS) (And this is exactly what happened in my last PNAS paper....).

Permalink to Comment

12. Anonymous BMS Researcher on August 29, 2008 2:04 PM writes...

PLOS ONE

Permalink to Comment

13. Jordan on August 31, 2008 6:33 AM writes...

We used to say it stands for "Papers Not Accepted in Science". Similar to #11. There's some good work in there, though.

Permalink to Comment

14. Morten on September 1, 2008 12:08 PM writes...

Love the fact that their articles are free after 6 months... hate that you don't have to deposit data from xray structures. Considering the kind of fraud and failure going on recently with regard to crystallography papers, a policy of not having to release structure factors is just wrong.

Permalink to Comment

15. Marc D on September 30, 2008 6:48 PM writes...

Interesting thread… If this is of any use, using impact factor data you can see that PNAS published 3306 articles in 2006, 3200 in 2005, and 3084 in 2004. I don’t have data for 2007, but 3200-3400 should be a good estimate.

Wasting a bit more time and looking at citation reports for 2006, as of today the top paper was cited 301 times, #2 was cited 139 times, and the next 10 all about 100-120 times. The paper ranked 330 (or ranked at 10 percentile) was cited 34 times. 419 papers were cited twice or less. The latter may be used to calculate a novel (?) junk index …

Wasting even more time, there are quite a few authors who published 4-8 papers in PNAS in 2006 and I rapidly checked their citations and a bunch were never cited (not even a self citation…duh).

I published a paper in PNAS in 2003 and this was one of the best experience ever. I had numerous journalists call me (including one from Science, which had turned down the paper!) and I will try to publish there again.

Permalink to Comment

16. Chris Neale on December 8, 2008 11:01 PM writes...

If it's a "proceedings" then let them send it out only to their members as a "newsletter", which is exactly what it is. While PNAS is at it, why not tell everybody about how to enlarge their **nis.

Chris.

Permalink to Comment

17. Xuebing Wu on February 21, 2009 9:52 PM writes...

one way to assess the quality of track i/iii papers is to check their citations (such as ISI/google scholar) and compare with those of track ii. has anyone done this before? I'm really interested in it.

Permalink to Comment

18. Xuebing Wu on February 21, 2009 10:02 PM writes...

oh, I find such a paper, here is the link http://biology.plosjournals.org/archive/1545-7885/4/5/pdf/10.1371_journal.pbio.0040157-L.pdf

They show that track ii paper indeed are much more likely to get more citations, especially after a relative long time. Interesting.

Permalink to Comment

19. kevin on July 2, 2009 3:00 PM writes...

i published in PNAS. it was a "communicated by". but i have to say the reviews were quite rigorous. and yes, my paper was reviewed and rejected by nature, science, and nat med but i find that the reviewers from PNAS were very good and improved my paper..

Permalink to Comment

20. Brian on August 1, 2009 2:17 AM writes...

I think lots of people who criticize track1 and track3 in PNAS editorial process are usually those who have never had a chance to publish their paper in a multidisciplinary journal. Meanwhile, it is hard fact that (1) PNAS is a multidisciplinary journal; (2) multidisciplinary journal has a higher impact factor than speciality journal and greater prestige; (3) the majority of the journals is "speciality journal".

Permalink to Comment

21. Giovanni on May 12, 2011 3:26 AM writes...

Considering that PNAS is publishing manuscript in mathematics geology and Physics (which are less cited in average than biomedicine) an impact factor of about 10 tell us a lot about the average interest raised by PNAS manuscript. Obviously direct submission is a very tough track with low acceptance rate. It is true that for a NAS member life is much easier on this journal but it very difficult to become a NAS member! I do not think that papers communicated by NAS member are overall of lower quality of lower quality than direct sumbmission. It will be interesting to compare citations of the two group of manuscript to test if this is the case

Permalink to Comment

22. Igor on December 19, 2011 7:47 PM writes...

PNAS eliminated Communicated submissions in July 2010. I find the following quiet interesting: "Academy members continue to make the final decision on all PNAS papers, unlike the process in place at such journals as Nature, Cell, and Science, where final editorial decisions are often made by staff rather than practicing researchers". http://www.pnas.org/content/106/37/15518.full

Permalink to Comment

23. Ross Boucek on June 25, 2013 9:45 PM writes...

Hi all, great information! I have learned alot!

Myself and others have just submitted a paper to PNAS for the first time. The paper we feel is best fit for this journal. It has been with the editors for 15 days now. Can anyone who has experience publishing with PNAS provide insight to how long papers usually sit with editors?

PNAS says 18 days on average papers are with editors, but I have read rejections without review are usually sent within a week.

Needless to say we are biting our nails waiting to hear from them.

Best!

Permalink to Comment

24. Ross Boucek on June 25, 2013 9:45 PM writes...

Hi all, great information! I have learned alot!

Myself and others have just submitted a paper to PNAS for the first time. The paper we feel is best fit for this journal. It has been with the editors for 15 days now. Can anyone who has experience publishing with PNAS provide insight to how long papers usually sit with editors?

PNAS says 18 days on average papers are with editors, but I have read rejections without review are usually sent within a week.

Needless to say we are biting our nails waiting to hear from them.

Best!

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
The Early FDA
Drug Repurposing
The Smallest Drugs
Life Is Too Short For Some Journal Feeds
A New Look at Phenotypic Screening
Small Molecules - Really, Really Small
InterMune Bought
Citable Garbage