Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« More From the Fourth | Main | Drug Prices in the US: Not So High After All? »

July 6, 2010

Commenting On Scientific Papers: How Come No One Does It?

Email This Entry

Posted by Derek

Why doesn't anyone comment on scientific papers? Let's narrow that down: why doesn't anyone comment on them when they have a comments field attached to them on a scientific publisher's web site?

Nature has been wondering about this for a while. In a new item about "Web 2.0" tools in science, they mention:

But deposition in arXiv is about as far as the scientific openness of even astronomers goes. The discussion that ensues is private. As Nature's experiment in open peer review showed (http://go.nature.com/N67mFk), and as can be seen from the lack of commenting on papers in Nature and other journals that encourage it, researchers see little to be gained from open discourse before or after publication. Not only are they busy, as the above quotes attest, but there's no credit to be gained, and some risk if one makes an erroneous or critical statement in public. What is more, astronomers and biologists register active discouragement of blogging — a form of communication that in their eyes carries no stamp of reliability or prestige. That picture of resistance to interactive discussion of science on the Internet is further amplified in a new survey, If You Build It, Will They Come? How Researchers Perceive and Use Web 2.0, to be published later this month by the UK Research Information Network.

Contrast that with a comment left here the other day, where a reader suggested that a great business plan might be to start a web site where people could comment on scientific publications. How to reconcile these world views? I think that one word goes a long way: anonymity. The first reaction to that is often a mental picture of the comments pages to (say) YouTube videos, and a brief shudder. And it's true that comments sections that allow easy anonymous accounts can attract all sorts of nasty stuff, as newspapers found out when they opened up their sites.

But anonymity has a long, distinguished history in science. Peer review! People will speak frankly about a paper under review, as long as they know that their comments will remain untraceable by the manuscript's authors. What makes it acceptable is that the editors of the journal know all the names involved, which (usually) keeps things above the midnight-ninja-assassination level.

So what would be needed, I think, would be a site where real e-mail and contact information would be required. You would be free to post under your real name, if you wanted to, or to take on whatever nom de guerre you wished, with the total assurance this this would not be revealed. At the same time, sheer invective and libel would be tossed out immediately, with the line to be drawn at the discretion of the site's editors. Personally, I'd delete comments that said only "This paper sucked", but I'd leave in the ones that said "This paper sucked because of X, Y, and Z." And the comments that started that way but then went on to talk about how the authors sucked in similar fashion would be truncated with some sort of standard mark meaning "ad hominem deleted".

That would be a lot of work. And there's a good chance that it would never take off at all, given the amount of trust involved. Would I wish to run such a site? No way - I already have a full-time job, thanks. But I'd like to see someone try.

Comments (30) + TrackBacks (0) | Category: The Scientific Literature


COMMENTS

1. Brian on July 6, 2010 7:59 AM writes...

Anonymous commenting site devoted to neuroscience:

http://thirdreviewer.com/
via:
http://scienceblogs.com/drugmonkey/2010/06/not_getting_the_point_on_the_t.php

I think there may be real potential in this model to undermine a traditional publisher's lock on mindshare to the point of, eventually, accepting manuscripts for "publication"

Permalink to Comment

2. Elizabeth on July 6, 2010 9:09 AM writes...

There's also, at least for me, in the way in which I read papers than in how I read blogs. For blogs, I read at the site and am right there to comment. For papers, I usually pull several PDFs on a topic to read later... at which point I'd have to go back to the journal to leave my thoughts. It's not an excessively high barrier, but it is one.

Permalink to Comment

3. Verpa on July 6, 2010 9:40 AM writes...

As you probably know, there are several resources for commenting on chemistry papers ( two run by Mitch of the Chemistry-Blog.com are: chemfeeds.com and reddit.com/r/chemistry, there is also what I like to call the ACS ghetto: http://pubs.acs.org/JACSbeta/journalclub/ ) but none are likely to take off unless the publishers themselves make a serious attempt to build a community instead of wading out a few feet into the future.

I've never been able to decide if I think the whole 'I'll sound stupid' excuse was a canard. Two things that make me think it is:
1. Everyone sounds stupid occasionally at group meetings ... if you don't, I don't think you're speaking up enough, and that's in front of people reasonably versed in the same subject who you'll be seeing the next day.
2. Who takes things said on the internet quite that seriously...personally I wouldn't judge a person for a stupid comment online, unless it was a string of stupid comments. Adding a means to retract a comment in addition to your anonymity idea would probably work too.

I'm going on a bit here, but one last point, I think the real issue is the game-theory of it. The benefits accrued from a commenting system mainly go to the journal/site and the reviewed, not the commenter. In the current model, reviewers get the prestige of being reviewers. Until we can work out a method that accrues some benefit to the commenter, it won't probably happen.

Maybe profs when hiring grad students / post docs could ask to look at their comments on papers in the group's field ... would benefit the profs ( can see the level of thought in the student ) and the student ( would have another means of standing out when applying ). Ok, that'd be a terrible idea, but you can kind of see my game theory point.

Permalink to Comment

4. John Harrold on July 6, 2010 9:46 AM writes...

Elizabeth,

I think this is a technical hurdle that can be easily overcome. The PDF document structure supports hyperlinks, and the publisher could easily embed a link to the discussion somewhere on the first page. Then after you read, you can scroll to the top, click on the link, and start commenting. It would be difficult to get the publishers on board, but the solution is there.

Permalink to Comment

5. Student on July 6, 2010 9:48 AM writes...

We need more opinions in chemistry. It's not like someone's going to get up in the Q&A session after a talk and say, "Dude, that topic is boring as hell, why are you using up good talent and resources investigating that trivial problem when there are so many important problems out there?" The only questions that are usually asked at seminars are not "big picture" questions, but are more "trivial" and involve data analysis or interpretations of the data. The only time you get honest "big picture" feedback is in grant reviews and anonymous paper peer review, unless you have a really close friends who are comfortable saying such things to you. We need more mechanisms for anonymous, honest opinions.

This is because grad students are trained in how to collect data and analyze results, but get almost no training in how to have good scientific taste. Good taste is a subjective opinionated thing, and since we never express opinions very much, but rather focus on the facts, students get the impression that this aspect is unimportant. But it is REALLY important; indeed, having good taste may be the most important thing for an R1 professor. Consequently, many students leave grad school with no feel for how to select an important problem, or to have good taste in what new problem or field to start, because no one wants to openly express an opinion, which frequently involves criticizing some area of chemistry (which their colleagues might be engaged in) or suggesting that a topic or problem is less important than something else.

Permalink to Comment

6. Nick K on July 6, 2010 9:55 AM writes...

Perhaps chemistry journals could have a "Letters to the Editor" section like newspapers.

Permalink to Comment

7. Andrew on July 6, 2010 11:00 AM writes...

This isn't an uncommon problem in other arenas either. Any forum that is remotely open is likely to attract a lot of useless comments and discourage constructive dialog. Wikipedia is a great example of this, and they try really, really hard to develop just such an environment.

www.daringfireball.net is run by a prominent tech blogger, John Gruber, who does not allow comments on his site. Instead, he posts his own comments on his site and links to the stories or blogs or articles that are of interest to him and any dialog is handled by the respective blogs of the people he is discussing an issue with, even with people that he's not particularly a fan of.

In so doing, no crap collects on his site, and everyone else is free to post and manage their own online persona as they see fit, allowing comments, or not, as appropriate.

The reality is that the forum for all this is already there - the internet itself - just a matter of getting people to take more ownership of their own online presence. I would host my own blog and be far more comfortable with complete control over that content than have to join several different forums and maintain a presence in each with almost zero control just to be able to be part of those communities.

Prominent researchers, students, people who publish papers or other findings should be able to post or publish their findings in places where they can be easily seen, reviewed and discussed by their peers such as online journals and the like, as well as maintain their own blog about their area of expertise where they can post follow-up information and commentary and links to relevant supporting material or discussion on other sites or forums.

Permalink to Comment

8. Cloud on July 6, 2010 11:09 AM writes...

You could try crowdsourcing the moderation, like Slashdot does.

It isn't a perfect system, but it does OK. Most flamebait comments are quickly marked as such.

Permalink to Comment

9. Anonymous on July 6, 2010 11:59 AM writes...

In cancer research, no one comments on anybody else's papers and no one ever says anything critical in anything but a completely private setting between friends.

Most of the cancer research literature (including many/most papers in the most high profile journals) does not represent biological reality and is populated by the same inside group who reviews each others papers and who then write the News and Views type articles on the papers they accept (which they then put on t