It's taken a while, but a traditional science publisher is starting to make the leap to blog-style comments for scientific papers. Nature has begun offering authors the option of having their paper commented upon by the teeming masses of researchers while it's still in the review phase. (Tyler Cowen speculated on the pluses and minuses of this idea earlier this year). (Update: and does again here in response to an article on this very experiment).
Nature's been talking about peer review and its evolution for a while now - see this section and this blog at their site. They seem to have decided that debate is all very well, but that we're not going to know how well these ideas work until we put them into practice, thus their peer review trial site.
So, how's it working? Looking it over, I can see (as of today) ten current papers that are available for comments, posted roughly since the beginning of this month. None have attracted any comments at all, which is a situation that many bloggers will be all too familiar with. The site, though powered by Movable Type, doesn't seem to have date-driven archive pages as such, although it does have categories. Looking at the "Recent Comments" sidebar, though, will take you back to the last paper that attracted some, which was posted on August 29th. The navigation links at the top of its page will then take you back, paper by paper.
Digging through the stack in this manner, the only papers with substantial comments are found here, here, here, here, and especially here. That takes us back to early July, and the first papers seem to have appeared about a month before. The great majority of papers have attracted no comments at all - I wonder what sort of traffic the site is getting?
It's interesting to compare the behavior patterns there with those at a regular blog. There are a few "nice paper!" one-liners, which out here in the rest of the world are the sign of spam, but which appear to be sincere (if not very useful) communications on the Nature site. The comments are moderated, and I'd like to know just how many they've had to excise. I ask because there are still some off-topic oddities that make it through, like this, where a Chinese researcher makes a rambling complaint of harassment by his government. And there's a planetary science paper here, two of whose five comments are clearly by loons, which must please the original authors no end.
Where real comments appear, they're often done in the style of a peer reviewer, starting with the obligatory "The authors present interesting data on the. . ." type of sentence, and going on in rather stilted fashion. It seems clear that very few of the commentors have much exposure to the regular blog world, or if they do, they're taking great pains to not allow any of that experience to leak over into the exalted world of Nature.
I wish them luck with the experiment. What I'd like to see is an idea that's been proposed before, but never implemented in chemistry: comments on papers after they've published. Think of how interesting Organic Letters would be with comments after each paper in the table of contents - heck, I'd go all out and put a hit counter on each paper, so you could see what's getting attention and what isn't. Does the ACS have the nerve?