Why doesn't anyone comment on scientific papers? Let's narrow that down: why doesn't anyone comment on them when they have a comments field attached to them on a scientific publisher's web site?
Nature has been wondering about this for a while. In a new item about "Web 2.0" tools in science, they mention:
But deposition in arXiv is about as far as the scientific openness of even astronomers goes. The discussion that ensues is private. As Nature's experiment in open peer review showed (http://go.nature.com/N67mFk), and as can be seen from the lack of commenting on papers in Nature and other journals that encourage it, researchers see little to be gained from open discourse before or after publication. Not only are they busy, as the above quotes attest, but there's no credit to be gained, and some risk if one makes an erroneous or critical statement in public. What is more, astronomers and biologists register active discouragement of blogging — a form of communication that in their eyes carries no stamp of reliability or prestige. That picture of resistance to interactive discussion of science on the Internet is further amplified in a new survey, If You Build It, Will They Come? How Researchers Perceive and Use Web 2.0, to be published later this month by the UK Research Information Network.
Contrast that with a comment left here the other day, where a reader suggested that a great business plan might be to start a web site where people could comment on scientific publications. How to reconcile these world views? I think that one word goes a long way: anonymity. The first reaction to that is often a mental picture of the comments pages to (say) YouTube videos, and a brief shudder. And it's true that comments sections that allow easy anonymous accounts can attract all sorts of nasty stuff, as newspapers found out when they opened up their sites.
But anonymity has a long, distinguished history in science. Peer review! People will speak frankly about a paper under review, as long as they know that their comments will remain untraceable by the manuscript's authors. What makes it acceptable is that the editors of the journal know all the names involved, which (usually) keeps things above the midnight-ninja-assassination level.
So what would be needed, I think, would be a site where real e-mail and contact information would be required. You would be free to post under your real name, if you wanted to, or to take on whatever nom de guerre you wished, with the total assurance this this would not be revealed. At the same time, sheer invective and libel would be tossed out immediately, with the line to be drawn at the discretion of the site's editors. Personally, I'd delete comments that said only "This paper sucked", but I'd leave in the ones that said "This paper sucked because of X, Y, and Z." And the comments that started that way but then went on to talk about how the authors sucked in similar fashion would be truncated with some sort of standard mark meaning "ad hominem deleted".
That would be a lot of work. And there's a good chance that it would never take off at all, given the amount of trust involved. Would I wish to run such a site? No way - I already have a full-time job, thanks. But I'd like to see someone try.