About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily

In the Pipeline

Category Archives

« The Dark Side | The Scientific Literature | Things I Won't Work With »

July 21, 2015

Publication Bias

Email This Entry

Posted by Derek

Replication of scientific results is a big deal these days, as anyone following the scientific literature is aware. Actually, you don't have to even be doing that - plenty of reports have made their way into the lay press about trouble with reproducibility. There are a number of efforts underway to both reproduce published research and to estimate how large the problem really is.

But there's a new paper that suggests that plain replication won't fix the underlying defect: publication bias. As long as you have to show positive results to have a good chance of getting your paper published, the literature will be skewed. And unless the replication studies have statistical power better than the original work, they're likely to just cloud up the issue even more. Replication studies will have publication bias, too - at the moment, they're hot, but that won't always be the case.

I agree with the authors that systematic publication bias is a big threat to scientific research. Let's start off down at the retail level, experiment by experiment. I think that if most of us (both in industry and academia) look back on our work, we'd find that the majority of the experiments we've done in our careers have never been published anywhere at all. I'm sure that's true in my case. To be sure, many of them aren't of much interest, but their value is non-zero, too.

Imagine a world - not our own, for sure - where every chemistry notebook is tied to some central, searchable repository of data. Those of us who work in the drug industry already experience a tiny bit of this, with electronic notebook systems. We can indeed search every experiment that someone in the company has committed to a notebook (and you'd better be committing them all to the notebook, if you know what's good for you). A hypothetical worldwide notebook infrastructure would be something to see; the number of compounds in it would be many, many times those found in Chemical Abstracts. A lot more chemistry has been done than anyone knows about.

But as you go up the scale from "Hey, I made this compound once" or "Hey, I tried this reaction, and that time it didn't seem to work", publication bias becomes even more of a killer. People decry (and rightly) the way that drug companies may decline to publish negative results on their own experimental compounds (although keep in mind, clinical trials do fail, publicly, and the requirement to register them is a big step in getting rid of this problem). But professors decline to publish things, too, even though the effect on public health isn't so potentially large. It could be worth knowing that Professor Y's group tried to find a stereoselective way to make Cycloaddition X work with better stereochemistry, and failed. But no one ever will - the time and effort spent by Professor Y to write up those results will almost certainly be wasted, because no one will publish the paper, and it would be perceived as doing no credit to the group even if it were. There are only so many hours in a day, particularly when it's grant-renewal or tenure-decision time.

Such bias really starts to hurt for the bigger results and the claimed breakthroughs, and that's what most people are thinking about when they think about a reproducibility problem. The advice given in that Retraction Watch post is sound: only conduct studies that are well-powered, statistically. It's harder, longer, and more expensive to do it that way. But your chances of producing something that can be believed in are far higher. The problem is, too many people are more concerned with producing something that can make a big splash on everyone's list of publications. . .

Comments (23) + TrackBacks (0) | Category: The Scientific Literature

June 23, 2015

Not 25%, But Still Not Good

Email This Entry

Posted by Derek

Here's a disturbing read for you: the author of this paper (Morten Oksvold, of Oslo University) sat down and did what none of us ever do. He chose three different journals in the oncology field, picked one hundred and twenty papers, at random, from their recent issues, and carefully looked every one of them over for duplications in the figures and data. On PubPeer, you can see what he found. Nearly a quarter of the papers had problems.

In case you're wondering, this proportion didn't vary significantly between the three journals, which were chosen at three different levels of prominence (as measured by impact factor). Time, chance, and figure duplication happeneth to them all. I should note that the duplication comes in several different flavors. The least concerning is the appearance of the same control experiments in more than one figure in a paper. One might wish for the controls to be run more than once - in fact, I'd most definitely wish for that - but the authors are not necessarily implying that these are separate experiments. (They're not dispelling any impression that they are separate, either). When the same control experiments (same gels) appear in more than one paper, that seems to be a further step down. The objects of the two papers are (presumably!) different, and there's even more reason to assume that the authors have, in fact, run this again and aren't just reusing the same control that looked so good that time. That's the problem - when you do this sort of thing, it makes a person wonder if there was only that one time.

There are plenty of less ambiguous cases, unfortunately. About half the cases are supposed to be from different experiments entirely. In both gel and microscope images, you can find examples of the same image representing what should be different things, and excuses run out at this point. It goes on. Oksvold then contacted the authors of all twenty-nine problematic papers to ask them about what he'd found. And simultaneously, he wrote the editorial staffs of all three journals, with the same information. What came of all this work? Well, "only 1 out of 29 cases were apparently clarified by the authors, although no supporting data was supplied", and he got no reply at all from any of the journal editors. Nice going, International Journal of Oncology, Oncogene, and Cancer Cell.

My take on all this is that this is a valuable study, with some limitations that haven't been appreciated by everyone commenting on it. Earlier this year, when the material started appearing on PubPeer, there were statements flying around that "25% of all recent cancer papers are non-reproducible!" This work doesn't show that. What it shows is that 25% of recent cancer papers appear to have duplicated figures in them (not that that's a good thing). But, as mentioned, at least half the examples are duplicated controls - correctly labeled, but reused. Even the nastier cases don't necessarily make the paper unreproducible. You'd have to dig into them and see how many of them affected the main conclusions. I'd guess that the majority of them do, but they don't have to - people can also cut corners in the scaffolding, just to get everything together and get the paper out the door. I am not defending that practice, but I don't want this study to be misinterpreted. It's worrisome enough as it is, without any enhancement.

I think what can be said, then, is that "25% of recent cancer papers have duplicated figures in them, which matter in some cases much more than others, since they appear for reasons ranging from expedience to apparent fakery". Not as catchy, admittedly, but still worth paying attention to. (More from Neuroskeptic here).

Comments (44) + TrackBacks (0) | Category: The Scientific Literature

June 22, 2015

A Retraction, Ten Years Later

Email This Entry

Posted by Derek

Here's an odd thing, noted by a reader of this site. Organic Letters has a retraction of a paper in the Baldwin group at Oxford, "Biomimetic Synthesis of Himbacine".

This Letter has been retracted, as it was found that (a) spectra of the linear precursor, compound 14, differed when its synthesis was repeated and (b) spectra published for several compounds resulting from compound 14 (compounds 3, 4, and 20) were scanned from other papers.

Those other papers are the ones from the Chackalamannil et al. synthesis of himbacine, which took someone a fair amount of nerve. I will assume that Jack Baldwin did not scan in the spectra and claim them for his own. The other authors on the paper are Kirill Tcabanenko, Robert Adlington, and Andrew R. Cowley, for whom I can find no recent information. There's a story here, for sure, but I don't know its details. . .

Comments (34) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 10, 2015

The Cost of Irreproducibility

Email This Entry

Posted by Derek

There's been a lot written about nonreproducibility in biomedical research - it's a topic that everyone with experience in the field can relate to. Now here's a paper that suggests that the cost of all these nonreproducible papers could be as much as $28 billion per year.

For this paper, we adopt an inclusive definition of irreproducibility that encompasses the existence and propagation of one or more errors, flaws, inadequacies, or omissions (collectively referred to as errors) that prevent replication of results. Clearly, perfect reproducibility across all preclinical research is neither possible nor desirable. Attempting to achieve total reproducibility would dramatically increase the cost of such studies and radically curb their volume. Our assumption that current irreproducibility rates exceed a theoretically (and perhaps indeterminable) optimal level is based on the tremendous gap between the conventional 5% false positive rate (i.e., statistical significance level of 0.05) and the estimates reported below and elsewhere . . .

The authors believe (reasonably) that four types of trouble contribute the most to the problem: (1) problems with overall study design, (2) incorrect biological reagents and/or reference materials, (3) poorly thought-out or poorly reported laboratory protocols, and (4) problems with data analysis and reporting. Their estimate is that the percentage of non-reproducible work out there can't be any lower than about 18%, and if you were to assume worst-case for everything, it would be as high as 88%. The middle of their probability distribution is still 53%. Yep, according to these estimates, more than half of the literature is not reproducible.

That seems high to me, and I'm fairly cynical about the literature. But they've based this estimate on the many publications that have addressed problems with antibodies, with study designs, with contaminated cell lines, and many other factors, and if you start adding all these things up, the numbers get pretty alarming. (Perhaps they could be brought down a bit by assuming that some of these papers have screwed up in more than one way? I admit that this is an odd way to look for solace). The authors do admit that the literature on this subject is not as large (or as rigorously defined) as it should be to make a hard estimate of these things, but maintain that there's little doubt that the problem is real, and large enough to worry about.

Getting the $28 billion dollar figure is fairly straightforward:

Extrapolating from 2012 data, an estimated US$114.8B in the United States is spent annually on life sciences research, with the pharmaceutical industry being the largest funder at 61.8%, followed by the federal government (31.5%), nonprofits (3.8%), and academia (3.0%). Of this amount, an estimated US$56.4B (49%) is spent on preclinical research, with government sources providing the majority of funding (roughly US$38B). Using a conservative cumulative irreproducibility rate of 50% means that approximately US$28B/year is spent on research that cannot be replicated (see Fig 2 and S2 Dataset). Of course, uncertainty remains about the precise magnitude of the direct economic costs—the conservative probability bounds approach reported above suggest that these costs could plausibly be much smaller or much larger than US$28B. Nevertheless, we believe a 50% irreproducibility rate, leading to direct costs of approximately US$28B/year, provides a reasonable starting point for further debate.

Even if that 50% estimate is wrong, and it had better be, the figures are still very large. For comparison, that $28 billion dollar figure is about 90% of the budget allocated to the entire NIH, so that's real money. (Note - I'm not saying that 90% of the NIH budget is wasted! I'm saying that if this paper is correct, an amount roughly equivalent to that is wasted by all parties every year, an alarming thought).

There's room for pessimism. Consider that this analysis takes a linear approach, just multiplying out the percentages. But research isn't linear like that. Some irreproducible papers will have a much greater impact, and waste vast amount of cash beyond that of the original research group. But balancing that out, there are many "silently irreproducible" ones, published on not-very-interesting topics in not-very-interesting journals, that lead to no real followup. That $28 billion figure, though, is just accounting for the latter case, the direct costs. The follow-on costs of trying to build on work that isn't right haven't been factored in. So even if the real irreproducibility rate is, say, 25%, that would lead to direct costs of $14 billion, but real costs would surely still be higher than that. Not good at all.

Update: here's a piece at Science's new site on this study. Second update: Nature News as well.

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

June 8, 2015

Paying For Faster Peer Review

Email This Entry

Posted by Derek

Earlier this year, the Nature Group publication Scientific Reports tried an experiment for a month with "">fast-track peer review". This meant that you could pay extra to have your manuscript reviewed more expeditiously - I was going to say "more quickly", but that makes it sound a bit too quick, as in "Yeah, OK, why not".

The speed-up was accomplished by a third party (Research Square) that pays reviewers for quality work done quickly, as opposed to the usual system, which (as far as I can see) runs on persistent editorial nagging and reviewer guilt. (It's not clear to me, a priori, which of these two regimes is more desirable, but I'd certainly be willing to give the incentive scheme a shot, because I'm a pretty poor fit for the regular method). The journal gave this a one-month trial, with the number of speedy review papers capped at a certain level, to see how things worked.

Here's the report. The editors acknowledge that this system opened the way for possible new abuses (but I'd add that changing any system full of human behavior opens the way for possible new abuses). A relatively small percentage of people asked for the new service, but there were takers. Geographically, they say that the highest percentage came from China, but I'd like to see that normalized according to the usual Scientific Reports manuscript distribution before drawing any conclusions.

Update: it appears that there was an attempt at a mass resignation from the journal's editorial board over this issue.

They've ended the experimental period, but are reserving the right to try this again. I think that most scientists can agree that the current peer-review system is probably not the best of all possible worlds - Doctor Pangloss never had to deal with the dreaded Third Reviewer, especially when the whole process took about six months. Whether there's anything better, though, has been a subject of much debate. I definitely appreciate NPG's willingness to experiment, and I wish that other scientific publishers were doing the same.

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

June 4, 2015

115 Years of JACS Titles

Email This Entry

Posted by Derek

Here's Stuart Cantrill with a look back at 115 years of JACS paper titles. Stuart has a lot of graphics showing the most popular 150 words from the titles of each period, and there are a lot of interesting things in there. (He has a follow-up post as well, about how some words like "catalytic" get diluted by all their different forms). I decided to look closely at his lists of just the top ten from each period, though, and found quite a few trends there as well.

The 1900s, for example, certainly have an old-timey feel to them. Popular words were things like "determination" (no, not grit and fortitude, although you certainly needed them - "determination of the purity of. . ."), "analysis" and "estimation". The number two word was "acid", and it then moved into the number one slot until the end of the 1950s. I would not have guessed that one. (The follow-up post mentions that "base" and "basic" seem to be diluted by "alkaline", etc., so that they don't show up as well).

But changes are going on through that period lower on the charts. "Determination" drops out of the top ten by the 1930s, and "solutions" takes a steady dive from prominence. "Compounds" rises, hitting its peak from 1930-1950, then heading back down again. And the big winner of the last 40 years, "Synthesis", first appears in the top ten in the 1930s, is never lower than number four afterwards, and has an extended run at number one.

One trend that I found interesting was the word "Reaction". It makes a big splash in the 1920s, having not appeared on the previous charts and suddenly showing up at number three. It's at that position or at number two all the way through the 1950s, but in the 1960s it's gone, replaced by "Reactions". That stays in the top three all the way through the 1990s, when (for some reason) "Reaction" returns (and the plural vanishes). I tend to think that this is some sort of text-string-search error, but if it's not, it makes you wonder about why we've come back to the singular form so strongly after all these years. Reactions of this whole class of things, versus reaction of this singular thing that I've discovered, but why the changes back and forth?

There are flash-in-the-pan words from some decades (an effect that's even stronger, naturally, when you look at the 150 most popular). But even in the top ten, "Ion" and "Resonance" only hit the list in the 1960s and 70s, making them the JACS-title equivalent of Nehru jackets, bell-bottom pants, and shiny disco shirts. "CO" is a top-ten phenomenon only of the 1980s in title words - lots of metal complexes, no doubt, which fits with my own memories (I started reading JACS, to some extent, around 1982-1983). The 1980s and 1990s are the only times that "Radical" makes the top ranks, and no, I think we can assume that this is in the single-electron sense and not "That was way radical, dude". The 2000s mark the peak of "NMR" near the top.

The 2010-2014 period looks to be pretty anomalous. Longtime favorite title words like "Formation" and "Effects" are taking swan dives off the top ten chart, and there are three words so far that have never cracked the top ten before: "Protein", "Via", and "Nanoparticles". Judging from Stuart's list, the perfect JACS paper of the last four years should have a title something like "Studies of Protein Complexes Via Molecular Synthesis of Nanoparticles" (seven of the top ten - it can't miss!)

Comments (9) + TrackBacks (0) | Category: The Scientific Literature

May 21, 2015

Guess What: Your Peers Are Already Reviewing You

Email This Entry

Posted by Derek

Looks like there are biologists who are getting a chance to figure out what social media can do to communication in their field. Nature News reports on the response to a PNAS paper published late last year from the Mouse ENCODE consortium. That one looked at gene expression profiling in various mouse tissue samples, and concluded that mice were, at that level, not as similar to humans as had been assumed.

Not everyone bought into that conclusion. (This seems to be the destiny of most papers that come out under an ENCODE banner). Yoav Gilad and his co-worker Orna Mizrahi-Man, both at Chicago, re-analyzed that data in the paper and concluded that batch effects invalidated its conclusions. Rather than go the usual route, Gilad opened fire on Twitter, with graphs attached, and the original authors felt, well, ambushed. (That analysis is now published, open-access, in F1000Research).

Michael Snyder, a geneticist at Stanford University in California and co-author of the original paper, stands by his team’s study and its conclusions and says that Gilad broke the “social norms” of science by initially posting the critique on Twitter. Gilad says that he took to social media to highlight his work, which might otherwise have been overlooked. . .

. . .Gilad says that he posted the original critique on Twitter because it would have been difficult to draw much attention to his findings through the traditional channels. “Papers that challenge results from ENCODE or similar large consortium projects are never published in the same glamour journals as the original reports, and these papers are usually largely ignored,” he says.

He's got a point, there. And he may well have a point with his criticism of the original paper; that news article has a number of on-the-record quotes saying that it should be re-examined or retracted. I'm not going to get involved in that fight; there are a lot of far more well-qualified people trading punches. But I did want to comment on the whole peer-review-by-Twitter aspect of things, since that comes up (or will) in every part of the scientific world.

I can understand the original authors feeling blindsided. That's partly because they surely weren't expecting a detailed critique to show up from that direction. And Twitter itself doesn't exactly have a reputation as your go-to place for serious scientific discourse (140 characters and all that). But that said, my sympathies are with Gilad and Mizrahi-Man. I think that if you publish a scientific paper, you are inviting criticism from all comers - the same, in fact, as you are when you publish anything else in a public forum. Complaining that any such criticisms "didn't go through channels" misses the point.

If there really is a problem with a published paper, I think that everyone can agree that it's better to know about it sooner than later, and it's better to get the details of the criticism out there where everyone concerned can read it and come to their own conclusions. Announcing something like this on Twitter, then, actually makes sense in those regards. I like it a lot better than, say, this approach, which was proposed as the reasonable-man method by the editors of ACS Nano after a blatantly faked paper in their journal got splattered all over Twitter and the chemistry blogs. That one only took fourteen months to get cleared up, through the normal channels.

Now, this latest controversy is certainly not about faked data - it's about experimental design and interpretation. One could argue that these more-subtle differences of opinion should be worked out in a more "appropriate" way, while the rough-and-tumble public sphere might be OK to call out papers that are just made up. (Not that the ACS journal editors would have agreed even with that - their take, which I found quaint, was that no one should even comment on such things out in the internet world until journal editors have had a chance to make a decision).

But in this latest case, the critics of the paper are saying that it's wrong, that its conclusions are invalid, because the data were not handled correctly. So it's not all that subtle, when you get down to it. The fact that reasonable observers could read the paper and come to those conclusions is something that everyone looking at the paper should know, and should have a chance to know. Readers can then come to their own conclusions, and that process (thinking hard about the data, and the issues around how it was collected) will do everyone involved some good. Science improves by such a process. Doing it all carefully and quietly, it seems to me, improves it less.

That's not to say that there aren't problems with wrestling out in the public forum. Unfounded reputation-smearing allegations are what everyone worries about when this sort of thing comes up, and it's a legitimate worry. But readers and fellow scientists should have the opportunity to judge the accusers as well. If they come in with solid reasons to question the original work, things that have clearly had thought and effort put into them, then more people will take them seriously. Anonymity has a part to play here, too: note that Gilad and Mizrahi-Man are serious enough, in this case, to do this under their own names, which also counts for a lot. Anonymous criticism needs to exist, but anonymous critics always realize (or should) that they're necessarily starting out with a credibility disadvantage. (Even so, sufficiently strong arguments - flurries of duplicated and swapped gel lanes, obviously cut-and-pasted "photographs" of nanostructures in the supporting information - speak for themselves so clearly that they could be made under any name at all).

So I think it's worth the risk to do this sort of thing out where everyone can see it. And honestly, it's not like we're going to stuff all these snakes, genies, and worms back into their respective cans and bottles at this point, anyway. I think that open, post-publication peer review in science is here, and it's here to stay. We'd better get used to it.

Update: via the comments, here's an analysis that suggests that the criticism of the mouse ENCODE work may well have something to it. And on the topic of post-publication peer review, I should really note that PubPeer is where a lot of this is taking place.

Comments (42) + TrackBacks (0) | Category: Biological News | The Scientific Literature

Crappy Antibodies: Available Now, and for the Foreseeable Future

Email This Entry

Posted by Derek

I made a brief mention of this article yesterday, but I wanted to highlight it. It's a look, from Nature New, at the broader implications of the antibody problem in research. Antibodies are, of course, universal reagents in molecular biology assays. If you suddenly declared their use illegal, the field would just collapse. But we can't live with 'em, either, because a really significant percentage of the antibodies used are not as good as they should be. A really disturbing percentage of the scientific literature is (at the very least) complicated by this problem, and some of it is flat-out invalidated by it.

As has been mentioned here several times, the same goes for small-molecule chemical probes, too, and how. That problem is, in principle, a bit more solvable (and I've been hearing about some efforts to try to help solve it - more on that as it develops). Small molecules are easier to assay for purity and identity, for one thing, and compared to antibodies, there are a lot fewer of them. The article estimates that there are around 300 companies selling something like two million antibodies. Which of these do what they're advertised to do, and under what conditions, well. . .that's hard to say:

Scientists often know, anecdotally, that some antibodies in their field are problematic, but it has been difficult to gauge the size of the problem across biology as a whole. Perhaps the largest assessment comes from work published by the Human Protein Atlas, a Swedish consortium that aims to generate antibodies for every protein in the human genome. It has looked at some 20,000 commercial antibodies so far and found that less than 50% can be used effectively to look at protein distribution in preserved slices of tissue5. This has led some scientists to claim that up to half of all commercially available antibodies are unreliable. . .

. . .Abgent, an antibody company based in San Diego, California, and a subsidiary of WuXi AppTec in Shanghai, China, tested all of its antibodies about a year ago. After reviewing the results it discarded about one-third of its catalogue.

So that should give you a rough estimate, and I don't think that many experienced assay development folks will be surprised. The people that are surprised, as usual, are the ones who just order out of the catalog and believe what's on the label. As the article mentions, a lot of people shop on price and speed of delivery, which (you'll be shocked to hear) are variables that don't always correlate well with reagent quality. And there are a lot of resuppliers out there, so even if you buy half a dozen antibodies against the same protein from different outfits, you may have only bought two. Or one. Who knows? And if you use up your supply of one that's working for you and re-order, will the new batch be the same as the old one? Who knows?

There are several online resources that are trying to address this problem (they're listed in the article), but many people don't even know about them. And as long people have the attitude that one (now more cautious) scientist expressed in the piece, the crappy reagents will continue to be sold. "I wasn't trained that you had to validate antibodies;" he says, "I was just trained that you ordered them."

Comments (22) + TrackBacks (0) | Category: Biological News | Drug Assays | The Scientific Literature

May 12, 2015

A Glance Across the Literature

Email This Entry

Posted by Derek

Time for another roundup of the chemistry journals. . .let's see what we have in the current issues:

JACS: "Science Rejected It, and Angewandte Couldn't Think Up a Bad Enough Joke, So Here We Are"

"A Paper Mentioning the Jahn-Teller Effect in the Title, So You May Scroll Right Along Without the Slightest Pang of Guilt"

Ang. Chem.: "A Metal-Organic Framework With Nanostructured BODIPY Ligands, Published Without Review on the Basis of the Title Alone"

"A Paper Mentioning the Jahn-Teller Effect in Its Title, Providing an Opportunity For a Brow-Furrowing Joke in the Abstract About Yawning Bank Tellers"

JOC: "A Piece of Organic Chemistry So Solid and So Reasonable That Even the Authors Are Bored With It"

"Someone Will Look Up This Paper, For Some Reason, About Ten Years From Now. But Until Then, Keep Scrolling."

J. Med. Chem.: "This Project Looks Good, But It Did Not Work. And 18 Out of the 23 Authors have Typographical Symbols Behind Their Names, Because The Work Took Place During Bush's First Term"

"We Hooked Two Totally Different Kinds of Drug Together With a Chain of Methylenes. Bet You Big Drug Companies Never Thought of That One!"

Nature Chemistry: "Whiplash-Inducing Subject Change From the Last Paper Before This One. You Only Get That in the Best Journals, Right?"

Org. Lett.: "Catalytic Photochemical Fluorinations of Photochemical Fluorination Catalysts"

Tet. Lett.: "Arrows Are All the Experimental Details You Need, Because You're Never Going to Run These Reactions, Anyway"

ACS Med. Chem. Lett.: "Holy Cow, Is This Ever Easier Than Writing a Full J. Med. Chem. Paper or What?"

Bioorg. Med. Chem. Letters: "Come For the Mini-Reviews, Stay For the. . .Well, Come For the Mini-Reviews, Anyway."

Comments (39) + TrackBacks (0) | Category: The Scientific Literature

May 4, 2015

The Reproducibility Initiative Weighs In

Email This Entry

Posted by Derek

in 2013 I mentioned the efforts by the Reproducibility Initiative to redo studies in the field of experimental psychology. That's a timely idea, since there have been many questions recently about the rigor of some of its results - for a fresh example, see here.

Now the results are in for the first 100 studies (here's an article at Nature News). This effort is being written up as a paper, and will probably change around a bit during that process, but the general points seem to be pretty clear. It depends on how you look at them, though, and what your priors were.

The majority of studies could not be reproduced. That's one way to interpret things. Based on the preselected criteria, the results of 61 of the 100 papers examined did not repeat. But 24 of those produced results, which while not on the mark, were "broadly similar" to what was originally reported, and here begins the arguing. How "broadly" do we mean? And is that enough? According to the Nature News piece, a number of psychology researchers have been taken aback by these figures, so the answer to the last question might well be "No, not really". Between one-third and two-thirds of the recent experimental psychology literature does not hold up to replication, and no one is very happy with either end of that range.

The very next questions that readers around here will have, then, is what would happen if we tried that out in our own fields. The famous "Amgen paper" on the irreproducibility of the oncology literature comes to mind. There have been arguments over that paper's methods and its interpretation, but we're eventually going to get a more in-depth look at things. As that last link mentions, the Reproducibility Initiative is also taking a look at the "fifty most impactful" recent papers in cancer biology. So we can have that discussion with some solid data when the time comes.

How about organic chemistry? I think that the rate of reproducibility would be solidly over fifty per cent, but beyond that, I'm not prepared to say. Depends on a lot on the journal you're getting the papers from! But if you pick the same sort of level that the two Reproducibility Initiative projects have for those fields - say, a list of papers from JACS and the like, I would like to think that we would come out better than experimental psychology, and probably better than cancer biology. This isn't some moral superiority at work - we just have fewer variables in chemistry. That's not to say that there aren't some fiendish complications hidden inside our work; there certainly can be. But overall, we have a lot better time of it proving that our reagents and products are what we think they are compared to those other areas.

But how would it go, reproducing 100 papers from Organic Letters? From Tetrahedron Letters? From Synthetic Communications? Not as well, I feel sure. I think that comparable journals in psychology or biology would fare even worse, but that thought is of limited comfort.

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

April 14, 2015

*Blog Post Title* Goes Here

Email This Entry

Posted by Derek

A colleague tells me that he just got a come-on from yet another unknown open-source journal, "Pharmaceutical Chemistry Review". Reproduced below, word-for-word, is the pitch. And it's hard to resist when they butter you up like this:

Dear *name*,
One of your papers has drawn our attention:
Titled: *title*
Published in: *source*.

Yep, personal attention will do the trick every time. The journal says that it publishes papers in (among other subject areas), "Lead Compounds and Enhance the New Drug Research", "Based on Potential Drug Targets for Life Science Research Reveals" (that's what it says, honest), and "The Penetration of Molecular Mechanics and Quantum Chemistry and Pharmaceutical Science". You will not be surprised to find that the go-getters behind this "journal", Biological and Chemical Publishing, are on the list of predatory publishers. And man, with bait like this, they must be reeling them in. . .

Comments (14) + TrackBacks (0) | Category: The Scientific Literature

April 9, 2015

Not 100%. Not Really.

Email This Entry

Posted by Derek

I've mentioned Tomas Hudlicky's views on the state of the current synthetic organic chemistry literature here before: he's not very complimentary, and he's good some good reasons not to be. I had an email from him the other day with another example of some of the problems that he's talking about.

Take a look at this paper, which just came out in JOC. It's a total synthesis of a natural product called brazilin, from a group in South Korea. Now, I have no doubt that they have made brazilin. And I have no doubt that they have made it by the route that they detail in the paper. But (like Hudlicky) I do have doubts that six reactions in their synthesis all went with flat 100% yields.

He's shown that if you do a standard workup and chromatography, the odds of you getting 95% and above are very small indeed. You can't even recover weighed amounts of known compounds to 100% with that treatment, much less clean up reaction mixtures. But that's just what happens in this paper. Now, this may seem like a minor point - OK, the yields were high, the reactions worked well, so what's the big deal with saying 97%? Or even 100%?

The big deal is that this is a symptom of a larger problem - the hyping of results, dressing things up to look better than they are. 100% yields are wishful thinking at best, and deception at worst (self-deception, most likely) and none of these are good things to let into the scientific literature, even at very dilute levels. The same impulse and the same tendencies can lead to much worse things. If we're all going to start thinking honestly and clearly about our work, maybe we could start small, and admit that there are no 100% yields after extraction and chromatography. Yes, yes, your hands are great and your technique is awe-inspiringly flawless. But you didn't get a 100% yield.

Comments (61) + TrackBacks (0) | Category: The Scientific Literature

March 30, 2015

People Only Fake Things That Are Valuable

Email This Entry

Posted by Derek

The fake peer review scandals just keep on coming. BioMed Central noticed problems with a few papers last fall, but they've now had to pull 43 papers from their journals:

Some of the manipulations appear to have been conducted by third-party agencies offering language-editing and submission assistance to authors. It is unclear whether the authors of the manuscripts involved were aware that the agencies were proposing fabricated reviewers on their behalf or whether authors proposed fabricated names directly themselves.

That's being very diplomatic. I would guess that the odds are very high that the authors involved either personally suggested fake friends to do the reviewing, or knew that they were paying someone to suggest some, or didn't care much one way or another as long as their paper got published. They're paying some agency to get that to happen, so why should they concern themselves with the details of how the goods are delivered?

What we have is a counterfeiting problem. In too many places, the currency of a scientific career is the number of papers that are attached to a person's name. And as with any valuable currency, the incentive exists to pass off fake versions of it as the real thing. Base metals are mixed into the coins; paper notes are copied. In some countries, generating a list of publications is the equivalent of printing off stacks of hundred-dollar bills down in the basement. In these days of modern times, as the Firesign Theater guys used to say, we now have third parties who will let you time-share on their basement printing press. You chip in for the ink and paper, and they'll run you off some notes.

Counterfeiting goes on as long as these notes are valuable. So as long as there are places that count papers for promotion, tenure, etc., there will be people faking papers and faking the methods to get them published. I applaud the efforts of various publishers to try to police this kind of thing, but the real problem is on the demand side.

Comments (29) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

March 11, 2015

Three More Retractions

Email This Entry

Posted by Derek

Ouch. When the "unclick" work from the Bielawski lab at Texas was found not to hold up, the word was that other papers involving the now-hard-to-find co-author (Kelly Wiggins) were being looked over.

And here come three more retractions. They all cite scientific misconduct on the part of "one of the co-authors" who was affiliated with the university at the time. This is quite the stink bomb for everyone involved, and as usual when something like this happens, you wonder how much could have been done to prevent it. But the unnerving truth is that if someone is willing to really go all-out in faking data, it can be rather hard to catch them. For one thing, you don't expect someone to be just making it all up - it can be hard to get your head around that idea (and early in my chemistry career, I encountered a case of just that, so I speak from a minor sort of experience). And if the data have been hocused well, the numbers and results can look quite convincing. In the end, we're often taking each other's word for stuff in science, and if you want to abuse that, you can: for a while.

Comments (25) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

March 3, 2015

Doing the Right Thing

Email This Entry

Posted by Derek

This is well outside my field of chemistry (a paper on iridium-catalyzed silane couplings), but I wanted to highlight it anyway. A grad student working on the project realized that one of the key products had been mischaracterized (not hard to do, with these compounds), and that this invalidated a good part of the published paper. The student went to their professor (Rory Waterman at Vermont) who promptly retracted the paper with a full explanation of what happened. That is exactly how it's supposed to work, and all the shenanigans that go on in the literature are enough to make you forget that. Thanks to Prof. Waterman, and I hope he goes on to give his student a glowing recommendation.

Comments (14) + TrackBacks (0) | Category: The Scientific Literature

February 26, 2015

Double-Blinded Peer Review

Email This Entry

Posted by Derek

Nature has decided to add an option for double-blind peer review - papers would be sent to the referees without author names or institutional affiliation on them. I think this is a worthwhile idea, but I agree with many of the points in this post over at Retraction Watch by David Vaux in Melbourne.

A big potential problem is that the double-blind system is optional. It's reasonable to assume that papers from Big Names at Big Labs won't bother, because they have more to lose by being covered up. So the double-blinded papers might end up disproportionately from smaller groups who are trying to even the playing field, and if that happens, it risks becoming a negative signal all its own. It might be better if Nature were to take the plunge and blind everything. And what about the editors? They're the ones deciding at the very beginning about whether to send a paper out for review at all, and at a journal like Nature that's a big step in itself. Should the papers be blinded even before they get to that stage? Why not?

It's true that in some cases a reviewer will be able to guess where a given paper came from, or at least to narrow it down. There's no way around that, but I still think that double-blind peer review is a worthwhile idea. Will any other big name journal follow Nature's lead, or go even further?

Comments (24) + TrackBacks (0) | Category: The Scientific Literature

February 20, 2015

Unclick Undone, Unsurprisingly

Email This Entry

Posted by Derek

The now-notorious "unclick" paper has been retracted. Last summer saw an editorial "Expression of Concern", and later it was reported by C&E News that a common author (Kelly Wiggins) of all three papers in this area had confessed to fabricating data.

In light of this, the retraction notice is interesting. It makes reference to the UT-Austin investigation, but notes that its results (other than a finding of misconduct) were not shared. The original corresponding author, C. W. Bielawski, concluded that the key results were not trustworthy, though. He and the other author of the Science paper agreed that it be withdrawn, and "After the conclusion of the investigation, authors Bielawski and Brantley volunteered to withdraw the paper; it has not been possible to contact author Wiggins". I would guess that we're probably not going to hear from her again. . .

Comments (13) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

January 30, 2015

Underpowered And Overinterpreted

Email This Entry

Posted by Derek

Time for another "watch those statistics" post. I did one about this time last year, and I could do one every couple of months, to be honest. Here's a good open-access paper from the Royal Society on the problem of p-values, and why there are so many lousy studies out there in the literature. The point is summed up here:

If you use p=0.05 to suggest that you have made a discovery, you will be wrong at least 30% of the time. If, as is often the case, experiments are underpowered, you will be wrong most of the time.

True, true, and true. If you want to keep the false discovery rate down to below 5%, the paper says, you should be going for p<0.001. And just how many studies, of all kinds, across all fields, hit that standard? Not too damn many, which means that the level of false discovery out there is way north of 5%.

(This paper) deals only with the very simplest ideal case. We ask how to interpret a single p-value, the outcome of a test of significance. All of the assumptions of the test are true. The distributions of errors are precisely Gaussian and randomization of treatment allocations was done perfectly. The experiment has a single pre-defined outcome. The fact that, even in this ideal case, the false discovery rate can be alarmingly high means that there is a real problem for experimenters. Any real experiment can only be less perfect than the simulations discussed here, and the possibility of making a fool of yourself by claiming falsely to have made a discovery can only be even greater than we find in this paper.

The author of this piece is David Coqulhoun, a fact that some people will have guessed already, because he's been beating on this topic for many years now. (I've linked to some of his prickly opinion pieces before). He's not saying something that a lot of people want to hear, but I think it's something that more people should realize. A 95% chance of being right, across the board, would be a high standard to aim for, possibly too high for research to continue at a useful pace. But current standards are almost certainly too low, and we especially need to look out for this problem in studies of large medical significance.

Update: what this post needed was this graphic from XKCD!

Comments (50) + TrackBacks (0) | Category: Clinical Trials | Drug Assays | General Scientific News | The Scientific Literature

January 28, 2015

The ACS Becomes Invisible (But Not For Long)

Email This Entry

Posted by Derek

Here's what looks like a weird publishing decision: the American Chemical Society seems to have blocked search engines for its publications. Try it: type "JACS" into Google, and you'll no longer get the journal's page showing up in the search results. In my hands, a search for "Organic Letters" has the Wikipedia page about the journal as the first hit, and nowhere does the actual ACS Publications page appear.

I'm sure that the ACS has a reason for doing this. But I sure can't think of what it might be at the moment. The society's journals, web pages, and the content therein have just become invisible to the most widely used search tools on the planet. Who benefits from this, and how?

Update: well, as they say, never attribute to malice what can be explained by incompetence. The ACS is out on Twitter saying that they're aware of some search problems with Google, and that they're working to resolve them. Did someone mess with the robots.txt file after having too many beers?

Update 2: fixed now. The journals are back to the top of the search rankings, and order has been restored.

Comments (12) + TrackBacks (0) | Category: The Scientific Literature

Cuckoo For Publishing Fees

Email This Entry

Posted by Derek

The scam publishers line up, once again, to publish a manuscript full of gibberish. This one is titled "Cuckoo For Cocoa Puffs", and is authors, ostensibly, by Pinkerton LeBrain and Orson Welles. Seventeen "publishers" have accepted it in the past two weeks, despite the entire text being random word salad.

Previous installments of this trick and similar ones can be found here, here, here, here, here, here, here, here, and here. And that illustrates the problem. People have been hoaxing these sleazeballs for years now, and the cockroach population shows no sign of diminishing. Open-access publishing is an idea that has many merits, and the reputable OA journals are perfectly good places to publish. But the other end of the field is a festering swamp, and what will drain it?

Comments (10) + TrackBacks (0) | Category: The Scientific Literature

January 21, 2015

Maybe Don't Refund That Grant Money After All

Email This Entry

Posted by Derek

Setting what is nearly a new personal record, I'm backtracking on my approval yesterday of the idea of granting agencies getting a refund for retracted papers. The problems I mentioned in that post, along with the arguments of many in the comments section, have swung me back around.

What I kept thinking about is how this might have applied to some recent cases of fraud and misconduct. Would anyone have retrieved any of the funding yet? I doubt it - there would still be all sorts of fighting going on about intention (carelessness versus fraud) and the like. And the perverse incentives brought on by this policy are very likely to be worse than the problem that it's trying to cure. This would, unfortunately, drive honest retractions from the world (and there really are some). It would also put a lot of pressure on journals and their staff, since they are so involved in whether or not a paper is retracted at all. And what do we do about lousy journals that never retract a thing?

No, while the current system has plenty of opportunities for abuse, I think that putting in this option would not improve the situation. I've thought of Ambrose Bierce's Devil's Dictionary, where he defines a conservative as "A statesman who is enamored of existing evils, as distinguished from the Liberal who wishes to replace them with others." This grant-clawback scheme would replace existing evils with new ones.

Comments (20) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

January 6, 2015

Nonstop Glamour and Prestige

Email This Entry

Posted by Derek

A longtime reader sent along these two items as indications of just how high-end it's getting in some parts of the scientific publishing game. First off, we have King Abdulaziz University (of Saudi Arabia) aggressively recruiting for their "International Affiliate Program". What might this be? Well, here's the deal: you, as a reasonably highly cited academic in some other country, sign up for a salary from KAU (as fans of its sports teams call it) and you only have to show up in Jeddah itself for three one-week visits a year. Did I mention that the salary is $6000/month? And that they pick up business-class airline tickets for you, and that you stay in a five-star hotel while in residence? Well, they do, you know. And what do you have to do in return?

Why, you just have to partner with some Saudi faculty member, work with them on some project or another, and make absolutely sure to publish papers with them. And you also have to make sure that you change your affiliation, in listings like ISI's and other citation-tracking services, to show that you're now part of the KAU team. What could be simpler? King Abdulaziz University gets to ratchet itself up the rankings, you get to sell your good name, and everyone's happy - right?

So from now on, I will assume the worst: if I see any publications from this institution that have a more-highly-cited co-author from outside Saudi Arabia, I will conclude that this person has prostituted him- or herself for cash and a few nights in some Saudi Arabian hotel. And I will also assume that the research itself is likely of little use or interest, other than in the service of boosting citation counts. You can also get a dog to play with you if you loop a piece of steak around your neck, while we're on the subject.

We now turn over another slimy rock, to find ways to build your publication record. Maybe even to the point where King Abdulaziz University might want to grease you with loot - the sky's the limit here. A close look at phrasing across a wide variety of published papers has revealed that certain sentences and paragraph structures seem to appear far more often than one might think. (The most suggestive of these refers to a statistical test that turns out not to exist). All of these seem to go back to teams of Chinese authors, interestingly, which suggests that either these authors, from different fields and institutions, are somehow finding ways to plagiarize each other, or that these are the fingerprints of a common work-for-hire source.

In November Scientific American asked a Chinese-speaking reporter to contact MedChina, which offers dozens of scientific "topics for sale" and scientific journal "article transfer" agreements. Posing as a person shopping for a scientific authorship, the reporter spoke with a MedChina representative who explained that the papers were already more or less accepted to peer-reviewed journals; apparently, all that was needed was a little editing and revising. The price depends, in part, on the impact factor of the target journal and whether the paper is experimental or meta-analytic. In this case, the MedChina rep offered authorship of a meta-analysis linking a protein to papillary thyroid cancer slated to be published in a journal with an impact factor of 3.353. The cost: 93,000 RMB—about $15,000.

Such a manuscript did indeed show up at a likely journal (whose editors had been tipped off by Scientific American), and it was rejected. I wonder if there's a refund policy? Some sort of sliding scale, prorated by the impact factor of where the paper lands? At any rate, these people are not taking such rejections lying down. The Chinese paper mills are actively working to remove the element of chance:

Within two weeks of being contacted by Scientific American, BioMed Central announced that it had identified roughly 50 manuscripts that had been assessed by phony peer reviewers. The publisher told the Retraction Watch blog that "a third party may be involved, and influencing the peer review process." It is possible that these manuscripts came from paper mills. We were able to look at the titles and authors of about half a dozen of those papers. All appear very similar in style and subject matter to other paper mill-written meta-analyses, and all were from groups of Chinese authors.

So let's run the numbers: how many papers do you have to publish, at ten to fifteen thousand dollars a whack, to get King Abdulaziz University to offer to pay you off at $72,000 per year? Keep in mind that you're also getting grant money and position/tenure out of all those papers, too. The people paying up don't seem to think that they're wasting their money, and the people on the sell side don't seem to think that they're overpaying. Isn't it an inspiring scene?

Comments (40) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

December 15, 2014

Our Disorganized Piles of Chemical Information

Email This Entry

Posted by Derek

Here's a good look (by an all-star cast of authors) at the availability of pharmaceutical chemistry data in accessible databases. There are several points made. For one, there is far more publicly available information than ever before, and its total looks to outstrip the proprietary databases (such as CAS/SciFinder). The commercial databases, though, have the advantage of much better curation, and they remain far better for doing freedom-to-operate type searches on potential chemical matter.

But even with that advantage, searching out prior art and prior knowledge on a given structural series is not easy. The paper goes through a detailed description of the steps that might be involved if, say, one of the NIH Molecular Libraries probes turns out to be an interesting hit. Some of these have extensive annotation, and some don't. Getting a searchable string for a given compound (CAS#, SMILES, etc.) is not always straighforward, either, with issues around tautomers, double bond geometries, and salt forms. Then once you do a search, you can find some odd things:

It should be noted that if the structure search within SciFinder fails to find a CAS registry number, the search can be repeated as a similarity search to ensure that the registry number was not missed because of a salt form. Once the CAS registry number is found, the total number of literature references with biological activity captured in SciFinder can be retrieved. It is at this point that any reference to the 2009 Goldfarb U.S. patent application on life extension in eukaryotic organisms (US 20090163545) should be noted.

US20090163545 contains a data table (Figure 16 in the patent) on 499 compounds with PubChem substance IDs. However, SciFinder abstracts 6018 substances. How can this be? The patent includes the phrase, referring explicitly to (PubChem assay ID) 775, “the contents of which are herein incorporated in their entirety by reference”. This is full data disclosure taken to an extreme via subsummation of public HTS data into a patent by reference. While only 5796 substances from the HTS were referenced as “use” substances in SciFinder, 132 781 compounds were specified in the HTS (i.e., 32% of the entire Molecular Libraries screening collection, MLSMR). Thus, while this may be an exceptional patent abstraction example in SciFinder, it nonetheless illustrates how intellectual property (IP) due diligence searching can be confounded. Across the set of 322 NIH MLP probes, 72 intersect with the CIDs from AID 775, so a significant proportion will also intersect with the US20090163545 exemplifications. We were initially worried that a reference to this patent application was somehow an indicator for a flawed or promiscuous compound. We now believe the prevalence of references to this single patent application is an example of how complete data disclosure can lead to unexpected and potentially harmful consequences when performing IP due diligence.

Furthermore, not all of the structures of the NIH's MLP chemical probes are even referenced in SciFinder, weirdly (thus the "parallel worlds" of the paper's title). There are many other problems - for example, as the authors note, probably about half of the "commercially available" compounds are not actually available at all, and many have never actually been made. They're just listed because one supplier or another regards them as makeable on demand (everyone who's spent any time ordering compounds has encountered this, surely). So should such things be indexed? Have they been exemplified, in the legal sense?

And we haven't even gotten to the associated biological assay data. Comparing assays across different groups and different times is notoriously difficult - even inside single drug companies, details of the assay protocols have to be available in great detail to make sure that things are relevant. And that level of detail is often just not available in many database searches. Here's an enzyme, here's an IC50, and there you are. Good luck! The situation is a bit chaotic:

This discussion leads us to ask whether compounds in databases without any experimental data and without any link to potential utility should be considered as prior art. This class of compounds is growing dramatically, especially in the public databases, and the utility is arguably markedly less than for prophetic compounds. . .in patents, which may not be real compounds in an experimental sense but for which the relationship to experimentally tested compounds is at least clear. Such prophetic compounds have been abstracted in SciFinder since December 2007. As we have described earlier, the days when one could assume SciFinder had captured everything relevant to the entire global realm of bioactive chemistry are perhaps well passed. By definition, no quantitative assessment (such as the statistics of structure matching) across databases is possible without access to all of them, and to our knowledge this has not been undertaken to date. . .

The authors end with a call for the various database vendors and curators to hold some sort of summit meeting to keep things from diverging any more. We could end up in a situation where searching across multiple incompatible semi-redundant databases would be the only way to have any confidence that one had found what there is to find. Actually, that's the situation we're already in. We need to make that better, and the first step is to keep it from getting any worse.

Comments (15) + TrackBacks (0) | Category: The Scientific Literature

December 9, 2014

More on the "Unclick" Fraud Case at Texas

Email This Entry

Posted by Derek

Back to the Bielawski lab's "unclick" trouble. Back in June, Science had published an "Expression of Concern" about the group's original paper. Two more journal articles have had similar notes added to them since then. And now C&E News reports that UT-Austin has concluded an investigation into the problem and has stated that a common author of all three papers has confessed to fabricating data.

As the article notes, there's only one common author other than Bielawski himself, so that sort of narrows things down. And that author (Kelly Wiggins) appears on several other papers from the group, which are now being re-examined in turn. Her whereabouts seem to be unknown, after leaving a postdoctoral position at Illinois. Bielawski himself is no longer at UT either, having taken a position at an institute in South Korea (on the face of it, an unusual choice). Nothing good has come out of this, as far as I can see, and nothing looks likely to, either.

Comments (35) + TrackBacks (0) | Category: The Scientific Literature

December 2, 2014

Reading Nature For Free

Email This Entry

Posted by Derek

This will be interesting to watch: Nature (and 48 other journals of the Nature Publishing Group) are experimenting with making their current paper and archives open to view, but not to save or print.

ReadCube, a software platform similar to Apple’s iTunes, will be used to host and display read-only versions of the articles' PDFs. If the initiative becomes popular, it may also boost the prospects of the ReadCube platform, in which Macmillan has a majority investment.

Annette Thomas, chief executive of Macmillan’s Science and Education division, says that under the policy, subscribers can share any paper they have access to through a link to a read-only version of the paper’s PDF that can be viewed through a web browser. For institutional subscribers, that means every paper dating back to the journal's foundation in 1869, while personal subscribers get access from 1997 on.

Anyone can subsequently repost and share this link. Around 100 media outlets and blogs will also be able to share links to read-only PDFs. Although the screen-view PDF cannot be printed, it can be annotated — which the publisher says will provide a way for scientists to collaborate by sharing their comments on manuscripts. PDF articles can also be saved to a free desktop version of ReadCube, similarly to how music files can be saved in iTunes.

I don't seem to be in the top 100 media outlets, or if I am, I haven't heard about it yet. But I'm looking forward to trying this out. My company will pay for whatever papers we need, but this will be a way to get lots of good information without having to go through that process, or that expense. This policy seems to split the difference in the great open-access war, and may make strong advocates of either side less than happy. More thoughts as the advantages (and disadvantages) of all this become clearer. . .

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

November 20, 2014

More From The Bottom of the Publishing Barrel

Email This Entry

Posted by Derek

Now, if you want to get a paper published in a prestigious journal like The International Journal of Advanced Computer Technology, you'd better make sure that you're up to it. You'd better make sure that you have good stuff to report, and that the paper is worth a spot in a venue like that. Most of all, you'd better make sure to send $150 to some guy named Tej Pal Singh.

That's how this paper made it through. It's from a ticked-off Australian engineer, and he gave it the arresting title of "Get Me Off Your F*#&ing Mailing List". That, by coincidence, is also the text of the abstract. And of the entire paper, over and over and over, just without those internal punctuation marks.

Accepted, with "minor revisions". Rated as "Excellent" by the "reviewer". Ready to publish. All that stands in the way are those hundred and fifty bucks, and somehow I think that Tej is going to have a bit of a wait.

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

November 13, 2014

Our Most Snorted-At Papers This Month. . .

Email This Entry

Posted by Derek

You know, this is something that I hadn't thought of. Those papers with the egregious mistakes in them, the ones that we all enjoy making fun of? The traffic that comes in to them skews the journal metrics. I wonder how many journals have some of their "Most viewed/shared articles" lists dominated by the ones that everyone comes to laugh at or roll their eyes about? (That link has a good one that I hadn't come across - see the first one on the list).

Comments (2) + TrackBacks (0) | Category: The Scientific Literature

November 11, 2014

Make Sure That's What You Want to Say

Email This Entry

Posted by Derek

You want to be careful when you add comments to manuscripts, you know. Sometimes your patent application will publish with all the legal back-and-forthing still in it. (There was an even more egregious example of this in an electronics patent application about ten years ago, but I'm having trouble putting my hands on it - someone at the law department had comments in the claims like "Not sure if we can claim this!")

Journal manuscripts can publish with this sort of thing, too. Don't assume that someone will catch, say, something like thisL

The "show comments" and "track changes" functions haven't ruined quite as many careers as the "reply all" button, but they have their moments. One example of the latter I remember from a former company was one of those farewell e-mails - you know, it's been great working with all of you over the past few years, etc. Someone sent one of those out to a gigantic list of people at the company, including many of the higher-ups, and a poor soul did a Reply To All saying something like "Take me with you! I wish I were getting out of this place, too!" Whoops. I often wondered if they got their chance sooner than they expected, after that. . .

Comments (14) + TrackBacks (0) | Category: The Scientific Literature

November 10, 2014

Scam, Scam, Scam, Scam, Scammity Scam, Wonderful Scam

Email This Entry

Posted by Derek

You know, it's really hard to explain just how ridiculous the bottom end of the scientific publishing world is. I've mentioned formerly reputable journals that now want you to wire money to a bank account in the Turks and Caicos Islands and long lists of people who will "review" and "publish" outright gibberish as long as the checks clear. Note that the money is the only real thing in that transaction, but note also that some reputable publishers have fallen for random nonsense under the traditional publishing model as well. And there are people who will add your name to a paper for a fee, or even whip up some reasonable-looking data and write the whole thing up, for a somewhat larger fee. Don't have a journal to send it to? They'll fake one up for you. It's just an endless garbage heap.

Some recent posts over at ScholarlyOA make this amusingly clear. Here's a letter (PDF) from one of these so-called publishers inviting submissions to the "American Based Researche Journal" (very much sic), and you know you're in for a good time when it starts off "Dear Dear Author". It's signed by "Dr. Merry Jeans, New York, USA", but the content and grammar of the letter makes it appear that Dr. Merry has been the victim of a recent severe concussion. Or several.

And how about the "Integrated Journal of British"? Integrated Journal of British what, you say, but that's because you're narrow-minded. This, folks, is the journal of British, full stop. As ScholarlyOA discovered, their spiffy logo appears to have been lifted from a home-improvement contractor in Wisconsin, and I am not making this up. This fine publication makes a big deal out of their impact factor, 3.3275 (note the significant figures on that one). How, you wonder, does a journal like this have an impact factor like that from Thomson-Reuters-ISI? Narrow-mindedness again, friend: they have something even better, an impressive-looking certificate from the helpful people at "Universal Impact Factor".

Who they? Good question. They appear to be a fake-impact-factor shop, there to slap numbers on laughable fake journals, doubtless for a fee. (My wife is fond of quoting an Iranian proverb that translates as "A thief robs a thief, and God smiles"). I may be wronging them in that assumption, though. Their page for submitting a new journal to the database makes no mention of any fees per se, and after all, it does say that, and I quote, "Journals those who submitted fake or faulty data, will not consider for Evaluation".

If you put any journal you've ever heard of into the UIFactor database, you will find nothing. They're not interested in rating journals you've heard of; it's not their market. But if you look through their coverage list, you will find treasure after treasure. There's the "World Journal of Pharmaceutical Research", whose home country is listed as "Bulagria", which might as well be correct. "Corea" makes an appearance, and there are three entries for the "European Journal of Experimental Biology", all with different impact factors, but all listed as coming from Iraq. And so on - there's all sorts of exotica on the list, but what they all have in common is that if you click on any of the journal names, the detailed information page for each of them is infested with HTML spam for "online abortion pills" where the journal URL should be. Every single time. Someone is clearly paying close attention here.

So that leaves us with a journal-rating website, itself apparently a scam, which rates piles of obscure journals, many of them scams of their own. And it in turn has been infected by still more scamsters. It's a long way down, that's for sure, and the bottom is nowhere to be seen.

Comments (17) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

November 7, 2014

This Took Fourteen Months

Email This Entry

Posted by Derek

Readers will remember the "nanorod paper" controversy from last year - two papers were published from the Pease group at Utah on the fabrication of a type of nanostructure, but the images therein were pretty clearly fabricated themselves, especially the paper in Nano Letters. That one looked like it had been done by a fourth-grader using Microsoft Paint. Here's a post that brought these to wider attention. One of the papers was withdrawn fairly quickly, but not before the editors of ACS Nano had told everyone to run along and not to be so nasty. My take on that is the same it's always been: if you don't want your work commented on and criticized by all comers, on whatever grounds and under whatever names they like, then don't publish it. Once you do, though, those invitations have been sent out, and it was ever so for authors of all kinds. Sad? Maybe. True? Oh, yeah.

It's taken a while, but the ACS Nano paper has now been withdrawn as well. Retraction Watch now brings word of the sequel to all this, the investigation at Utah.

Ultimately, the school pinned the blame on graduate student Rajasekhar Anumolu and exonerated principal investigator Leonard F. Pease. Botkin also told us the investigation found that no federal money had been used in the experiments, despite notes on the two papers indicating otherwise. . .

[From Jeffrey Botkin, research integrity officer at the Univeristy of Utah}:

"The investigation determined that all of the images in the Nano Letters paper were fabricated and, therefore, none of the data were valid.

The supplemental figure S2c in the ACSNano paper was the only manipulated image identified in that paper. The manipulation consisted of a cut and past ³patch² over two relatively small areas of the image. These manipulations represent data falsification. For the ACSNano publication, the Investigation Committee could not determine a rationale for the image manipulation as the ³patches² did not appear to cover significant data elements in the image.

Mr. Anumolu, a graduate student in Chemical Engineering, was, by all accounts, primarily responsible for data acquisition and manuscript preparation for the Nano Letters publication and for Figure S2c in the ACSNano publication. The Investigation Committee determined that Mr. Anumolu was responsible for the image manipulations and was guilty of research misconduct. The other authors were found not guilty of research misconduct. Mr. Anumolu was not awarded his degree and is no longer affiliated with the University of Utah. Both papers have been retracted."

I can well believe that the grad student was at fault here, but it's worth remembering that Mr. Anomalous worked for Professor Pease, who is supposed to be looking over the manuscripts that go out with his name on them. (Mind you, the referees at the two journals are supposed to be looking at them, too, so there's that). But Prof. Pease also at the time apparently threatened legal action against Chemistry Blog for calling attention to the story. (The legal threat is no longer mentioned in that post, but was noted by Chembark at the time). He also asked that nothing be written until the University of Utah had finished their investigation, and we now see how long it look the wheels to turn in that case.

No, while there's room for abuse in the post-publication-commentary world, there's been a lot of abuse of the scientific publication process already - from the authors. And sometimes from their institutions, and from some of the publishers as well. Too many of the complaints from those parties about these situations seem to amount to "Please refrain from making unfavorable comments on my paper where people can see them", or perhaps "Please refrain from making unfavorable comments on this paper that we are charging the scientific community to read". As far as I know, every paper that's created a big post-publication fuss about its validity has been withdrawn, with a few cases of substantial revision and survival.

Comments (32) + TrackBacks (0) | Category: The Scientific Literature

November 5, 2014

Publish On Sight

Email This Entry

Posted by Derek

This paper is probably fine. But its subject matter is editor-bait if I've ever seen it: ionic liquids on the surface of carbon nanotubes. If they can get a fluorescent glowing BODIPY-laced sample in the graphical abstract as well, the world is theirs.

Comments (10) + TrackBacks (0) | Category: The Scientific Literature

October 30, 2014

Down With the Western Blot?

Email This Entry

Posted by Derek

A reader sends along a thought that touches on a lot of the publication scandals that have happened in molecular biology over the years. A very common feature of these is manipulation of Western blots (see that graphic the other day on the Sarkar case, assembled by a reader of Retraction Watch who goes under the name of "Scrutineer", for an excellent example). In the hands of the unscrupulous, images get duplicated, swapped, stretched, flipped, contrast-adjusted and digitally whitewashed, all in the name of making a good story.

But gel images seem to be an essential part of publication in these fields. Papers in any kind of protein-level biology work are full of lane after lane illustrating experimental changes and controls - no one would believe the work without them, but it's getting to the point that people are wondering what work to believe with them. The question is, is there something better?

Because when you get right down to it, a picture of a blot is a pretty low-information object. Bands are there, or they aren't. If there are a lot of bands, it's not like anyone knows what they all are - the only thing you can say is that this sample isn't so clean, or this protein didn't express so well. Contrast this to the sorts of characterization that you can do on smaller organic molecules. You have distinctive fingerprints of proton and carbon NMR, high-res mass spec (with its isotopic distributions and fragmentation patterns) - even good old IR, which no one uses much any more, gives a lot of information in a compound-specific readout. But for molecular biology, it's "this lane has a band" versus "this lane doesn't have a band". It's as if the chemistry journals were full of TLC plate images, which they most certainly aren't.

It's for sure that characterization of proteins is a much harder business than the characterization of small molecules. I can see why the blot-image standard got started, and why it persists. But it's too easy to fake, too easy to manipulate. Is there anything to replace it, or to enhance it to the point where you can't just whip one up in ten minutes at the keyboard? I know that editorial staffs at the major journals have been thinking about this problem, but how much progress is being made? It's still a key part of work at the bench, but when it comes time to publish, is it time to say goodbye to the plain old Western?

Update: I should have said that no one uses IR for small-molecule characterization. A strong dissent from the polymer side.

Comments (71) + TrackBacks (0) | Category: Biological News | The Scientific Literature

October 27, 2014

Fazlul Sarkar Subpoenas PubPeer

Email This Entry

Posted by Derek

Last month I mentioned that a professor at Wayne State, Fazlul Sarkar, was thinking of suing the PubPeer site or its commenters, after a host of negative comments on his papers disrupted his move to the University of Mississippi. Well, he's making good on that threat, according to Retraction Watch.

The court papers have all the details on just what a deal he was getting at Ole Miss:

First, we learn that in addition to a salary of $350,000, which has been previously reported, the University of Mississippi had offered Sarkar “Commitment to ‘help us realize the $2 million level on endowed professorship,’ “Relocation expenses up to $15,000,” “Laboratory and office space in two locations, Research Assistant Professors, up to two additional Research Associates, and administrative support,” “A start up package of $750,000,” and “Moving expenses for the laboratory and senior personnel.”

Sarkar had already signed the papers and submitted his resignation back in Detroit (and put his house on the market) when everything hit the skids. According to the court documents:

[I]n a letter dated June 19, 2014 – just eleven days before Dr. Sarkar was to begin his active employment – Dr. [Larry Walker, the Director of the National Center for Natural Products Research at the University of Mississippi Cancer Institute] rescinded that employment, as additionally confirmed by the Chancellor Jones on June 27, in effect terminating Dr. Sarkar before he’d even begun. Dr. Walker’s June 19, 2014 letter cited PubPeer as the reason, stating in relevant part that he had “received a series of emails forwarded anonymously from (sic?), containing several posts regarding papers from your lab. . .

Now the question is, who is Professor Sarkar suing, and what does he hope to accomplish? PubPeer itself is surely protected by the provisions of the Communications Decency Act that shield web site owners from comments made by users - this has been tested several times, to the best of my knowledge, and has held up every time. And Sarkar isn't suing PubPeer itself, apparently. The professor seems to mostly want the site's administrators to divulge the identities of the commenters who left so many damaging details (and who, presumably, had a lot to do with sending all those details to a list of people at Mississippi, just in case they'd missed them).
I'm not sure how far that's going to get - I mean, what could the PubPeer folks have? IP addresses? Which probably resolve to a block of Comcast or Verizon stuff that can't be narrowed down much further? The NSA might know who these people are, but I'll bet PubPeer doesn't. And what, exactly, is Professor Sarkar going to do even if his lawyers manage to track someone down? Let's be real - a university does not suddenly throw the brakes on a big tenured hire just because of a bunch of misty allegations and unsourced grumbing. No, one of the comments at Retraction Watch provides an example (at left) of the sort of thing that appears to be at the root of the problem, from this paper, which Google Scholar says has 183 citations. (A larger version can be found here, where you can see all the little artifacts of the gel lanes, and what appear to be the same artifacts as they rotate, flip, and duplicate across the figures). And this one hadn't even been posted to PubPeer yet, so my impression is that there's a lot of this stuff.

If this is the sort of thing that was mailed to everyone at Ole Miss, I can see how it might have caused some rethinking. (The university probably didn't have a Distinguished Chair of Photoshop position open). At the very least, someone in Prof. Sarkar's lab appears to have been putting together a portfolio for such a job. Can someone be defamed by having the figures from his own papers reproduced?

At any rate, I'm glad to help draw attention to his plight, and to bring details of it to people who might otherwise have missed out. Lawsuits tend to have that effect, especially lawsuits against internet sites. I will watch the progress through the legal system of Prof. Sarkar's case with great interest. Commenters to the previous post here noted that this will not be an easy one (nor a cheap one) to pursue. A comment at Retraction Watch mentions that Sarkar is on the editorial board of a host of journals, many of which are on the Beall list of predatory publishers. Perhaps he has that to fall back on?

Update: got Prof. Fazlul's name correct, and consistently spelled. Better copy-and-paste skills would have come in handy there, I have to admit.

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

October 16, 2014

The Electromagnetic Field Stem Cell Authors Respond

Email This Entry

Posted by Derek

The authors of the ACS Nano paper on using electromagnetic fields to produce stem cells have responded on PubPeer. They have a good deal to say on the issues around the images in their paper (see the link), and I don't think that argument is over yet. But here's what they have on criticisms of their paper in general:

Nowhere in our manuscript do we claim “iPSCs can be made using magnetic fields”. This would be highly suspect indeed. Rather, we demonstrate that in the context of highly reproducible and well-established reprogramming to pluripotency with the Yamanaka factors (Oct4, Sox2, Klf4, and cMyc/or Oct4 alone), EMF influences the efficiency of this process. Such a result is, to us, not surprising given that EMF has long been noted to have effects on biological system(Adey 1993, Del Vecchio et al. 2009, Juutilainen 2005)(There are a thousand of papers for biological effects of EMF on Pubmed) and given that numerous other environmental parameters are well-known to influence reprogramming by the Yamanaka factors, including Oxygen tension (Yoshida et al. 2009), the presence of Vitamin C (Esteban et al. 2010), among countless other examples.

For individuals such as Brookes and Lowe to immediately discount the validity of the findings without actually attempting to reproduce the central experimental finding is not only non-scientific, but borders on slanderous. We suggest that these individuals take their skepticism to the laboratory bench so that something productive can result from the time they invest prior to their criticizing the work of others.

That "borders on slanderous" part does not do the authors any favors, because it's a rather silly position to take. When you publish a paper, you have opened the floor to critical responses. I'm a medicinal chemist - no one is going to want to let me into their stem cell lab, and I don't blame them. But I'm also familiar with the scientific literature enough to wonder what a paper on this subject is doing in ACS Nano and whether its results are valid. I note that the paper itself states that ". . .this physical energy can affect cell fate changes and is essential for reprogramming to pluripotency."

If it makes the authors feel better, I'll rephrase: their paper claims that iPSCs can be made more efficiently by adding electromagnetic fields to the standard transforming-factor mixture. (And they also claim that canceling out the Earth's magnetic field greatly slows this process down). These are very interesting and surprising results, and my first impulse is to wonder if they're valid. That's my first impulse every time I read something interesting and surprising, by the way, so the authors shouldn't take this personally.

There are indeed many papers in PubMed on the effects of electromagnetic fields on cellular processes. But this area has also been very controversial, and (as an outside observer) my strong impression is that there have been many problems with irreproducibility. I have no doubt that people with expertise in stem cell biology will be taking a look at this report and trying to reproduce it as well, and I am eager to see what happens next.

Comments (28) + TrackBacks (0) | Category: Biological News | The Scientific Literature

October 1, 2014

No More Prearranged Editors at PNAS

Email This Entry

Posted by Derek

While we're on the topic of the literature, I see that PNAS has made some changes to their system:

Although the largest number of submissions to PNAS are through the Direct Submission route, there continues to linger a general perception that to publish a paper in PNAS, an author requires sponsorship of an NAS member. In July 2010, the member Communicated route was eliminated, but to ensure and promote submission of exceptional papers that either were interdisciplinary and hence needed special attention, or were unique and perhaps ahead of their time, authors were able to use the Prearranged Editor (PE) process for Direct Submissions. The PE process was intended to be used on rare occasions but, since we formalized the PE process, more than 11,000 papers have been submitted by authors with a PE designation. Although we are certain that some of these papers truly needed special attention, the vast majority likely did not, and therefore we are discontinuing the PE process as of October 1, 2014. We will continue to honor the current PE submissions.

They're setting up a new submission process, which (from what I can see) will make the journal very much like the rest of the field. Are there any odd routes to a PNAS publication left? As for the whole literature landscape, I'm sticking with my characterizations and there are plenty more in the comments, for now.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

No, They Really Aren't Reproducible

Email This Entry

Posted by Derek

Here's an interview with Nobel winner Randy Schekman, outspoken (as usual) on the subject of the scientific literature. This part caught my attention:

We have a problem. Some people claim that important papers cannot be replicated. I think this is an argument that has been made by the drug companies. They claim that they take observations in the literature and they can't reproduce them, but what I wonder is whether they're really reproducing the experiments or simply trying to develop a drug in an animal model and not exactly repeating the experiments in the publication. But I think it is unknown what fraction of the literature is wrong, so we're conducting an experiment. We've been approached by an organization called the Reproducibility Project, where a private foundation has agreed to provide funds for experiments in the fifty most highly cited papers in cancer biology to be reproduced, and the work will be contracted out to laboratories for replication. And we've agreed to handle this and eventually to publish the replication studies, so we'll see, you know, at least with these fifty papers. How many of them really have reproducibility. The reproducibility studies will be published in eLife. We're just getting going in that, so it may be a couple of years, but that's what we'd like to do.

OK, then. As one of those drug-company people, I can tell you that we actually do try to run the exact experiment in these papers. We may run that second, after we've tried to reproduce things in our own assays and failed, but we never write things off unless we've tried to do exactly what the paper said to do. And many times, it still doesn't work. Or it gives a readout in that system, but we have strong reason to believe that the assay in the original work was flawed or misinterpreted. We are indeed trying to develop drugs, but (and I speak from personal experience here, and more than once), when we call something irreproducible, that's because we can't actually reproduce it.

And the problem with trying to reproduce the 50 most-cited papers (see Update below!) is that most of those are probably going to work pretty well. That's why they're so highly cited. The stuff that doesn't work are the splashy new press-released papers in Nature, Cell, Science, or PNAS, the one that say that Protein X turns out to be a key player in Disease Y, or that Screening Compound Z turns out to be a great ligand for it. Some of those are right, but too many of them are wrong. They haven't been in the literature long enough to pick up a mound of citations, but when we see these things, we get right to work on them to see if we believe them.

And there really are at least two layers of trouble, as mentioned. Reproducibility is one: can you get the exact thing to happen that the paper reports? That's the first step, and it fails more than it should. But if things do work as advertised, that still doesn't mean that the paper's conclusions are correct. People miss things, overinterpret, didn't run a key control, and so on. If someone reports a polyphenolic rhodanine as a wonderful ligand for The Hot Protein of the Moment, odds are that you can indeed reproduce the results in their assay. But that doesn't mean that the paper is much good to anyone at all, because said rhodanine is overwhelmingly unlikely to be of any use. Run it through an assay panel, and it lights up half the board - try interpreting cell data from that, and good luck. So you have Non-Reproducible, and Reproducible, For All the Good That Does.

But if Shekman and the Reproducibility Project are looking for tires to kick, I recommend picking the fifty papers from the last two or three years that caused the most excitement, press releases, and press coverage. New cancer target! Stem cell breakthrough! Lead structure for previously undruggable pathway! Try that stuff out, and see how much of it stands up.

Update: this interview turns out not to be quite correct about the papers that will be reproduced. More details here, and thanks to Tim in the comments for this. It's actually the "50 most impactful" papers from 2010 through 2012, which sounds a lot more like what I have in mind above. Here's the list. This will be quite interesting. . .

Comments (29) + TrackBacks (0) | Category: The Scientific Literature

September 24, 2014

See You in Court? Not Likely

Email This Entry

Posted by Derek

There had been talk of some sort of trouble brewing via PubPeer, the open-source post-publication review site, and this appears to be it. There's more on the issue at Science: Farzul Sarkar at Wayne State is suing, if he can find the right people to sue.

The issue first came to light in August, when PubPeer’s (anonymous) moderators announced that the site had received a “legal threat.” Today, they revealed that the scientist involved is Fazlul Sarkar, a cancer researcher at Wayne State University in Detroit, Michigan. Sarkar, an author on more than 500 papers and principal investigator for more than $1,227,000 in active grants from the U.S. National Institutes of Health, has, like many scientists, had his work scrutinized on PubPeer. More than 50 papers on which he is an author have received at least one comment from PubPeer users, many of whom point out potential inconsistencies in the papers’ figures, such as perceived similarities between images that are supposed to depict different experiments.

Recently, PubPeer was contacted about those comments by Nicholas Roumel, an attorney at Nacht, Roumel, Salvatore, Blanchard & Walker P.C. in Ann Arbor, Michigan, who represents Sarkar and spoke to ScienceInsider on his behalf. On 9 June, the University of Mississippi Medical Center announced that Sarkar would join the faculty in its school of pharmacy. Records from a meeting of the Mississippi Board of Trustees of State Institutions of Higher Learning note that he was offered a tenured position and a salary of $350,000 per year, effective 1 July.

But on 19 June, Roumel says, Sarkar got a letter from the University of Mississippi revoking its offer. Science has not seen the letter, but Roumel says that in his view, “it made it crystal clear the PubPeer postings were the reason they were rescinding the job offer.” A representative for the University of Mississippi declined to comment on the case, citing prospective employees’ confidentiality.

Yeah, that would put a person in the mood to sue, for sure: tenure, 350K/year, and a chance to leave Detroit. I'm not in a position to say whether Sarkar's been defamed or not, though: there have been many complaints. And I'm also not sure that he's going to get very far trying to find out who the commenters are, because the folks at PubPeer may well have no idea themselves. (I generally have no idea who the people leaving comments on this site might be!) Under US law (the Communications Decency Act of 1996, it's pretty hard to come after the owners or moderators of a web site for the comments left by its users. So I don't think this is going to get very far, but we shall see.

Comments (49) + TrackBacks (0) | Category: The Scientific Literature

September 19, 2014

A Day of Irreproducibility In Cambridge

Email This Entry

Posted by Derek

I wanted to mention that there's an interesting symposium on "Irreproducibility in Target Validation" taking place at Novartis (Cambridge, MA) next month, October 23. This is a topic that many an industrial biopharma researcher can relate to, and as academic centers get into more drug research, they're joining the rueful party, too. There are a number of good speakers from both academia and industry on the schedule, so if you're in the area, it's worth a look. More information here.

Comments (3) + TrackBacks (0) | Category: Academia (vs. Industry) | The Scientific Literature

Big Journals, Big Retractions

Email This Entry

Posted by Derek

Big high-impact journals have more retractions, it seems. I can see how that would be, because there are several forces at work. People want to publish their splashy, cutting-edge results in the big-name journals, and a higher percentage of those papers are wrong to start with, as opposed to more incremental ones. And the big papers in the big journals get more scrutiny, so they're more likely to be picked apart when there's something wrong with them. I would have been surprised if this correlation had come out any other way, actually.

Comments (2) + TrackBacks (0) | Category: The Scientific Literature

September 15, 2014

There Must Have Been Multiple Chances to Catch This

Email This Entry

Posted by Derek

Battery%20abstract%20figure.jpgIt's time for a hang-heads-in-shame moment. This is another off the Twitter feed, and the only place to see the figure in its native state is to go the the Chemical Reviews table of contents and scroll down to the article titled "Aqueous Rechargable Li and Na Ion Batteries". A perfectly reasonable topic, but take a look at the graphical abstract figure. Oh, dear.

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

September 10, 2014

Peer Review, Up Close and Personal

Email This Entry

Posted by Derek

Retraction Watch has a rare look behind the peer review curtain in the (now notorious) case of the STAP stem cell controversy. This was the publication that claimed that stem-like cells could be produced by simple acid treatment, and this work has since been shown to be fraudulent. Damaged reputations, bitter accusations, and one suicide have been the result so far, and there are still bent hubcaps wobbling around on the asphalt.

The work was published in Nature, but it had been rejected from Science and Cell before finding a home there. That's not unusual in itself - a lot of groundbreaking work has had a surprisingly difficult time getting published. But the kinds of referee reports this got were detailed, well-argued, and strongly critical, which makes you wonder what Nature's reviewers said, and how the work got published in the form it did, with most (all?) of the troublesome stuff left in.

Retraction Watch has obtained the complete text of the referee comments from the Science submission process and published them. Here are some highlights from just the first reviewer:

. . .This is such an extraordinary claim that a very high level of proof is required to sustain it and I do not think this level has been reached. I suspect that the results are artifacts derived from the following processes: (1) the tendency of cells with GFP reporters to go green as they are dying. (2) the ease of cross contamination of cell lines kept in the same lab. . .

. . .The DNA analysis of the chimeric mice is the only piece of data that does not fit with the contamination theory. But the DNA fragments in the chimeras don’t look the same as those in the lymphocytes. This assay is not properly explained. If it is just an agarose gel then the small bands could be anything. Moreover this figure has been reconstructed. It is normal practice to insert thin white lines between lanes taken from different gels (lanes 3 and 6 are spliced in). Also I find the leading edge of the GL band suspiciously sharp in #2-#5. . .

This report and the other two go on to raise a long list of detailed, well-informed criticisms about the experimental design of the work and the amount of information provided. Solutions and reagents are not described in enough detail, images of the cells don't quite show what they're supposed to be showing, and numerous useful controls and normalizations are missing outright. The referees in this case were clearly very familiar with stem cell protocols and behavior, and they did exactly what they were supposed to do with a paper whose claims were as extraordinary as these were.

Had any of this stuff been real, meeting the objections of the reviewers would have been possible, and would have significantly improved the resulting paper. This process, in fact, handed the authors a list of exactly the sorts of objections that the scientific community would raise once the paper did get published. And while rejections of this sort are not fun, that's just what they're supposed to provide. Your work needs to be strong enough to stand up to them.

Congratulations to the Science and Cell editorial teams (and reviewers) for not letting this get past them. I would guess that publication of these reports will occasion some very painful scenes over at Nature, though - we'll see if they have any comment.

Comments (11) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

September 4, 2014

Are Your Compounds Ugly? Do You Know?

Email This Entry

Posted by Derek

A reader sends along this paper, on some small molecules targeting the C2 domain of coagulation factor VIII. It illustrates some points that have come up around here over the years, that's for sure. The target is not a particularly easy one: a hit would have to block the interaction of that protein domain with a membrane surface. There is something of a binding pocket down in that region, though, and there were some hits reported from a screen back in 2004. Overall, it looks like a lot of targets that show up, especially these days - you're trying to affect protein conformation by going after a not-necessarily-made-for-small-molecules cavity. Possible, but not something that's going to light up a screening deck, either.

And many of the things that do show up are going to be false positives of one sort or another. That's always the tricky part of doing low-hit-rate screening. The odds are excellent that any given "hit" will turn out not to be real, since the odds are against having any hits at all. This is especially a red flag when you screen something like this and you get a surprisingly decent hit rate. You should suspect fluorescence interference, aggregation, impurities, any of the other myriad ways that things can be troublesome rather than assume that gosh, this target is easier than we thought.

It's often a chemist who's in charge of dumping these buckets of cold water (if you have the help of the people who set up the assay, so much the better). Traditionally, it's one of the biology project champions who gets enthusiastic about the great list of compounds, but if you have someone who's been burned by false positives a few times, then so much the better, too. It's not fun to knock down all these "hits" and "leads", but someone's got to do it, otherwise everyone's time will be wasted to an even more painful extent.

And you should be especially worried when your screen turns up compounds like some of the ones in this paper. Yep, it's our old friends the rhodanines, everybody's cheap date of the screening deck. These compounds have come up around here many times, because they keep on showing up in the flippin' literature. In this case, the authors did some virtual screening over the ChemBridge collection and then moved on to assays against the protein itself, eventually finding a number of active compounds in the micromolar range. The compounds look a lot like the ones from 2004, since those were used as the template for screening, and that was a pretty ugly rhodanine-infested set, too.

Indeed most of the compounds they found are pretty unattractive - the aforementioned rhodanines, lots of nitroaromatics, some other heterocycles that also hit more often than one would like. I would feel better about these sorts of papers if the authors acknowledged somewhere that some of their structures are frequent hitters and might be problematic, but you don't often see that: a hit is a hit, and everything's equally valid, apparently. I would also feel better if there were something in the experimental section about how all the compounds were assayed by LC/MS and NMR, but you don't often see that, either, and I don't see it here. Implicitly trusting the label is not a good policy. Even if the particular compounds are the right ones in this case, not checking them shows a lack of experience (and perhaps too trusting a nature where organic chemistry is concerned).

But let's cross our fingers and assume that these are indeed the right compounds. What does it mean when your screening provides you with a bunch of structures like this? The first thing you can say is that your target is indeed a low-probability one for small molecules to bind to - if most everything you get is a promiscuous-looking ugly, then the suspicion is that only the most obliging compounds in a typical screening collection will bother looking at your binding site at all. And that means that if you want something better, you're really going to have to dig for it (and dig through a mound of false positives and still more frequent hitters to find it).

Why would you want to do that? Aren't these tool compounds, useful to find out more about the biology and behavior of the target? Well, that's the problem. If your compounds are rhodanines, or from other such badly-behaved classes, then they are almost completely unsuitable as tool compounds. You especially don't want to trust anything they're telling you in a cellular (or worse, whole-animal) assay, because there is just no telling what else they're binding to. Any readout from such an assay has to be viewed with great suspicion, and what kind of a tool is that?

Well then, aren't these starting points for further optimization? It's tempting to think so, and you can give it a try. But likely as not, the objectionable features are the ones that you can't get rid of very easily. If you could ditch those without paying too much of a penalty, you would have presumably found more appealing molecules in your original screen and skipped this stage altogether. You might be better off running a different sort of screen and trying for something outside of these classes, rather than trying to synthesize a silk purse out of said sow's ear. If you do start from such a structure, prepare for a lot of work.

As mentioned, the problem with a lot of papers that advance such structures is that they don't seem to be aware of these issues at all. If they are, they certainly don't being them up (which is arguably even worse). Then someone else comes along, who hasn't had a chance to learn any of this yet, either, and reads the paper without coming out any smarter. They may, in fact, have been made slightly less competent by reading it, because now they think that there are these good hits for Target Z, for one thing, and that the structures shown in the paper must be OK, because here they are in this paper, with no mention of any potential problems.

The problem is, there are a lot of interesting targets out there that tend to yield just these sorts of hits. My own opinion is that you can then say that yes, this target can (possibly) bind a small molecule, if those hits are in fact real, but just barely. If you don't even pick up any frequent hitters, you're in an even tougher bind, but if all you pick up are frequent hitters, it doesn't mean that things are that much easier.

Comments (21) + TrackBacks (0) | Category: Academia (vs. Industry) | Drug Assays | The Scientific Literature

August 27, 2014

Life Is Too Short For Some Journal Feeds

Email This Entry

Posted by Derek

What scientific journals can you not be bothered to keep up with? I know, sometimes it's tempting to answer "all of them", but a well-informed chemist really should watch what comes out in the better ones. But how about the not-so-better ones? The "Life's too short" ones? Reading journals by RSS gives a person some perspective on signal-to-noise.

One problem is that Elsevier's RSS feeds are sort of perpetually hosed. Are they working now? I haven't checked in a while, because I finally gave up on them. And that means that I don't regularly look at Tetrahedron Letters or Bioorganic and Medicinal Chemistry Letters, even though (once in a while) something interesting turns up there. I look at ACS Medicinal Chemistry Letters more often, just because it has a working RSS feed (and I should note that I've rotated off their editorial board, by the way). Overall, though, I can't say that I miss either of those Elsevier journals, because you have to scroll through an awful lot of. . .stuff. . .to see something worth noting.

The same goes, I'm afraid, for Chemical Communications, and that makes me wonder if it's possible to keep up with the Letters/Communications style journals usefully at all. There are just so many papers pouring through them, and since Chem Comm takes them in from every sort of chemistry there is, vast numbers of them are of little interest to any particular reader. Their mini-review articles are perhaps an attempt to counteract this problem, and the journal also seems to have a slant towards "hot" topics. It's still in my RSS feed, but I look at the numbers of papers that pile up in it, and wonder if I should just delete and get it over with.

Organic Letters, on the other hand, I seem to be able to stay on top of, perhaps because it's focused down to at least organic chemistry (as opposed to Chem Comm). And I find a higher percentage of papers worth looking at than I do in Tet Lett (do others feel the same way?) And as for the other short-communications organic chemistry journals, I don't have them in the feed. Synthesis, Syn Comm, Synlett - writing this prompts me to go in and add them, but we'll see over the next couple of months if I regret it.

What it comes down to is that there's room for only a certain number of titles that can be followed as the papers publish. (The rest of them turn up in literature searches, responses to directed queries). And there are only a certain number of titles that are worth following in real time. So to get back to the question at the start of the post, which well-known journals do you find to be not worth the trouble?

Comments (17) + TrackBacks (0) | Category: The Scientific Literature

August 25, 2014

Citable Garbage

Email This Entry

Posted by Derek

Experimental and Clinical Cardiology used to be a reputable journal. Now it's a trash heap piled with crap. No, literally - the Ottawa Citizen newspaper has proof, thanks to reporter Tom Spears (who's an experienced hand at this). The journal was sold last year, and the new owners will publish absolutely anything you send them, as long as you send them $1200 to their bank account in the Turks and Caicos Islands. I wish I were making all that up, but that is exactly how it goes, offshore banking and all.

Spears whipped together a gibberish cardiology paper by taking one about HIV and doing a find-and-replace to substitute "cardiology" for "HIV" wherever it occurred. I'm sure it reads just fine, if you're high on crack. He stripped out all the graphics, wrote up some captions for new ones, but didn't send any graphs or figures with his submission. No problemo, dude! Paper accepted! As soon as the money shows up under that palm tree in the Caribbean, this junk will become the latest contribution to the medical literature.

The "journal" lists an affiliation with the International Academy of Cardiovascular Sciences in Winnipeg, which organization is pretty upset about that, since there's no connection at all any more. But how to get that fixed? The phone number listed for the editorial office doesn't work. And they don't respond to any emails that they don't feel like responding to, which I'd guess are all the ones that don't involve the possibility of $1200 wire transfers.

The wonderful people behind this scam will ride it as long as a shred of reputation clings to the journal's name, or as long as people send them money, whichever comes first. The journal's web site, which I will consider linking to if they pay me twelve hundred dollars, looks legit, except for the slightly-shaky-English-style notice that "Starting from Jan 1, 2013, Experimental and Clinical Cardiology Journal will operate under new publishing group". If you click "Editorial Board", it tells you that a new one is coming soon. And this part is pretty interesting, too - they say that they provide:

. . .outstanding service to authors through a clear and fast editorial process. Review and decision will be fast and our editorial policy is clear: we will either accept your manuscript for publication or not, our editors will not ask for additional research.

All submissions will be peer reviewed, and our reviewers are asked to focus their attention to data presented in the article. Your manuscript, after the review process can be or accepted or declined. Three independent reviewers are reviewing each manuscript and if two of them accept the manuscript then your work will be published without any further corrections. Note that we will not reject a manuscript because it is out of scope or for its perceived importance, novelty or ability to attract citations: we will publish any study that is scientifically sound.

Yeah boy! But as it says under "Publication Fees", "Open access publishing is not without its costs". One of those costs should be the scientific credibility of anyone who sends a paper in to the place these days. I've looked over the most recent papers listed on the web site - there's one from a hospital in Barcelona, a university in Turkey, an institute in China, some group from Italy whose paper doesn't load well, and a bunch of people with German-sounding names whose paper appears to be two pages long and consists of one figure and no text. An erratum? Who can tell? And who would bother? You might as well copy-and-paste some old Star Wars fan-fiction; no one's going to notice. Every single one of these lead authors probably had their paper turn around within a couple of days, and sent $1200 to the flipping Turks and Caicos without batting an eye, for a journal that's supposedly based in Switzerland. For shame.

No getting around it: if you send money to any of the publishers on Beall's List, you are funding a bunch of scam artists. And if you use such a paper to pad your own c.v., then you've decided to become a scam artist yourself.

Comments (8) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

July 11, 2014

My Imaginary Friends Would Be Glad to Serve as Referees

Email This Entry

Posted by Derek

Here's the biggest fake-peer-review operation I've heard of yet. Retraction Watch, which does not seem to be in any danger of running out of material, reports that a researcher in Taiwan decided to not leave the review process at the Journal of Vibration and Control up to chance. He set up scores of fake identities in their online submission database, with as many as 130 fabricated e-mail addresses, and guess who got to review his manuscripts?

The journal has retracted sixty papers going back to 2010, and I'd like to know if that's the record. I haven't heard of anything better - well, worse, you know what I mean. The professor involved has been removed from his position in Taiwan, as well he might, and the editor of the journal has resigned. As well he might, too - that editor is not implicated in the publication scam, as far as I can tell, but what exactly were his editorial duties? Dozens of papers come pouring in every year from some obscure university in Taiwan, all of them with overlapping lead or co-authors, and you don't even so much as look up from your desk? Hardly a month goes by without another bulletin from the wildly productive engineers at Pingtung U, sometimes four or five of the damn things at once, and you think you're doing your job? And nobody else who reads this journal - assuming anyone ever does - wonders what's going on, either?

If the professor involved was really getting something out of this (tenure, promotion, grant money, what have you), then the people who awarded those to him were idiots, too. In fact, that's how I'd sum up the whole affair: a fool, faking papers for a bunch of incompetents, and rewarded for it by idiots. What a crew. You really cannot underestimate the low end of the scientific publishing industry, nor its customers.

Comments (14) + TrackBacks (0) | Category: The Scientific Literature

July 7, 2014

That Retracted Stressed Stem Cell Work

Email This Entry

Posted by Derek

This article from David Cyranoski at Nature News is an excellent behind-the-scenes look at all the problems with the "STAP" stem-cell work, now retracted and apparently without any foundation at all. There were indeed problems with all of it from the start, and one of the key questions is whether these things could have been caught:

The committee was more vexed by instances of manipulated and duplicated images in the STAP papers. Obokata had spliced together gel lanes from different experiments to appear as one. And she had used an image of cells in a teratoma — a tumorous growth that includes multiple types of tissue — that had also appeared in her PhD dissertation. The captions indicated that the image was being used to represent different types of cell in each case. The committee judged that in both instances, although she might not have intended to mislead, she should have been “aware of the danger” and therefore found her guilty of misconduct. Obokata claimed that they were mistakes and has denied wrongdoing. . .

. . .Philip Campbell, editor-in-chief of Nature, says: “We have concluded that we and the referees could not have detected the problems that fatally undermined the papers.” But scientists and publishers say that catching even the less egregious mistakes raises alarm bells that, on further investigation, can lead to more serious problems being discovered.

Many say that the tests should be carried out on all papers. Christopher says that it takes about one-third of her working week to check all accepted manuscripts for the four journals published by EMBO Press. At Nature and the Nature research journals, papers are subjected to random spot-checking of images during the production process. Alice Henchley, a spokeswoman for Nature, says that the journal does not check the images in all papers because of limitations in resources, and that the STAP papers were not checked. But she adds that as one outcome of this episode, editors “have decided to increase the number of checks that we undertake on Nature’s papers. The exact number or proportion of papers that will be checked is still being decided.”

A complication is that some of the common image manipulations (splicing gel lanes, for example) are done in honest attempts to present the data more clearly, or just to save space in a figure. My guess is that admitting this up front, along with submitting copies of the original figures to the editors (and for inclusion in the Supplementary Material?) would help to clear that up. The article raises another good point - that editors are actually worried about confronting every example of image manipulation that they see, for fear of raising the competence of the average image manipulator. There's an evolutionary-arms-race aspect to all this that can't be ignored.

In the end, one gets the impression that Nature's editorial staff (a separate organization from the News people) very much regret ever having accepted the work, as well they might. Opinion seems divided about whether they could have caught the problems with the papers themselves - this was one of those cases where a number of reputable co-authors, at reputable institutions, all screwed up simultaneously when it came to cross-checking and verification. What remains is a portrait of how eager people can be to send in groundbreaking results for publication, and how eager editors can be to publish it. Neither of those are going to change any time soon, are they?

Update: from the comments, see also this timeline of events for a look at the whole story.

Comments (14) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

July 1, 2014

Scientific Journals: Who Pays What?

Email This Entry

Posted by Derek

If you've ever wondered about those deals where the large scientific publishers offer bundled discounts to libraries, wonder no more. There's a paper in PNAS whose authors used Freedom of Information Act requests to track down what various university libraries really paid for these deals, and it reveals that everyone paid something different.

Here's a comment in Nature on the study, which they can do with a straight face, since the Nature Publishing Group wasn't included in the study (although the authors seem to think, in retrospect, that they should have done so). These deals are always secret - the publishers make it a requirement not to disclose the terms. And that, as you might easily expect, benefits the publishers, since the library systems don't have a good way of finding out what the market price might be. The PNAS study reveals some odd discrepancies, with some universities getting noticeably better (and worse) deals than others. Wisconsin and Texas bargained hard, it appears, while BYU and Georgia could have done better for themselves.

As the article details, publishers used site licenses to take care of arbitrage opportunities, and the "Big Deal" bundles were used as incentives for the library systems and as tools for the publishers to figure out how much each customer might be willing to pay (using the print-based subscription data as a starting point). As you might have guessed, Elsevier comes out at the top of the pricing list when you just look at the dollar figures. On a cost-per-citation basis, though, they don't look so bad - in fact, they're the most cost-effective of the big publishers by that metric. (Sage and Taylor & Francis both look pretty bad in that table). For reference, the ACS bundle looks pretty decent, and it turns out that nearly 60% of the libraries that deal with the ACS choose the whole package (a high percentage compared to many other publishers). Interestingly, it turns out that some very wealthy schools (Harvard, MIT, Caltech) still don't take the full Elsevier bundle.

And the bundles are, naturally, a mixed bag. It's their whole purpose to be a mixed bag:

It would cost about $3.1 million at 2009 á la carte prices to buy all of the journals in Elsevier’s bundle, the “Freedom Collection.” The average research 1 university paid roughly $1.2 million, or 40% of the summed title-by- title prices, for access to the Freedom Collection. However, this bundle price is by no means equivalent to a 60% discount from journal-by-journal pricing. The Freedom Collection includes about 2,200 journals, many of which are expensive but rarely cited. The least cost-effective 1,100 journals contained in this collection supply fewer than 5% of the citations, but their prices add to more than 25% of the total of á la carte prices. A library that spent $1.2 million on Elsevier journals at listed catalog prices, selecting journals for cost-effectiveness, could obtain access to journals providing 79% of the citations to journals found in the Freedom Collection. Thus, for the average research 1 institution, the citation-scaled discount obtained from the Freedom Collection is about 21%.

Elsevier, though, drops its prices for smaller universities more quickly than many other publishers, and for Master's-level schools it's actually a better deal than many of the nonprofit publishers. We wouldn't know this, though, if these authors hadn't dug up all the info from FOIA requests, and I guess that's the take-home here: scientific publishing is a very opaque, inefficient market. And the publishers like it that way.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

June 27, 2014

Varieties of Scientific Deception

Email This Entry

Posted by Derek

Some may remember a paper from 2011 on the "reverse click" reaction, an interesting one where triazoles were pulled apart with mechanical force. This was an interesting system, because we really know surprisingly little, down on the molecular level, about what happens when bonds are physically stressed in this way. What do molecular orbitals look like when you grab both ends of the molecule and tug hard? Which bonds break first, and why? Do you get the reverse of the forward reaction, or do different mechanisms kick in (free radical intermediates, etc.)? (Note that the principle of microscopic reversibility doesn't necessarily apply when the conditions change like this).

Unfortunately, there seems to be trouble associated with this example. Science has an editorial "expression of concern" on the paper now, and it appears that much of it is not, in fact, reproducible (see this report in C&E News).

The paper was from the Bielawski lab at UT-Austin, and Bielawski is reported as saying that a former group member has confessed to manipulating data. But he also says that the conclusions of the paper are unchanged, which is interesting. My guess is that the "unclick" does happen, then, but nowhere as smoothly as reported. Someone may have sweetened things to make it all look better. At any rate, a correction is coming soon in Science, so we should get more information at that point.

This reminds me of the scheme I use to rate political and economic corruption. Stage I is paying someone off to do something they wouldn't normally do (or aren't authorized to do) for you. This happens everywhere, to some extent. Stage II is when you're bribing them just to do the job they're supposed to be doing in the first place. Many countries suffer from institutional cases of this, and it's supremely annoying, and a terrible drag on the economy. And Stage III, the worst, is when you're paying them not to harm you - a protection racket with the force of law behind it. Cynics may adduce examples from the US, but I'm thinking about countries (Russia, among others) where the problem is far worse.

Similar levels apply to fakery in the scientific literature. Here's how I break it down:

Stage I is what we may have in this case: actual conclusions and effects are made to look cleaner and better than reality. Zapping solvent peaks in the NMR is a perfect small-scale example of this - for organic chemists, solvent peaks are sometimes the training wheels of fakery. The problem is, once you're used to altering data, at what point do you find it convenient to stop? It's far better not to take that first step into matters-of-degree territory.

Stage II is unfortunately common as well, and there's a nice slippery path from Stage I that can land you here. This is when you're convinced that your results are correct, but you're having such a hard time getting things to work that you decide to "fake it until you make it". That's a stupendously bad idea, of course, because a lot of great results were never real in the first place, which leaves you hung out to dry, and even the ones that can be finally filled in don't have to do so in the way that you were faking them to happen. So now a real result is tainted by deception, which will call the whole thing into doubt when the inconsistencies become clear. And faked results are faked results, even if they're done in what you might think is a good cause. Many big cases of scientific fraud have started off this way, with someone just trying to fill in that one little gap, just for now.

Stage III, the bottom, is when something is faked from beginning to end. There was no question of it even working in the first place - it never did. Someone's just trying to get a paper, or a degree, or tenure, or fame, or something, and they're taking the shortcut. I think that there are two main classes of fakery in this category. In one group (IIIa?), you have people whipping up bogus results in low-profile cases where no one may notice for years, if ever, because no one cares. And you have IIIb, the famous high-profile cases (see Jan-Hendrik Schön, among too many others) where impressive, splashy, look-at-that stuff turns out to have been totally faked as well. Those cases are a study in human psychology. If you report a big result in superconductors, stem cells, cancer therapy or any other field where a lot of smart, competent people are paying very close attention, you will be found out at some point. How can you not be? We're in Bernie Madoff territory here, where someone comes into work every day of every week knowing that their whole reputation is a spray-painted scrim of deception that could have a hole punched through it any minute. How people can possibly live this way I don't really know, but people do. The self-confidence displayed by this sort of personality is a wonder of nature, in its way. IIIa cases are initiated by the desperate, stupid, and/or venal. IIIb cases, though, are brought on by people born to their task.

Update: as pointed out by several good comments, there are plenty of not-quite-fraud sins that neighbor these. Those are worth a separate post, partly because they're even more common than straight-up fraud.

Comments (54) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 6, 2014

The Robustness Test: Why Don't We Just Require It?

Email This Entry

Posted by Derek

There's a follow-up paper to that one on the robustness of new synthetic methods that I blogged about here. This one's in Nature Protocols, so it's a detailed look at just how you'd go about their procedure for shaking down a new reaction.

The reaction they've chosen is a rhodium-catalyzed indole formation (phenylhydrazine plus substituted alkyne), which is a good test bed for the real world (heterocycles, metal-catalyzed mechanism). The authors suggest a matrix of additives and reaction conditions, analyzed by GC, as in their original paper, to profile what can be tolerated and what can't. It's good to have the detailed reference out there, and I hope it gives referees and journal editors something to point at.

But will they? I can imagine a world where new reactions all have a "standard additives" grid somewhere in the paper, showing how the yields change. You could even color-code them (the old stoplight slide scheme, red/yellow/green, would be fine), and then we'd all have a way to compare synthetic methods immediately. Aldrich and others could sell the pre-assembled kit of the standard compounds to use. This would also point out reactions where more useful work could be done, since it would be immediately obvious that the new Whatsit Cyclization fails in the presence of tertiary amines, etc. Too often now you have to work that our for yourself, usually by seeing what the authors left out.

So why don't we all do this? It's more work, that's for sure, but not an incredible amount more work. If the major synthetic chemistry journals starting asking for it, that would be that. It would also make the publication landscape even more clear, because the titels that don't ask for an extra few days to be spent on the reaction conditions would be hard-pressed to make a case that they weren't just venues for hackwork (or for people with something to hide). I'd rather read about reactions with a clear statement of what they'll work on and what they won't - wouldn't you?

Comments (13) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

June 2, 2014

No More Acid Stem Cells

Email This Entry

Posted by Derek

In case you hadn't seen it, the "acid-washed stem cells" business has gone as far into the dumper as it can possibly go. It now appears that the whole thing was a fraud, from start to finish - if that's not the case, I'll be quite surprised, anyway. The most senior author of the (now retracted) second paper, Teruhiko Wakayama, has said that he doesn't believe its results:

The trigger, he told Bioscience, was his discovery—which he reported to Riken a few weeks ago--that two key photos in the second paper were wrong. Obokata, lead author on both papers, had in April been found by Riken guilty of misconduct on the first paper: the falsification of a gel electrophoresis image proving her starting cells were mature cells, and the fabrication of images proving resulting STAP stem cells could form the three major tissue types of the body.

But Riken had not yet announced serious problems with the second paper.

Last week, however, there was a flurry of activity in the Japanese press, as papers reported that two photos—supposed to show placenta made from STAP cells, next to placenta made from embryonic stem (ES) cells—were actually photos of the same mouse placenta.

As with so many cases before this one, we now move on (as one of Doris Lessing's characters once put it) to having interesting thoughts about the psychology of lying. How and why someone does this sort of thing is, I'm relieved to say, apparently beyond me. The only way I can remotely see it is if these results were something that a person thought were really correct, but just needed a bit more work, which would be filled in in time to salvage everything. But how many times have people thought that? And how does it always seem to work out? I'm back to being baffled. The stem cell field has attracted its share of mentally unstable people, and more.

Comments (13) + TrackBacks (0) | Category: Biological News | The Dark Side | The Scientific Literature

May 28, 2014

The Science Chemogenomics Paper is Revised

Email This Entry

Posted by Derek

The Science paper on chemogenomic signatures that I went on about at great length has been revised. Figure 2, which drove me and every other chemist who saw it up the wall, has been completely reworked:

To improve clarity, the authors revised Fig. 2 by (i) illustrating the substitution sites of fragments; (ii) labeling fragments numerically for reference to supplementary materials containing details about their derivation; and (iii) representing the dominant tautomers of signature compounds. The authors also discovered an error in their fragment generation software that, when corrected, resulted in slightly fewer enriched fragments being identified. In the revised Fig. 2, they removed redundant substructures and, where applicable, illustrated larger substructures containing the enriched fragment common among signature compounds.

Looking it over in the revised version, it is indeed much improved. The chemical structures now look like chemical structures, and some of the more offensive "pharmacophores" (like tetrahydrofuran) have now disappeared. Several figures and tables have been added to the supplementary material to highlight where these fragments are in the active compounds (Figure S25, an especially large addition), and to cross-index things more thoroughly.

So the most teeth-gritting parts of the paper have been reworked, and that's a good thing. I definitely appreciate the work that the authors have put into making the work more accurate and interpretable, although these things really should have been caught earlier in the process.

Looking over the new Figure S25, though, you can still see what I think are the underlying problems with the entire study. That's the one where "Fragments that are significantly enriched in specific sets of signature compounds (FDR ≤ 0.1 and signature compounds fraction ≥ 0.2) are highlighted in blue within the relevant signature compounds. . .". It's a good idea to put something like that in there, but the annotations are a bit odd. For example, the compounds flagged as "6_cell wall" have their common pyridines highlighted, even though there's a common heterocyclic core that that all but one those pyridines are attached to (it only varies by alkyl substitutents). That single outlier compound seems to be the reason that the whole heterocycle isn't colored in - but there are plenty of other monosubstituted pyridines on the list that have completely different signatures, so it's not like "monosubstituted pyridine" carries much weight. Meanwhile, the next set ("7_cell wall") has more of the exact same series of heterocycles, but in this case, it's just the core heterocycle that's shaded in. That seems to be because one of them is a 2-substituted isomer, while the others are all 3-substituted, so the software just ignores them in favor of coloring in the central ring.

The same thing happens with "8_ubiquinone biosynthesis and proteosome". What gets shaded in is an adamantane ring, even though every single one of the compounds is also a Schiff base imine (which is a lot more likely to be doing something than the adamantane). But that functional group gets no recognition from the software, because some of the aryl substitution patterns are different. One could just as easily have colored in the imine, though, which is what happens with the next category ("9_ubiquinone biosynthesis and proteosome"), where many of the same compounds show up again.

I won't go into more detail; the whole thing is like this. Just one more example: "12_iron homeostasis" features more monosubstituted pyridines being highlighted as the active fragment. But look at the list: there's are 3-aminopyridine pieces, 4-aminomethylpyridines, 3-carboxylpyridines, all of them substituted with all kinds of stuff. The only common thread, according to the annotation software, is "pyridine", but those are, believe me, all sorts of different pyridines. (And as the above example shows, it's not like pyridines form some sort of unique category in this data set, anyway).

So although the most eye-rolling features of this work have been cleaned up, the underlying medicinal chemistry is still pretty bizarre, at least to anyone who knows any medicinal chemistry. I hate to be this way, but I still don't see anyone getting an awful lot of use out of this.

Comments (6) + TrackBacks (0) | Category: Biological News | Chemical Biology | Chemical News | The Scientific Literature

May 14, 2014

A Unique Correction

Email This Entry

Posted by Derek

A reader sent this correction notice along this morning, and I agree: I've never seen anything quite like it. Here's how it goes (emphasis added):

According to the recent reconsideration and re-evaluation of every author’s contribution of this work, all of authors have reached the final conclusion that we should rearrange the order of the authors and remove Dr. Usui from the author list as observed below.

(List of authors follows)

All of authors have already approved this correction. Dr. Usui, our supervisor, also authorized and approved it.

There's surely a story here, but I'm sure that I don't quite know what it is, either!

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

April 22, 2014

J. Appl. Drivel or Gibberish Lett.? Choices, Choices.

Email This Entry

Posted by Derek

People keep hoaxing the predatory "scholarly" publishers out there, and the publishers keep falling for whatever drivel is slung at them. Here's the latest example from a reporter at the Ottawa Citizen, Tom Spears. He molded a pile of steaming gibberish into the rough shape of a manuscript, and that was more than enough:

I have just written the world’s worst science research paper: More than incompetent, it’s a mess of plagiarism and meaningless garble. . .

. . .I copied and pasted one phrase [in the title] from a geology paper online, and the rest from a medical one, on hematology.

I wrote the whole paper that way, copying and pasting from soil, then blood, then soil again, and so on. There are a couple of graphs from a paper about Mars. They had squiggly lines and looked cool, so I threw them in.

Footnotes came largely from a paper on wine chemistry. The finished product is completely meaningless.

The university where I claim to work doesn’t exist. Nor do the Nepean Desert or my co-author. Software that catches plagiarism identified 67 per cent of my paper as stolen (and that’s missing some). And geology and blood work don’t mix, even with my invention of seismic platelets.

And you guessed it - the acceptances came rolling in, and pretty damned quickly, too. Peer-reviewed, refereed, and edited within 24 hours - where are you going to find an honest journal with service like that? 16 of the 18 bottom-feeding "journals" accepted it, and one of the rejections suggested that it just needed a bit of tweaking to be accepted. And one of the publishers has asked Spears to serve on an editorial advisory board, so he's clearly got what it takes.

Of course, as yesterday's post shows, even a peer-reviewed journal with a recognizable name can publish gibberish. But I assume that Drug Discovery Today and Elsevier didn't charge the author $1000 to do it. On the other hand, they might have taken more than 18 hours to review the manuscript. Or not.

Comments (14) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

April 21, 2014

Molecular Printing of Drug Molecules. Say What?

Email This Entry

Posted by Derek

Update: the author of this paper has appeared in the comments here (and elsewhere) saying that he's withdrawing the paper. These are apparently reviewer's comments on it, although I have no way of verifying that. Many of them don't sound like the comments I might have expected. There's more here as well.

Here we have one of the oddest papers to appear in Drug Discovery Today, which is saying something. The journal has always ranged wider than some of the others in this space, but this is the furthest afield I've seen to date. The title is "DrugPrinter: print any drug instantly", and I don't think I can do better than letting the abstract speak for itself:

In drug discovery, de novo potent leads need to be synthesized for bioassay experiments in a very short time. Here, a protocol using DrugPrinter to print out any compound in just one step is proposed. The de novo compound could be designed by cloud computing big data. The computing systems could then search the optimal synthesis condition for each bond–bond interaction from databases. The compound would then be fabricated by many tiny reactors in one step. This type of fast, precise, without byproduct, reagent-sparing, environmentally friendly, small-volume, large-variety, nanofabrication technique will totally subvert the current view on the manufactured object and lead to a huge revolution in pharmaceutical companies in the very near future.

Now, you may well read that and ask yourself "What is this DrugPrinter, and how can I get one?" But note how it's all written in the conditional - lots of woulds and coulds, which should more properly be mights and maybes. Or maybe nots. The whole thing is a fantasy of atomic-level nanotechnology, which I, too, hope may be possible at some point. But to read about the DrugPrinter, you'd think that someone's ready to start prototyping. But no one is, believe me. This paper "tells" you all the "steps" that you would need to "print" a molecule, but it leaves out all the details and all the hard parts:

Thus, if DrugPrinter can one day become a reality it will be a huge step forward in drug discovery. The operator needs only to sit down in front of a computer and draw the structure of compound, which is then inputted into the computer, and the system will automatically search by cloud computing for suitable reaction conditions between bond and bond. . .

That actually captures the tone of this paper pretty well - it exists on a slightly different plane of reality, and what it's doing in Drug Discovery Today is a real mystery, because there's not much "Today" in it, for one thing. But there's something else about it, too - try this part out and see what you think:

Thus, this novel protocol only needs one step instead of the five-to-ten steps of the current synthesis process. In actual fact, it is even better than click chemistry, with lower costs and with better precision of synthesis. A world-leading group led by Lee Cronin has made advances with the technology named ‘Chemputer’. However, it is different to our concept. We specifically address the detail of how to pick up each atom and react. We also disagree that it is possible for anyone to simply download the software (app) from the internet and use it to print one's own drug. It is not feasible and should be illegal in the future.

Some of this, naturally, can be explained by non-native English usage, although the editorial staff at DDT really should have cleaned that up a bit. But there's an underlying strain of grandiose oddness about the whole manuscript. It makes for an interesting reading experience, for sure.

The paper proposes a molding process to fit the shape of the desired target molecule, which is not prima facie a crazy idea at all (templated synthesis). But remember, we're down on the atomic scale here. The only thing to build the mold out of is more atoms, at the same scale as the material filling the mold, and that's a lot harder than any macroscale molding process that you can make analogies to. The MIP (molecularly imprinted polymer) idea is the closest real-world attempt at this sort of thing, but it's been around for quite a while now without providing a quick route into molecular assembly. There is no quick route into molecular assembly, and you’re certainly not going to get one from stuff like this:

Benzene has six carbon atoms joined in a ring, with one hydrogen atom attached to each carbon atom. It can be divided into six reactors for three atoms: C, H and C (Fig. 3). After inputting the chemical structure of benzene, the system will search for the best synthesis condition for each bond. The best optimal condition will be elucidated by computer and controlled by a machine with optical tweezers to pick up the reactant and the atoms of carbon and hydrogen. The carbon atom will be picked up by optical tweezers in the right position in these tiny reactors (just like a color laser printer). DrugPrinter technology will work just like a color laser printer but instead of a four-color (red, yellow, blue and black) printer toner cartridge there will be various atoms.

Right. The computer will search for the best reaction conditions for building up benzene by individual carbon atoms? There are no best conditions for that. You can make benzene from acetylene, if you’re so minded, but you need metal catalysts (Reppe chemistry). And how are these “conditions” to work inside some sort of benzene-shaped mold? How are the intermediates (propene? butadiene?) to be held in there while another carbon atom comes in? Making benzene in this manner would be Nobel-level stuff, and this paper’s just getting warmed up:

. . .The chamber for the storage of elements is divided into three parts based on the character of each atom according to the periodic table of elements. Roughly, there are three categories: nonmetals, metals and transition metals. Of course, most drugs are organic compounds, thus it is reasonable to expect that carbon (C), hydrogen (H) and oxygen (O) will be the major consumables (just as the black toner cartridge always runs out before the other three colors in a printer). . .

I don’t know what the author’s background is, but honestly, you get the impression that it doesn’t include much organic chemistry. The whole paper is written about a world where you take individual atoms from these reservoirs and fly them down small channels “with lasers or plasma” to be caught by optical tweezers and put into the right position. Apparently, things are just going to snap together like so many molecular model pieces once that happens. Reaction mechanisms, thermodynamics, reactivity and selectivity make no appearance at all that I can see. What does make an appearance is stuff like this:

Big data is applied suddenly in any field. For DrugPrinter, we allow the user to upload their desired compound by a webserver. A cloud computing system and fast dealing and optimal of the chemical reaction must be searched immediately. All the bond–bond reactions will be collected in an intelligent system by cloud computing. Because we built a world-first intelligent cloud computing drug screening system called iScreen ( and an integrated webserver ( including the world’s largest traditional Chinese medicine (TCM) database (, this has enabled our technology. . .

I’m not trying to be unkind here, but some of this reads rather like the spam comments that pile up on this blog and others. “The buzzword will be made by high-tech buzzword by use of buzzword systems”. None of this is real, and as speculation it’s not too exciting, either. Eric Drexler is far more interesting reading – you can certainly find many things to argue about with him (as Richard Smalley famously did), but he’s thought about these problems in a useful way, as have many others. Drexler’s name, by the way, appears nowhere in this current paper, although the whole thing reads like a smudged tenth-generation photocopy of his work from the 1980s.

And that brings up an editorial question: who reviewed this? How did the staff at Drug Discovery Today find this worth publishing in its current form? I have no problem with them running papers about speculative nanotech chemical synthesis, I should add. I like that stuff; I like reading about it. But I don’t like reading hand-waving hoo-hah illustrated with videos of traditional egg-cake molding machines (I kid you not). As published, I found this paper to be an irritating, head-shaking, eye-rolling waste of time, and I would gladly have said so in a referee report. I see that Chemjobber is baffled as well. Who wouldn’t be?

Comments (46) | Category: Chemical News | The Scientific Literature

April 16, 2014

Professor Fukuyama's Solvent Peaks

Email This Entry

Posted by Derek

See Arr Oh expresses some doubts about all the NMR spectral corrections we've been seeing lately. He's specifically referring to Bethany Halford's interview piece, and he has this to say after reading it:

If your group focuses on "clean up your spectra" more than "purify your compounds better," that's a communications issue. If a professor with a large group sees nothing but perfect spectra all day, two thoughts should crop up:

1. "I must have the smartest, most efficient students in the world," or...
2. "Something's fishy here."

Perfect-looking data should always be a cause for concern in any experiment. My guess is that Prof. Fukuyama was closer to Option One, though, possibly in the variant of "My group has such high standards!" But high standards or not, a series of perfect, flat, NMR spectra with no solvent and no impurities is rather hard to achieve in total synthesis, considering the quantities that are being used. Load up the tube with 50mg of material and you can make a lot of stuff look good, but you don't have fifty mgs at step thirty-four, do you? I remember putting everything I had into one NMR tube (or worse, one polarimeter tube) in my own total synthesis days, and I carried the thing down to the machine like it was a bag of gold.

But there's no doubt that in a big group, there will be people who try to slip things past the boss. I've seen it myself; I'm sure that a lot of you have. And if you're giving the boss exactly what the boss wants to see - perfection - then it's going to be a lot easier. These spectral problems look like a collaborative effort to me - expectations from above, willingness from below. And there are a lot of other groups that have done (and, I feel sure, still do) the same thing. Zapping the solvent peaks in the NMR is the least of it, in some cases.

Update: added a direct link to the Fukuyama/Yokoshima interview.

Comments (19) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

April 14, 2014

More on the Science Chemogenomic Signatures Paper

Email This Entry

Posted by Derek

This will be a long one. I'm going to take another look at the Science paper that stirred up so much comment here on Friday. In that post, my first objection (but certainly not my only one) was the chemical structures shown in the paper's Figure 2. A number of them are basically impossible, and I just could not imagine how this got through any sort of refereeing process. There is, for example, a cyclohexadien-one structure, shown at left, and that one just doesn't exist as such - it's phenol, and those equilibrium arrows, though very imbalanced, are still not drawn to scale.
Well, that problem is solved by those structures being intended as fragments, substructures of other molecules. But I'm still positive that no organic chemist was involved in putting that figure together, or in reviewing it, because the reason that I was confused (and many other chemists were as well) is that no one who knows organic chemistry draws substructures like this. What you want to do is put dashed bonds in there, or R groups, as shown. That does two things: it shows that you're talking about a whole class of compounds, not just the structure shown, and it also shows where things are substituted. Now, on that cyclohexadienone, there's not much doubt where it's substituted, once you realize that someone actually intended it to be a fragment. It can't exist unless that carbon is tied up, either with two R groups (as shown), or with an exo-alkene, in which case you have a class of compounds called quinone methides. We'll return to those in a bit, but first, another word about substructures and R groups.
Figure 2 also has many structures in it where the fragment structure, as drawn, is a perfectly reasonable molecule (unlike the example above). Tetrahydrofuran and imidazole appear, and there's certainly nothing wrong with either of those. But if you're going to refer to those as common fragments, leading to common effects, you have to specify where they're substituted, because that can make a world of difference. If you still want to say that they can be substituted at different points, then you can draw a THF, for example, with a "floating" R group as shown at left. That's OK, and anyone who knows organic chemistry will understand what you mean by it. If you just draw THF, though, then an organic chemist will understand that to mean just plain old THF, and thus the misunderstanding.

If the problems with this paper ended at the level of structure drawing, which many people will no doubt see as just a minor aesthetic point, then I'd be apologizing right now. Update: although it is irritating. On Twitter, I just saw that someone spotted "dihydrophyranone" on this figure, which someone figured was close enough to "dihydropyranone", I guess, and anyway, it's just chemistry. But they don't. It struck me when I first saw this work that sloppiness in organic chemistry might be symptomatic of deeper trouble, and I think that's the case. The problems just keep on coming. Let's start with those THF and imidazole rings. They're in Figure 2 because they're supposed to be substructures that lead to some consistent pathway activity in the paper's huge (and impressive) yeast screening effort. But what we're talking about is a pharmacophore, to use a term from medicinal chemistry, and just "imidazole" by itself is too small a structure, from a library of 3200 compounds, to be a likely pharmacophore. Particularly when you're not even specifying where it's substituted and how. There are all kinds of imidazole out there, and they do all kinds of things.
So just how many imidazoles are in the library, and how many caused this particular signature? I think I've found them all. Shown at left are the four imidazoles (and there are only four) that exhibit the activity shown in Figure 2 (ergosterol depletion / effects on membrane). Note that all four of them are known antifungals - which makes sense, given that the compounds were chosen for the their ability to inhibit the growth of yeast, and topical antifungals will indeed do that for you. And that phenotype is exactly what you'd expect from miconazole, et al., because that's their known mechanism of action: they mess up the synthesis of ergosterol, which is an essential part of the fungal cell membrane. It would be quite worrisome if these compounds didn't show up under that heading. (Note that miconazole is on the list twice).
But note that there are nine other imidazoles that don't have that same response signature at all - and I didn't even count the benzimidazoles, and there are many, although from that structure in Figure 2, who's to say that they shouldn't be included? What I'm saying here is that imidazole by itself is not enough. A majority of the imidazoles in this screen actually don't get binned this way. You shouldn't look at a compound's structure, see that it has an imidazole, and then decide by looking at Figure 2 that it's therefore probably going to deplete ergosterol and lead to membrane effects. (Keep in mind that those membrane effects probably aren't going to show up in mammalian cells, anyway, since we don't use ergosterol that way).

There are other imidazole-containing antifungals on the list that are not marked down for "ergosterol depletion / effects on membrane". Ketonconazole is SGTC_217 and 1066, and one of those runs gets this designation, while the other one gets signature 118. Both bifonazole and sertaconazole also inhibit the production of ergosterol - although, to be fair, bifonazole does it by a different mechanism. It gets annotated as Response Signature 19, one of the minor ones, while sertaconazole gets marked down for "plasma membrane distress". That's OK, though, because it's known to have a direct effect on fungal membranes separate from its ergosterol-depleting one, so it's believable that it ends up in a different category. But there are plenty of other antifungals on this list, some containing imidazoles and some containing triazoles, whose mechanism of action is also known to be ergosterol depletion. Fluconazole, for example, is SGTC_227, 1787 and 1788, and that's how it works. But its signature is listed as "Iron homeostasis" once and "azole and statin" twice. Itraconzole is SGTC_1076, and it's also annotated as Response Signature 19. Voriconazole is SGTC_1084, and it's down as "azole and statin". Climbazole is SGTC_2777, and it's marked as "iron homeostasis" as well. This scattering of known drugs between different categories is possibly and indicator of this screen's ability to differentiate them, or possibly an indicator of its inherent limitations.

Now we get to another big problem, the imidazolium at the bottom of Figure 2. It is, as I said on Friday, completely nuts to assign a protonated imidazole to a different category than a nonprotonated one. Note that several of the imidazole-containing compounds mentioned above are already protonated salts - they, in fact, fit the imidazolium structure drawn, rather than the imidazole one that they're assigned to. This mistake alone makes Figure 2 very problematic indeed. If the paper was, in fact, talking about protonated imidazoles (which, again, is what the authors have drawn) it would be enough to immediately call into question the whole thing, because a protonated imidazole is the same as a regular imidazole when you put it into a buffered system. In fact, if you go through the list, you find that what they're actually talking about are N-alkylimidazoliums, so the structure at the bottom of FIgure 2 is wrong, and misleading. There are two compounds on the list with this signature, in case you were wondering, but the annotation may well be accurate, because some long-chain alkylimidazolium compounds (such as ionic liquid components) are already known to cause mitochondrial depolarization.

But there are several other alkylimidazolium compounds in the set (which is a bit odd, since they're not exactly drug-like). And they're not assigned to the mitochondrial distress phenotype, as Figure 2 would have you think. SGTC_1247, 179, 193, 1991, 327, and 547 all have this moeity, and they scatter between several other categories. Once again, a majority of compounds with the Figure 2 substructure don't actually map to the phenotype shown (while plenty of other structural types do). What use, exactly, is Figure 2 supposed to be?

Let's turn to some other structures in it. The impossible/implausible ones, as mentioned above, turn out to be that way because they're supposed to have substituents on them. But look around - adamantane is on there. To put it as kindly as possible, adamantane itself is not much of a pharmacophore, having nothing going for it but an odd size and shape for grease. Tetrahydrofuran (THF) is on there, too, and similar objections apply. When attempts have been made to rank the sorts of functional groups that are likely to interact with protein binding sites, ethers always come out poorly. THF by itself is not some sort of key structural unit; highlighting it as one here is, for a medicinal chemist, distinctly weird.

What's also weird is when I search for THF-containing compounds that show this activity signature, I can't find much. The only things with a THF ring in them seem to be SGTC_2563 (the complex natural product tomatine) and SGTC_3239, and neither one of them is marked with the signature shown. There are some imbedded THF rings as in the other structural fragments shown (the succinimide-derived Diels-Alder ones), but no other THFs - and as mentioned, it's truly unlikely that the ether is the key thing about these compounds, anyway. If anyone finds another THF compound annotated for tubulin folding, I'll correct this post immediately, but for now, I can't seem to track one down, even though Table S4 says that there are 65 of them. Again, what exactly is Figure 2 supposed to be telling anyone?

Now we come to some even larger concerns. The supplementary material for the paper says that 95% of the compounds on the list are "drug-like" and were filtered by the commercial suppliers to eliminate reactive compounds. They do caution that different people have different cutoffs for this sort of thing, and boy, do they ever. There are many, many compounds in this collection that I would not have bothered putting into a cell assay, for fear of hitting too many things and generating uninterpretable data. Quinone methides are a good example - as mentioned before, they're in this set. Rhodanines and similar scaffolds are well represented, and are well known to hit all over the place. Some of these things are tested at hundreds of micromolar.

I recognize that one aim of a study like this is to stress the cells by any means necessary and see what happens, but even with that in mind, I think fewer nasty compounds could have been used, and might have given cleaner data. The curves seen in the supplementary data are often, well, ugly. See the comments section from the Friday post on that, but I would be wary of interpreting many of them myself.
There's another problem with these compounds, which might very well have also led to the nastiness of the assay curves. As mentioned on Friday, how can anyone expect many of these compounds to actually be soluble at the levels shown? I've shown a selection of them here; I could go on. I just don't see any way that these compounds can be realistically assayed at these levels. Visual inspection of the wells would surely show cloudy gunk all over the place. Again, how are such assays to be interpreted?

And one final point, although it's a big one. Compound purity. Anyone who's ever ordered three thousand compounds from commercial and public collections will know, will be absolutely certain that they will not all be what they say on the label. There will be many colors and consistencies, and LC/MS checks will show many peaks for some of these. There's no way around it; that's how it is when you buy compounds. I can find no evidence in the paper or its supplementary files that any compound purity assays were undertaken at any point. This is not just bad procedure; this is something that would have caused me to reject the paper all by itself had I refereed it. This is yet another sign that no one who's used to dealing with medicinal chemistry worked on this project. No one with any experience would just bung in three thousand compounds like this and report the results as if they're all real. The hits in an assay like this, by the way, are likely to be enriched in crap, making this more of an issue than ever.

Damn it, I hate to be so hard on so many people who did so much work. But wasn't there a chemist anywhere in the room at any point?

Comments (39) + TrackBacks (0) | Category: Biological News | Chemical Biology | Chemical News | The Scientific Literature

April 11, 2014

Biology Maybe Right, Chemistry Ridiculously Wrong

Email This Entry

Posted by Derek

Note: critique of this paper continues here, in another post.

A reader sent along a puzzled note about this paper that's out in Science. It's from a large multicenter team (at least nine departments across the US, Canada, and Europe), and it's an ambitious effort to profile 3250 small molecules in a broad chemogenomics screen in yeast. This set was selected from an earlier 50,000 compounds, since these realiably inhibited the growth of wild-type yeast. They're looking for what they call "chemogenomic fitness signatures", which are derived from screening first against 1100 heterozygous yeast strains, one gene deletion per, representing the yeast essential genome. Then there's a second round of screening against 4800 homozygous deletions strain of non-essential genes, to look for related pathways, compensation, and so on.

All in all, they identified 317 compounds that appear to perturb 121 genes, and many of these annotations are new. Overall, the responses tended to cluster in related groups, and the paper goes into detail about these signatures (and about the outliers, which are naturally interested for their own reasons). Broad pathway effects like mitrochondrial stress show up pretty clearly, for example. And unfortunately, that's all I'm going to say for now about the biology, because we need to talk about the chemistry in this paper. It isn't good.

phenol.pngAs my correspondent (a chemist himself) mentions, a close look at Figure 2 of the paper raises some real questions. Take a look at that cyclohexadiene enamine - can that really be drawn correctly, or isn't it just N-phenylbenzylamine? The problem is, that compound (drawn correctly) shows up elsewhere in Figure 2, hitting a completely different pathway. These two tautomers are not going to have different biological effects, partly because the first one would exist for about two molecular vibrations before it converted to the second. But how could both of them appear on the same figure?

And look at what they're calling "cyclohexa-2,4-dien-1-one". No such compound exists as such in the real world - we call it phenol, and we draw it as an aromatic ring with an OH coming from it. Thiazolidinedione is listed as "thiazolidine-2,4-quinone". Both of these would lead to red "X" marks on an undergraduate exam paper. It is clear that no chemist, not even someone who's been through second-year organic class, was involved in this work (or at the very least, involved in the preparation of Figure 2). Why not? Who reviewed this, anyway?

There are some unusual features from a med-chem standpoint as well. Is THF really targeting tubulin folding? Does adamantane really target ubiquinone biosynthesis? Fine, these are the cellular effects that they noted, I guess. But the weirdest thing on Figure 2's annotations is that imidazole is shown as having one profile, while protonated imidazole is shown as a completely different one. How is this possible? How could anyone who knows any chemistry look at that and not raise an eyebrow? Isn't this assay run in some sort of buffered medium? Don't yeast cells have any buffering capacity of their own? Salts of basic amine drugs are dosed all the time, and they are not considered - ever - as having totally different cellular effects. What a world it would be if that were true! Seeing this sort of thing makes a person wonder about the rest of the paper.

Nitro.pngMore subtle problems emerge when you go to the supplementary material and take a look at the list of compounds. It's a pretty mixed bag. The concentrations used for the assays vary widely - rapamycin gets run at 1 micromolar, while ketoconazole is nearly 1 millimolar. (Can you even run that compound at that concentration? Or that compound at left at 967 micromolar? Is it really soluble in the yeast wells at such levels? There are plenty more that you can wonder about in the same way.

And I went searching for my old friends, the rhodanines, and there they were. Unfortunately, compound SGTC_2454 is 5-benzylidenerhodanine, whose activity is listed as "A dopamine receptor inhibitor" (!). But compound SGTC_1883 is also 5-benzylidenerhodanine, the same compound, run at similar concentration, but this time unannotated. The 5-thienylidenerhodanine is SGTC_30, but that one's listed as a phosphatase inhibitor. Neither of these attributions seem likely to me. There are other duplicates, but many of them are no doubt intentional (run by different parts of the team).

I hate to say this, but just a morning's look at this paper leaves me with little doubt that there are still more strange things buried in the chemistry side of this paper. But since I work for a living (dang it), I'm going to leave it right here, because what I've already noted is more than troubling enough. These mistakes are serious, and call the conclusions of the paper into question: if you can annotate imidazole and its protonated form into two different categories, or annotate two different tautomers (one of which doesn't really exist) into two different categories, what else is wrong, and how much are these annotations worth? And this isn't even the first time that Science has let something like this through. Back in 2010, they published a paper on the "Reactome" that had chemists around the world groaning. How many times does this lesson need to be learned, anyway?

Update: this situation brings up a number of larger issues, such as the divide between chemists and biologists (especially in academia?) and the place of organic chemistry in such high-profile publications (and the place of organic chemists as reviewers of it). I'll defer these to another post, but believe me, they're on my mind.

Update 2 Jake Yeston, deputy editor at Science, tells me that they're looking into this situation. More as I hear it.

Update 3: OK, if Figure 2 is just fragments, structural pieces that were common to compounds that had these signatures, then (1) these are still not acceptable structures, even as fragments, and (2), many of these don't make sense from a medicinal chemistry standpoint. It's bizarre to claim a tetrahydrofuran ring (for example) as the key driver for a class of compounds; the chance that this group is making an actual, persistent interaction with some protein site (or family of sites) is remote indeed. The imidazole/protonated imidazole pair is a good example of this: why on Earth would you pick these two groups to illustrate some chemical tendency? Again, this looks like the work of people who don't really have much chemical knowledge.

0560-0053.pngA closer look at the compounds themselves does not inspire any more confidence. There's one of them from Table S3, which showed a very large difference in IC50 across different yeast strains. It was tested at 400 micromolar. That, folks, was sold to the authors of this paper by ChemDiv, as part of a "drug-like compound" library. Try pulling some SMILES strings from that table yourself and see what you think about their drug likeness.

Comments (131) + TrackBacks (0) | Category: Chemical Biology | Chemical News | The Scientific Literature

April 3, 2014

More Fukuyama Corrections

Email This Entry

Posted by Derek

The Fukuyama group has another series of corrections out, this time in JACS. Here's one, and the other follow right behind it in the ASAP queue. This adds to the string of them in Organic Letters. It's more whiteout stuff - vanishing solvent peaks and impurities. These presumably don't affect the conclusions of the paper, but they don't make a person any more confident, either. One hopes that these high-profile cases will shake people up. . .

Comments (24) + TrackBacks (0) | Category: The Scientific Literature

March 28, 2014

More on the UT-Austin Retraction Case

Email This Entry

Posted by Derek

I mentioned an unusual retraction from Organic Letters here last year, and here's some follow-up to the story:

Nearly six years after Suvi Orr received a Ph.D. in chemistry from the University of Texas, the university told her it has decided to do something that institutions of higher learning almost never do: revoke
the degree. Orr, in turn, has sued UT in an effort to hold onto the doctorate that launched her career in the pharmaceutical industry.

Her lawsuit in state district court in Travis County contends that revocation is unwarranted and that the university violated her rights by not letting her defend herself before the dissertation committee that condemned her research long after she graduated. In addition, she says, the committee relied heavily on her former professor, who, she claims, was motivated to “cast the blame elsewhere.”

What a mess. More details as things develop. . .

Comments (17) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

March 21, 2014

Dosing by Body Surface Area

Email This Entry

Posted by Derek

We were talking about allometry around here the other day, which prompts me to mention this paper. It used the reports of resveratrol dosing in animals, crudely extrapolated to humans, to argue that the body surface area normalization (BSA) method was a superior technique for dose estimation across species.

Over the years, though, the BSA method has taken some flak in the literature. It's most widely used in oncology, especially with cytotoxics, but there have been calls to move away from the practice, calling it a relic with little scientific foundation. (The rise of a very obese patient population has also led to controversy about whether body weight or surface area is a more appropriate dose-estimation method in those situations). At the same time, it's proven useful in some other situations, so it can't be completely ignored.

But it seems that the FASEB paper referenced in the first paragraph, which has been cited hundreds of times since 2008, may be overstating its conclusions. For example, it says that "BSA normalization of doses must be used to determine safe starting doses of new drugs because initial studies conducted in humans, by definition, lack formal allometric comparison of the pharmacokinetics of absorption, distribution, and elimination parameters", and cites its reference 13 for support. But when you go to that reference, you find that paper's authors concluding with things like this:

The customary use of BSA in dose calculations may contribute to the omission of these factors, give a false sense of accuracy and introduce error. It is questionable whether all current cancer treatment strategies are near optimal, or even ethical. BSA should be used for allometric scaling purposes in phase I clinical trials, as the scaling of toxicity data from animals is important for selecting starting doses in man, but the gradual discontinuation of BSA-based dosing of cytotoxic drugs in clinical practice is seemingly justified.

Citing a paper for support that flatly disagrees with your conclusions gets some points for bravado, but otherwise seems a bit odd. And there are others - that reference that I linked to in the second paragraph above, under "taken some flak", is cited in the FASEB paper as its reference 17, as something to do with choosing between various BSA equations. And it does address that, to be sure, but in the context of wondering whether the whole BSA technique has any clinical validity at all.

This is currently being argued out over at PubPeer, and it should be interesting to see what comes of it. I'll be glad to hear from pharmacokinetics and clinical research folks to see what they make of the whole situation.

Comments (17) + TrackBacks (0) | Category: Pharmacokinetics | The Scientific Literature

March 20, 2014

Years Worth of the Stuff

Email This Entry

Posted by Derek

bAP15.pngThis time last year I mentioned a particularly disturbing-looking compound, sold commercially as a so-called "selective inhibitor" of two deubiquitinase enzymes. Now, I have a fairly open mind about chemical structures, but that thing is horrible, and if it's really selective for just those two proteins, then I'm off to truck-driving school just like Mom always wanted.

Here's an enlightening look through the literature at this whole class of compound, which has appeared again and again. The trail seems to go back to this 2001 paper in Biochemistry. By 2003, you see similar motifs showing up as putative anticancer agents in cell assays, and in 2006 the scaffold above makes its appearance in all its terrible glory.

The problem is, as Jonathan Baell points out in that post, that this series has apparently never really had a proper look at its SAR, or at its selectivity. It wanders through a series of publications full of on-again off-again cellular readouts, with a few tenuous conclusions drawn about its structure - and those are discarded or forgotten by the time the next paper comes around. As Baell puts it:

The dispiriting thing is that with or without critical analysis, this compound is almost certainly likely to end up with vendors as a “useful tool”, as they all do. Further, there will be dozens if not hundreds of papers out there where entirely analogous critical analyses of paper trails are possible.

The bottom line: people still don’t realize how easy it is to get a biological readout. The more subversive a compound, the more likely this is. True tools and most interesting compounds usually require a lot more medicinal chemistry and are often left behind or remain undiscovered.

Amen to that. There is way too much of this sort of thing in the med-chem literature already. I'm a big proponent of phenotypic screening, but setting up a good one is harder than setting up a good HTS, and working up the data from one is much harder than working up the data from an in vitro assay. The crazier or more reactive your "hit" seems to be, the more suspicious you should be.

The usual reply to that objection is "Tool compound!" But the standards for a tool compound, one used to investigate new biology and cellular pathways, are higher than usual. How are you going to unravel a biochemical puzzle if you're hitting nine different things, eight of which you're totally unaware of? Or skewing your assay readouts by some other effect entirely? This sort of thing happens all the time.

I can't help but think about such things when I read about a project like this one, where IBM's Watson software is going to be used to look at sequences from glioblastoma patients. That's going to be tough, but I think it's worth a look, and the Watson program seems to be just the correlation-searcher for the job. But the first thing they did was feed in piles of biochemical pathway data from the literature, and the problem is, a not insignificant proportion of that data is wrong. Statements like these are worrisome:

Over time, Watson will develop its own sense of what sources it looks at are consistently reliable. . .if the team decides to, it can start adding the full text of articles and branch out to other information sources. Between the known pathways and the scientific literature, however, IBM seems to think that Watson has a good grip on what typically goes on inside cells.

Maybe Watson can tell the rest of us, then. Because I don't know of anyone actually doing cell biology who feels that way, not if they're being honest with themselves. I wish the New York Genome Center and IBM luck in this, and I still think it's a worthwhile thing to at least try. But my guess is that it's going to be a humbling experience. Even if all the literature were correct in every detail, I think it would be one. And the literature is not correct in every detail. It has compounds like that one at the top of the entry in it, and people seem to think that they can draw conclusions from them.

Comments (19) + TrackBacks (0) | Category: Biological News | Cancer | Chemical Biology | Drug Assays | The Scientific Literature

March 19, 2014

More Things Synthetic Chemists Hate

Email This Entry

Posted by Derek

I enjoyed this post over at Synthetic Remarks on "Five things synthetic chemists hate". And I agree; I hate all of 'em, too. Allow me to add a few to the list:

1. The Mysterious Starting Material. How many times have you looked through an experimental section only to see a synthesis start cold, from a non-commercial compound whose preparation isn't given, or even referenced? One that doesn't seem to have any foundation anywhere else in the literature, either? I think that this is a bit more common in the older literature, but it shouldn't be happening anywhere.

2. It Works on Benzaldehyde; What More Do You Want? What about those new method papers that include a wide, diverse array of examples showing how versatile the new reaction is - but when you look at the list, you realize that it's full of things like cyclohexanone, benzaldehyde. . .and then 4-methylcyclohexanone, p-fluorobenzaldehyde, and so on? Turns out that the reaction lands flat on its nose, stretched out on the sand if there's a basic amine within five hundred yards. But you have to find that out for yourself. It ain't in the text.

3. The Paper Chase. In these days of humungous supplementary info files, what excuse is there to write a paper where all the reactions use one particular reagent - and then send people back to your previous paper to learn how to make it? Sure, reference yourself. But don't march everyone back to a whole other experimental. Are authors getting some sort of nickel-a-page-view deal from the publishers now that I haven't heard about?

4. If I Don't See It, It Isn't There. When I review papers, one of the things I end up dinging people about, more than anything else, is the reluctance to cite relevant literature. In some cases, it's carelessness, but in others, well. . .everyone's seen papers that basically rework someone else's reaction without ever citing the original. And in these days of modern times, as the Firesign Theatre guys used to say, what excuse is there?

5. Subtle Is the Lord. Once in a while, you find an experimental writeup that makes you wrinkle your brow and wonder if someone's pulling your leg. The reaction gets run at -29 degrees C, for 10.46 hours, whereupon it's brought up to -9 and quenched with pH 7.94 buffer solution. That kind of thing. If you're going to put that Proustian level of detail in there, you'd better have a reason (Proust did). No one just stumbles into conditions like that - what happened when you ran your reaction like a normal human, instead of like Vladimir Nabokov on Adderall?

Comments (39) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

March 17, 2014

What If Total Syntheses Had Only 25 Steps to Work In?

Email This Entry

Posted by Derek

Via the Baran lab's Twitter feed, here's a provocative article on whether total organic synthesis has a place in the modern world or not.

One may wonder why this situation has passed undisputed for such a long time. Currently however, wide parts of the chemical community look upon total synthesis as a waste of time, resources and talents. Behind the scene, it may even be argued that the obsession to synthesize almost any natural product irrespective of its complexity and practical importance has blocked the development of other more relevant fields. Therefore, it is high time to consider a reorientation of the entire discipline

That's a bit of a straw man in that paragraph, and I have to note it, even though I do feel odd sticking up for total synthesis (about which I've been pretty caustic myself, for many years now.). I don't think that there's been an "obsession to synthesize almost any natural product", although it's true that many new synthetic methods have used some natural product or another as demonstration pieces. But the author, Johann Mulzer, came out of the Corey group in the old days, and has spent his career doing total synthesis, so he's speaking from experience here.

He goes on to argue that the field does have a place, but that it had better shape up. Short syntheses have to take priority over "first syntheses", because (let's face it), just about anything can be made if you're willing to throw enough time, money, and postdocs at it. The paper is full of examples from Mulzer's own career (and others'), and if you read it carefully, you'll see some unfavorable contrasts drawn to some Nicolaou syntheses. He finishes up:

In conclusion, this article tries to show how various strategies may be used to streamline and to shorten otherwise long synthetic routes to complex target molecules. The reader may get the impression that it pays very well to think intensively about cascade reactions, intramolecular cycloadditions, suitable starting materials and so on, instead of plunging into a brute-force and therefore mostly inefficient sequence. After all, there is an iron maxim: if a target cannot be reached within, say, 25 steps, it is better to drop it. For what you will get is a heroic synthesis, at best, but never an efficient one.

A 25-step limit would chop an awful lot out of the synthetic literature, wouldn't it? But it's not fair to apply that retrospectively. What if we apply it from here on out, though? What would the total synthetic landscape look like then?

Comments (46) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

March 14, 2014

Going After Poor Published Research

Email This Entry

Posted by Derek

This should be interesting - John Ioannidis, scourge of poorly reproducible published results, is founding an institute at Stanford. The Economist has more:

They will create a “journal watch” to monitor scientific publishers’ work and to shame laggards into better behaviour. And they will spread the message to policymakers, governments and other interested parties, in an effort to stop them making decisions on the basis of flaky studies. All this in the name of the centre’s nerdishly valiant mission statement: “Identifying and minimising persistent threats to medical-research quality.”

It will be most interesting to see what comes of this. Better quality research is in everyone's best interest, to put it mildly, and I hope that this leads to some.

Comments (13) + TrackBacks (0) | Category: The Scientific Literature

March 6, 2014

We Are Pleased To Publish Your Senseless Ravings

Email This Entry

Posted by Derek

There's been some (justified) hand-wringing in scientific publishing circles over the revelation that at least 120 abstracts and papers out there in the literature are complete nonsense generated by SciGen. (A few previous SciGen adventures can be found here and here) Some news reports have made it seem like these were regular full papers, but they're actually published conference proceedings which (frankly) are sort of the ugly stepchild of the science journal world to begin with. They're supposed to be reviewed, and they certainly should have been reviewed enough for someone to catch on to the fact that they were devoid of meaning, but if you're going to fill the pages of a reputable publisher with Aphasio-matic ramblings, that's the way to do it.

And these were reputable publishers, Springer and the IEEE. Springer has announced that they're removing all this stuff from their databases, since the normal retraction procedure doesn't exactly seem necessary. They're also trying to figure out what loophole let this happen in the first place, and they've contacted Cyril Labbé, the French researcher who wrote the SciGen-detecting software, for advice. The IEEE, for its part, has had this problem before, has had it for years, has been warned about it, but still seems to be ready and willing to publish gibberish. I don't know if Springer has had bad experiences with SciGen material, but the IEEE journals sure have, and it's apparently done no good at all. Live and don't learn. The organization has apparently removed the papers, but has made (as far as I can tell) no public statement whatsoever about the whole incident.

So who went to all this trouble, anyway? That Scholarly Kitchen link above has some speculations:

An additional (and even more disturbing) problem with the proceedings papers most recently discovered is emerging as the investigation continues: at least one of the authors contacted had no idea that he had been named as a coauthor. This suggests that the submissions were more than spoofs — spoofing can easily be accomplished by using fake names as well as fake content. The use of real scientists’ names suggests that at least some of these papers represent intentional scholarly fraud, probably with the intention of adding bulk to scholars’ résumés.

This takes us back to the open-access versus traditional publisher wars. When this sort of thing happens to OA journals, the response from some of the other publishers, overtly at times, is "Well, yeah, sure, that's what you get when you don't go with the name brand". And there's certainly a lot of weird crap that shows up - take a look at this thing, from the open access journal Cancer Research and Management, and see if you can make head or tail of it. Clozapine might have helped it a bit, but maybe not:

Our findings suggest that we are dealing with true reverse biologic system information in an activated collective cancer stem cell memory, in which physics participates in the elaboration of geometric complexes and chiral biomolecules that serve to build bodies with embryoid print as it develops during gestation.

That's a little more coherent than what SciGen will give you, but not so coherent as to, you know, make any sense. And if by any chance you were OK with that extract, the rest of the abstract mentions comets and the Large Hadron Collider, so take that. It lacks only gyres (don't we all?) But what this latest incident tells us is that this paper would have waltzed right in at plenty of other publishers, open-access or not. The authors should be aiming higher, y'know?

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

March 3, 2014

Sydney Brenner on the State of Science

Email This Entry

Posted by Derek

Via Retraction Watch, here's an outspoken interview with Sydney Brenner, who's never been the sort of person to keep his opinions bottled up inside him. Here, for example, are his views on graduate school in the US:

Today the Americans have developed a new culture in science based on the slavery of graduate students. Now graduate students of American institutions are afraid. He just performs. He’s got to perform. The post-doc is an indentured labourer. We now have labs that don’t work in the same way as the early labs where people were independent, where they could have their own ideas and could pursue them.

The most important thing today is for young people to take responsibility, to actually know how to formulate an idea and how to work on it. Not to buy into the so-called apprenticeship. I think you can only foster that by having sort of deviant studies. That is, you go on and do something really different. Then I think you will be able to foster it.

But today there is no way to do this without money. That’s the difficulty. In order to do science you have to have it supported. The supporters now, the bureaucrats of science, do not wish to take any risks. So in order to get it supported, they want to know from the start that it will work. This means you have to have preliminary information, which means that you are bound to follow the straight and narrow.

I can't argue with that. In academia these days, it seems to me that the main way that something really unusual or orthogonal gets done is by people doing something else with their grant money than they told people they'd do. Which has always been the case to some extent, but I get the impression it's more so than ever. The article also quotes from Brenner's appreciation of the late Fred Sanger, where he made a similar point:

A Fred Sanger would not survive today’s world of science. With continuous reporting and appraisals, some committee would note that he published little of import between insulin in 1952 and his first paper on RNA sequencing in 1967 with another long gap until DNA sequencing in 1977. He would be labelled as unproductive, and his modest personal support would be denied. We no longer have a culture that allows individuals to embark on long-term—and what would be considered today extremely risky—projects.

Here are Brenner's mild, temperate views on the peer-review system and its intersection with academic publishing:

. . .I don’t believe in peer review because I think it’s very distorted and as I’ve said, it’s simply a regression to the mean.

I think peer review is hindering science. In fact, I think it has become a completely corrupt system. It’s corrupt in many ways, in that scientists and academics have handed over to the editors of these journals the ability to make judgment on science and scientists. There are universities in America, and I’ve heard from many committees, that we won’t consider people’s publications in low impact factor journals.

Now I mean, people are trying to do something, but I think it’s not publish or perish, it’s publish in the okay places [or perish]. And this has assembled a most ridiculous group of people. I wrote a column for many years in the nineties, in a journal called Current Biology. In one article, “Hard Cases”, I campaigned against this [culture] because I think it is not only bad, it’s corrupt. In other words it puts the judgment in the hands of people who really have no reason to exercise judgment at all. And that’s all been done in the aid of commerce, because they are now giant organisations making money out of it.

I don't find a lot to disagree with there, either. The big scientific publishers have some good people working for them, but the entire cause is more and more suspect. THere's a huge moral hazard involved, which we don't seem to be avoiding very well at all.

Comments (35) + TrackBacks (0) | Category: General Scientific News | The Scientific Literature

February 26, 2014

Changes in Papers

Email This Entry

Posted by Derek

Here's a look at some of the changes in JACS papers over the decades. Several trends are clear - there are more authors now, and single-author papers have almost vanished. Reference lists are much longer (which surely reflects both the sie of the literature and the relative ease of bibliography compared to the bound-volume/index card days). Have a look at the charts - François-Xavier Coudert, the blog's author, says that he'll be putting up some more later on, and I look forward to seeing what comes up.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

February 20, 2014

The NIH Takes a Look At How the Money's Spent

Email This Entry

Posted by Derek

The NIH is starting to wonder what bang-for-the-buck it gets for its grant money. That's a tricky question at best - some research takes a while to make an impact, and the way that discoveries can interact is hard to predict. And how do you measure impact, by the way? These are all worthy questions, but here's apparently the way things are being approached:

Michael Lauer's job at the National Institutes of Health (NIH) is to fund the best cardiology research and to disseminate the results rapidly to other scientists, physicians, and the public. But NIH's peer-review system, which relies on an army of unpaid volunteer scientists to prioritize grant proposals, may be making it harder to achieve that goal. Two recent studies by Lauer, who heads the Division of Cardiovascular Sciences at NIH's National Heart, Lung, and Blood Institute (NHLBI) in Bethesda, Maryland, raise some disturbing questions about a system used to distribute billions of dollars of federal funds each year.

(MiahcalLauer recently analyzed the citation record of papers generated by nearly 1500 grants awarded by NHLBI to individual investigators between 2001 and 2008. He was shocked by the results, which appeared online last month in Circulation Research: The funded projects with the poorest priority scores from reviewers garnered just as many citations and publications as those with the best scores. That was the case even though low-scoring researchers had been given less money than their top-rated peers.

I understand that citations and publications are measurable, while most other ways to gauge importance aren't. But that doesn't mean that they're any good, and I worry that the system is biased enough already towards making these the coin of the realm. This sort of thing worries me, too:

Still, (Richard) Nakamura is always looking for fresh ways to assess the performance of study sections. At the December meeting of the CSR advisory council, for example, he and Tabak described one recent attempt that examined citation rates of publications generated from research funded by each panel. Those panels with rates higher than the norm—represented by the impact factor of the leading journal in that field—were labeled "hot," while panels with low scores were labeled "cold."

"If it's true that hotter science is that which beats the journals' impact factors, then you could distribute more money to the hot committees than the cold committees," Nakamura explains. "But that's only if you believe that. Major corporations have tried to predict what type of science will yield strong results—and we're all still waiting for IBM to create a machine that can do research with the highest payoff," he adds with tongue in cheek.

"I still believe that scientists ultimately beat metrics or machines. But there are serious challenges to that position. And the question is how to do the research that will show one approach is better than another."

I'm glad that he doesn't seem to be taking this approach completely seriously, but others may. If only impact factors and citation rates were real things that advanced human knowledge, instead of games played by publishers and authors!

Comments (34) + TrackBacks (0) | Category: The Scientific Literature | Who Discovers and Why

February 19, 2014

Robo-Combing the Chemistry Literature For Mistakes

Email This Entry

Posted by Derek

This is a very timely post indeed from Peter Murray-Rust. He's describing a system that his group has developed (ChemVisitor) to dig through the chemical literature looking for incorrect structures (and much more).

He shows examples from an open-access paper, in which one of the structures is in fact misdrawn. But how would Elsevier, Nature, the ACS, Wiley or the other big publishers take to having these things highlighted every day of the week. Not well:

So try it for yourself. Which compound is wrong? (*I* don’t know yet) How would you find out? Maybe you would go to Chemical Abstracts (ACS). Last time I looked it cost 6USD to look up a compound. That’s 50 dollars, just to check whether the literature is right. And you would be forbidden from publishing what you found there (ACS sent the lawyers to Wikipedia for publishing CAS registry numbers). What about Elsevier’s Reaxys? Almost certainly as bad.

But isn’t there an Open collection of molecules? Pubchem in the NIH? Yes, and ACS lobbied on Capitol Hill to have it shut down as it was “socialised science instead of the private sector”. They nearly won. (Henry Rzepa and I ran a campaign to highlight the issue). So yes, we can use Pubchem and we have and that’s how Andy’s software discovered the mistake.

This was the first diagram we analysed. Does that mean that every paper in the literature contains mistakes?

Almost certainly yes.

But they have been peer-reviewed.

Yes – and we wrote software (OSCAR) 10 years ago that could do the machine reviewing. And it showed mistakes in virtually every paper.

So we plan to do this for every new paper. It’s technically possible. But if we do it what will happen?

If I sign the Elsevier content-mining click-through (I won’t) then I agree not to disadvantage Elsevier’s products. And pointing out publicly that they are full of errors might just do that. And if I don’t?…

This comment on Ycombinator is from someone who's seen some of the Murray-Rost group's software in action, and is very interesting indeed:

They can take an ancient paper with very low quality diagrams of complex chemical structures, parse the image into an open markup language and reconstruct the chemical formula and the correct image. Chemical symbols are just one of many plugins for their core software which interprets unstructured, information rich data like raster diagrams. They also have plugins for phylogenetic trees, plots, species names, gene names and reagents. You can develop plugins easily for whatever you want, and they're recruiting open source contributors (see,

As a side effect of how their software works, it can detect tiny suggestive imperfections in images that reveal scientific fraud. I was shown a demo where a trace from a mass spec (like this was analysed. As well as reading the data from the plot, it revealed a peak that had been covered up with a square - the author had deliberately obscured a peak in their data that was inconvenient. Scientific fraud. It's terrifying that they find this in most chemistry papers they analyse.

Peter's group can analyse thousands or hundreds of thousands of papers an hour, automatically detecting errors and fraud. . .

Unless I'm very much mistaken, we'll be hearing a lot more about this. It touches on the quality of the literature, the quality of the people writing the papers, and the business model(s) of the people publishing it all. And these are very, very relevant topics that are are getting more important all the time. . .

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

February 18, 2014

Thoughts on Overpublishing

Email This Entry

Posted by Derek

My recent memorial notice for Alan Katritzky brought an interesting comment, by someone who did the math that I didn't. His total publication count actually seems to be 2215 papers published since 1953. That comes to one paper every ten days over sixty-one years. I have trouble even imagining that. This is a vast amount of work, both the chemistry and the writing, and it's a monument that very, very few people will leave behind them.

Should anyone, though? I mean no disrespect to Katritzky's memory by asking this question, let me note quickly. But I wrote about this here a few years ago, the idea that a person can, in fact, publish too many papers. My example then was H. C. Brown - I'm not sure how many papers he co-authored, but the figure is a large one and brings on similar thoughts.

There have been many scientists on the other end of the scale, going back to Isaac Newton, who had to be badgered into letting everyone know that he'd revolutionized physics. Lars Onsager is a good example from the physics/chemistry borderlands - there are stories told about the work he had stored away in his filing cabinets, a terrifying stockpile that most other people would have found very much to their credit to have published. This approach is clearly not the way to go, either - people don't have a chance to build on your work when they don't know about it, and others may spend a good amount of effort duplicating things when you could (and should) have saved them the effort.

But publishing thousands of papers doesn't seem like a good alternative, to be honest. People will have trouble sorting out the good parts, and they can't all be good parts. Time will settle that question, you might think, but time could bungle that job. Time has dropped the ball before.

Update: See Arr Oh did a very interesting list last year of the organic chemists who publish the most. Katritzky did indeed lead the field.

Comments (46) + TrackBacks (0) | Category: The Scientific Literature

Not Again - Stem Cell Results in Trouble?

Email This Entry

Posted by Derek

Oh, @#$!. That was my first comment when I saw this story. That extraordinary recent work on creating stem cells by subjected normal cells to acid stress is being investigated:

The RIKEN centre in Kobe announced on Friday that it is looking into alleged irregularities in the work of biologist Haruko Obokata, who works at the institution. She shot to fame last month as the lead author on two papers published in Nature that demonstrated a simple way to reprogram mature mice cells into an embryonic state by simply applying stress, such as exposure to acid or physical pressure on cell membranes. The RIKEN investigation follows allegations on blog sites about the use of duplicated images in Obokata’s papers, and numerous failed attempts to replicate her results.

PubPeer gets the credit for bringing some of the problems into the light. There are some real problems with figures in the two papers, as well as earlier ones from the same authors. These might be explicable as cimple mistakes, which is what the authors seem to be claiming, if it weren't for the fact that no one seems to be able to get the stem-cell results to reproduce. There are mitigating factors there, too - different cell lines, perhaps the lack of a truly detailed protocol from the original paper. But a paper should have enough details in it to be reproduced, shouldn't it?

Someone on Twitter was trying to tell me the other day that the whole reproducibility issue was being blown out of proportion. I don't think so. The one thing we seem to be able to reproduce is trouble.

Update: a list of the weirdest things (so far) about this whole business.

Comments (25) + TrackBacks (0) | Category: Biological News | The Scientific Literature

February 17, 2014

The Org Lett Spectral Hammer Falls Again

Email This Entry

Posted by Derek

I see that as of this moment, the five articles at the top of the Organic Letters ASAP feed are all corrections from the Nakada group. It looks to be the same situation as the Fukuyama corrections: NMR editing. The corrections (visible for free because you can see the first page) all mention that the conclusions of the papers are not altered, nor are the yields of products. So Amos Smith really is serious about his data-cleaning crusade, and I can see where he's coming from. Falsus in unum, falsus in omnibus. The only place to draw the line is right back at the start.

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

February 14, 2014

"It Is Not Hard to Peddle Incoherent Math to Biologists"

Email This Entry

Posted by Derek

Here's a nasty fight going on in molecular biology/bioinformatics. Lior Pachter of Berkeley describes some severe objections he has to published work from the lab of Manolis Kellis at MIT. (His two previous posts on these issues are here and here). I'm going to use a phrase that Pachter hears too often and say that I don't have the math to address those two earlier posts. But the latest one wraps things up in a form that everyone can understand. After describing what does look like a severe error in one of the Manolis group's conference presentations, which Pachter included in a review of the work, he says that:

. . .(they) spun the bad news they had received as “resulting from combinatorial connectivity patterns prevalent in larger network structures.” They then added that “…this combinatorial clustering effect brings into question the current definition of network motif” and proposed that “additional statistics…might well be suited to identify larger meaningful networks.” This is a lot like someone claiming to discover a bacteria whose DNA is arsenic-based and upon being told by others that the “discovery” is incorrect – in fact, that very bacteria seeks out phosphorous – responding that this is “really helpful” and that it “raises lots of new interesting open questions” about how arsenate gets into cells. Chutzpah. When you discover your work is flawed, the correct response is to retract it.

I don’t think people read papers very carefully. . .

He goes on to say:

I have to admit that after the Grochow-Kellis paper I was a bit skeptical of Kellis’ work. Not because of the paper itself (everyone makes mistakes), but because of the way he responded to my review. So a year and a half ago, when Manolis Kellis published a paper in an area I care about and am involved in, I may have had a negative prior. The paper was Luke Ward and Manolis Kellis “Evidence for Abundant and Purifying Selection in Humans for Recently Acquired Regulatory Functions”, Science 337 (2012) . Having been involved with the ENCODE pilot, where I contributed to the multiple alignment sub-project, I was curious what comparative genomics insights the full-scale $130 million dollar project revealed. The press releases accompanying the Ward-Kellis paper (e.g. The Nature of Man, The Economist) were suggesting that Ward and Kellis had figured out what makes a human a human; my curiosity was understandably piqued.

But a closer look at the paper, Pachter says, especially a dig into the supplementary material (always a recommended move) shows that the conclusions of the paper were based on what he terms "blatant statistically invalid cherry picking". See, I told you this was a fight. He also accuses Kellis of several other totally unacceptable actions in his published work, the sorts of things that cannot be brushed off as differences in interpretations or methods. He's talking fraud. And he has a larger point about how something like this might persist in the computational biology field (emphasis added):

Manolis Kellis’ behavior is part of a systemic problem in computational biology. The cross-fertilization of ideas between mathematics, statistics, computer science and biology is both an opportunity and a danger. It is not hard to peddle incoherent math to biologists, many of whom are literally math phobic. For example, a number of responses I’ve received to the Feizi et al. blog post have started with comments such as

“I don’t have the expertise to judge the math, …”

Similarly, it isn’t hard to fool mathematicians into believing biological fables. Many mathematicians throughout the country were recently convinced by Jonathan Rothberg to donate samples of their DNA so that they might find out “what makes them a genius”. Such mathematicians, and their colleagues in computer science and statistics, take at face value statements such as “we have figured out what makes a human human”. In the midst of such confusion, it is easy for an enterprising “computational person” to take advantage of the situation, and Kellis has.

You can peddle incoherent math to medicinal chemists, too, if you feel the urge. We don't use much of it day-to-day, although we've internalized more than we tend to realize. But if someone really wants to sell me on some bogus graph theory or topology, they'll almost certainly be able to manage it. I'd at least give them the benefit of the doubt, because I don't have the expertise to call them on it. Were I so minded, I could probably sell them some pretty shaky organic chemistry and pharmacokinetics.

But I am not so minded. Science is large, and we have to be able to trust each other. I could sit down and get myself up to speed on topology (say), if I had to, but the effort required would probably be better spent doing something else. (I'm not ruling out doing math recreationally, just for work). None of us can simultaneously be experts across all our specialities. So if this really is a case of publishing junk because, hey, who'll catch on, right, then it really needs to be dealt with.

If Pachter is off base, though, then he's in for a rough ride of his own. Looking over his posts, my money's on him and not Kellis, but we'll all have a chance to find out. After this very public calling out, there's no other outcome.

Comments (32) + TrackBacks (0) | Category: Biological News | In Silico | The Dark Side | The Scientific Literature

February 12, 2014

Genius, Sheer Genius

Email This Entry

Posted by Derek

So we have bogus scientific journals out there, and we have people who will sell you results so you can make just-add-water papers for the real journals. And we have impact factors, the overuse of which leads to still more bogosity in every part of scientific publishing. So. . .why not sell fake impact factors? That way, you can harvest publication fees from credulous impact-seeking bozos. My Iranian wife tells me that there's a saying in Farsi that translates as "A thief robs a thief, and God smiles".

Comments (11) + TrackBacks (0) | Category: The Scientific Literature

February 11, 2014

Pesticide Toxicity?

Email This Entry

Posted by Derek

There's been a report on the toxicity of various pesticides in the literature suggesting that they're far more toxic to human cells than had been thought. My eyebrows went up a bit when I heard this, because these sorts of assays had been done many times before. Then I realized that this was another paper from the Séralini group, and unfortunately, that alone is enough to account for the variance.

Update: commentors to this post have noted that the cell culture conditions used in the paper are rather unusual. Specifically, they're serum-free during the testing period, which puts the cells under stress to begin with. There's also the general problem, which others have brought up, about what it means to dispense these things directly onto cell cultures in diluted DMSO, since that's rather far from how they're going to be presented in the real world. Cell assays get run like that in the drug industry, to be sure, but you've got to be very careful drawing toxicological or other whole-animal conclusions from them. And we already have whole-animal studies on these formulations, don't we? I mean, juiced broccoli straight from the organic farmer's market might well have similar effects under these conditions.

Here's a story from Science with more background. Seralini is the guy who made headlines a couple of years ago with another report that genetically modified corn caused tumors in rodents, but that one was so poorly run and poorly controlled that its conclusions (which have not been seen in any other study) cannot be taken seriously. That's Séralini's problem right there: from all appearances, he's a passionate advocate for his positions, and he appears to be ready to go with whatever results line up with his beliefs. This is human nature, for sure, but science is about trying to work past those parts of human nature. The key is to keep the curious, inquisitive side, and correct for the confirmation bias I-know-I'm-right side. At this point, even if Séralini were to discover something real (and really worth taking seriously), it would have a hard time gaining acceptance, because his previous papers have been so unreliably over-the-top.

I'm not the only person who thinks that. An editor of the journal this latest Seralini paper appeared in has actually resigned because it got published:

When Ralf Reski read the latest paper from controversial French biologist Gilles-Eric Séralini, he quickly decided he wanted nothing to do with it. Séralini’s report in BioMed Research International describes how pesticides kill cultured human cells, with the hair-raising conclusion that pesticides may be vastly more toxic than assumed by regulatory authorities. Some scientists are criticizing the findings as neither surprising nor significant—but they have touched off a firestorm, with environmental groups calling for changes in how pesticides are regulated. That was too much for Reski. Within hours of reading the paper last week, the plant scientist at the University of Freiburg in Germany resigned as an editor of the journal and asked for his name to be removed from its website. "I do not want to be connected to a journal that provides [Séralini] a forum for such kind of agitation," he wrote in his resignation e-mail to the publisher, Hindawi Publishing Corporation.

Should pesticide toxicity be a subject of investigation? Absolutely. Should people be alert to assays that have not been run that should be investigated? Definitely. Are there things that we don't know about pesticide exposure that we should? I would certainly think so. But Séralini's history makes him (scientifically) one of the least effective people to be working on these questions. As a headline-grabber, though, he's pretty efficient. Which I suspect is the real point. If you're sure you're right, any weapon you can pick up is a good one.

Comments (17) + TrackBacks (0) | Category: The Scientific Literature | Toxicology

February 10, 2014

Today We Have Naming of Parts

Email This Entry

Posted by Derek

I can strongly recommend this article by Carmen Drahl in C&E News on the way that we chemists pick fights over nomenclature. She has examples of several kinds of disagreement (competing terms for the same thing, terms that overlap but are still different, competing ways to measure something in different ways, and terms that are fuzzy enough that some want to eliminate them entirely).

As several of the interviewees note, these arguments are not (always) petty, and certainly not always irrational. Humans are good at reification - turning something into a "thing". Name a concept well, and it sort of shimmers into existence, giving people a way to refer to it as if it were a solid object in the world of experience. This has good and bad aspects. It's crucial to the ability to have any sort of intellectual discussion and progress, since we have to be able to speak of ideas and other entities that are not actual physical objects. But a badly fitting name can do real harm, obscuring the most valuable or useful parts of an idea and diverting thoughts about it unproductively.

My own favorite example is the use of "agonist" and "antagonist" to describe the actions of nuclear receptor ligands. This (to my way of thinking) is not only useless, but does real harm to the thinking of anyone who approaches nuclear receptors having first learned about GPCRs. Maybe the word "receptor" never should have been used for these things in the first place, although realizing that would have required supernatural powers of precognition.

There are any number of examples outside chemistry, of course. One of my own irritants is when someone says that something has been "taken to the next level". You would probably not survive watching a sports channel if that phrase were part of a drinking game. But it presupposes that some activity comes in measurable chunks, and that everyone agrees on what order they come in. I'm reminded of the old blenders with their dials clicking between a dozen arbitrary "levels", labeled with tags like "whip", "chop", and "liquify". Meaningless. It's an attempt to quantify - to reify - what should have been a smooth rheostat knob with lines around it.

OK, I'll stop before I bring up Wittgenstein. OK, too late. But he was on to something when he told people to be careful about the way language is used, and to watch out when you get out onto the "frictionless ice" of talking about constructions of thought. His final admonition in his Tractacus Logico-Philosophicus, that if we cannot speak about something that we have to pass over it in silence, has been widely quoted and widely unheeded, since we're all sure that we can, of course, speak about what we're speaking about. Can't we?

For today's post title, see here

Comments (20) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

A Timeline from Cell

Email This Entry

Posted by Derek

Here's a very interesting feature from Cell - an interactive timeline on the journal's 40th anniversary, highlighting some of the key papers it's published over the years. This installment takes us up into the early 1980s. When you see the 1979 paper that brings the news that tyrosine groups on proteins actually get phosphorylated post-translation, the 1982 discovery of Ras as involved in human cancer cells, or another 1982 paper showing that telomeres have these weird repeating units on them, you realize how young the sciences molecular and cell biology really are.

Comments (3) + TrackBacks (0) | Category: Biological News | The Scientific Literature

February 4, 2014

Put Down That White-Out; Drop That Photoshop Cursor

Email This Entry

Posted by Derek

I've had some e-mails about this today: an impressive series of six corrections in Organic Letters (scroll down here). I guess Amos Smith was serious about his editorial last year on manipulation of spectra. All of these corrections are for NMR cleanup of just that variety (disappearing solvent and impurity peaks, along with restated reaction yields). We'll see if it can encourager les autres.

Update: See Arr Oh asks if the Fukuyama lab jumped, or were they pushed?

Comments (44) + TrackBacks (0) | Category: The Scientific Literature

January 24, 2014

Are There Good Writers In This Business?

Email This Entry

Posted by Derek

Here's a question sent in by a reader that I actually have trouble answering: who would you say are the best writers of scientific papers in our field? I'm not necessarily talking about the results in those papers, as much as clear presentation and layout, written in a way that gets the point of the paper across without having to decode the thing.

It's tough, because writing a journal publication means operating under a variety of constraints. One of the biggest of those is word count (or page length). The fierce compression that this can bring on makes unpacking some papers a sentence-by-sentence excavation job, but it's not that the authors would write it like that if they weren't forced to. The opposite situation would be a disaster, too, though - there are, I'm sure, plenty of people who would just ramble on and on given unlimited page space to work with. Pascal's apology for writing such a long letter ("There was not enough time to write a shorter one") is germane. Rare is the first draft that can't be tightened up substantially, scientific publication or not.

But many journal articles are tightened up so hard that they twang when you try to read them. Maintaining clarity and flow under these conditions isn't easy, and I'd be interested to hear about people who manage to stand out enough to be noticed. And since this is a blog, and this is the internet, feel free to bring up examples from the other end of the scale - people whose papers are inevitably a chore. Nominees?

(My impression, by the way, is that well-written papers are more likely to be found in the older literature. I'm not sure if that's just selection bias, since we get to choose over a wider range of time and subject matter that way, or if journal editors were a bit looser about what they would allow back then. The original Watson and Crick DNA paper, for example, would surely never be written up that way today).

Update: here's a Curious Wavefunction post on this from a couple of years ago, with some of his own nominations.

Comments (43) + TrackBacks (0) | Category: The Scientific Literature

January 21, 2014

Throwing Out the Files

Email This Entry

Posted by Derek

Since I'm in the process of moving my office, I've been taking time to do something that's needed to be done for quite a while: cleaning out my files. Somewhere around 2007 or so, I made the switchover to keeping PDFs as my primary filing system, with paper copies when needed. There was a transitional period, which I ended up splicing together by checking through my recent printed copies and backfilling those into my digital archive, but after that, it was all digital. (For the record, I'm still using Zotero for that purpose, although there are several equally valid alternatives, both commercial and freeware).

But I still had a pretty massive filing cabinet full of stuff, and I let that remain undisturbed, even though I knew some of it was surely junk. Only when I started digging into it did I realize just how much of it was little more than that. I'd estimate that I've thrown at least 80% of my files into the recycling bin, an act that would have made me uneasy only a few years ago, and horrified me in, say, 2004. It was easier than I thought, though.

That's because the folders easily fell into several broad categories. In the medical/biological sections of the cabinet, there were "Topics I'm Unlikely to Revisit - And When I Do, It Won't Be With These References". Those went right into the recycling bin. And there were "Topics I May Well Revisit, But When I Do, It Won't Be With These References". Those, after a glance through their contents, went into the bin as well. These were folders on (for example) disease areas that I've worked on in the past, and might conceivably work on again, but a folder full of ten-year-old biomedical articles is not that useful compared to the space it takes up and the trouble it takes to move it. And if that sounds borderline to you, how about the ones that hadn't been updated since the late 1990s? Junk. Nothing in the literature goes out of date faster than a state-of-current-disease-research article.

Moving to the chemistry folders, I was quickly surprised at how many of those I was throwing away as well. The great majority of the printed papers I kept were chemistry ones, but the great majority of what I started out with went into the recycling bin anyway. Digging through them was, in many cases, a reminder of what keeping up with the literature used to be like, back in the day. It was a time when if you found a useful-looking paper, you copied it out and put it in your files, because there was no telling when or if you'd be able to find it again. If you were one of the supremely organized ones, you drew a key reaction or two on an index card and filed that according to some system of your own devising - that's before my time, but I saw people doing that back when I was a grad student. The same sort of pack-ratting persisted well into the 1990s, though, but eroded in the face of better access to Chemical Abstracts (and the rise of competing databases). Finding that reaction, or others like it, or even better references than the ones you knew about, became less and less of a big deal.

So in my files, over in the section for "Synthesis of Amines", there was a folder on the opening of epoxides by amines. And in it were several papers I'd copied in the late 1980s. And some printed-out hits from SciFinder searches in about 1993. And a couple of reactions that I'd seen at conferences, and a paper from 1997 showing how you could change the site of ring opening, sometimes, with some systems. Into the bin it went, despite the feeling (not an inaccurate one) that I was throwing away work that I'd put into assembling all that. But if I find myself wanting to run such a reaction, I can probably set something up that'll work fairly well, and if it doesn't, I can probably find a review article (or two) where someone else has assembled the previous literature.

One of the biggest problems with my chemistry files, I realized, was the difficulty of searching them. I'd gotten used to the world of SciFinder and Reaxsys and Google and PubMed, where information can be called up any way you like. File folders, though, do not speak of their contents. Unless you have the main points of that content committed to memory, you have to open them up and flip through them, hoping for something relevant to pop up. I can well remember doing that in the early 1990s with some of these very folders ("Hmm, let's see what methods I have for such-and-such"), but that style of searching disappeared many years ago. You can now see what methods everyone has, and quickly find out what's been added to the pile since the last time you looked. Younger researchers who've grown up in that world may find it odd that I'm pointing out that water is wet, but my earliest file-cabinet folders were started in another time. File folders are based on tagging (and in its purest form, a physical label), and I agree with people who say that the ability to search is more important and useful than the ability to tag.

So, what did I keep? Folders on specialized topics that I recalled were very difficult to assemble, in a few cases. Papers that I know that I've referred to several times over the years. Papers that refer directly to things that I'm currently working on. Some stuff that's so old that it falls under the category of memorabilia. And finally, papers on more current topics that I want to make sure that I also have in digital form, but didn't have time to check just now. But that three-inch-thick collection of nuclear receptor papers from 2000-2002? The papers on iron dienyl reagents that I copied off during a look at that chemistry in 1991, and never had a need to refer to after about ten days? A folder of reductive amination conditions from the late 1980s? Into the big blue bin with all of it.

Comments (23) + TrackBacks (0) | Category: Life in the Drug Labs | The Scientific Literature

January 20, 2014

A Long Fight Over Allegations of Fraud

Email This Entry

Posted by Derek

Here's a long article from the Raleigh News and Observer (part one and part two) on the Eaton/Feldheim/Franzen dispute in nanoparticles, which some readers may already be familiar with (I haven't covered it on the blog myself). The articles are clearly driven by Franzen's continued belief that research fraud has been committed, and the paper makes the most of it.

The original 2004 publication in Science claimed that RNA solutions could influence the crystal form of palladium nanoparticles, which opened up the possibility of applying the tools of molecular biology to catalysts and other inorganic chemistry applications. Two more papers in JACS extended this to platinum and looked at in vitro evolutionary experiments. But even by 2005, Franzen's lab (who had been asked to join the collaboration between Eaton and Feldheim, who were now at Colorado and a startup company) was generating disturbing data: the original hexagonal crystals (a very strange and interesting form for palladium) weren't pure palladium at all - on an elemental basis, they were mostly carbon. (Later work showed that they were unstable crystals of (roughly) Pd(dba)3, with solvated THF. And they were produced just as well in the negative control experiments, with no RNA added at all.

N. C. State investigated the matter, and the committee agreed that the results were spurious. But they found Feldheim guilty of sloppy work, rather than fraud, saying he should have checked things out more thoroughly. Franzen continued to feel as if justice hadn't been done, though:

In fall 2009, he spent $1,334 of his own money to hire Mike Tadych, a Raleigh lawyer who specializes in public records law and who has represented The News & Observer. In 2010, the university relented and allowed Franzen into the room where the investigation records were locked away.

Franzen found the lab notebooks, which track experiments and results. As he turned the pages, he recognized that Gugliotti kept a thorough and well-organized record.

“I found an open-and-shut case of research fraud,” Franzen said.

The aqueous solution mentioned in the Science article? The experiments routinely used 50 percent solvent. The experiments only produced the hexagonal crystals when there was a high level of solvent, typically 50 percent or more. It was the solvent creating the hexagonal crystals, not the RNA.

On Page 43 of notebook 3, Franzen found what he called a “smoking gun.”

(Graduate student Lina) Gugliotti had pasted four images of hexagonal crystals, ragged around the edges. The particles were degrading at room temperature. The same degradation was present in other samples, she noted.

The Science paper claimed the RNA-templated crystals were formed in aqueous solution with 5% THF and were stable. NC State apparently offered to revoke Gugliotti's doctorate (and another from the group), but the article says that the chemistry faculty objected, saying that the professors involved should be penalized, not the students. The university isn't commenting, saying that an investigation by the NSF is still ongoing, but Franzen points out that it's been going on for five years now, a delay that has probably set a record. He's published several papers characterizing the palladium "nanocrystals", though, including this recent one with one of Eaton and Feldheim's former collaborators and co-authors. And there the matter stands.

It's interesting that Franzen pursued this all the way to the newspaper (known when I Iived in North Carolina by its traditional nickname of the Nuisance and Disturber). He's clearly upset at having joined what looked like an important and fruitful avenue of research, only to find out - rather quickly - that it was based on sloppy, poorly-characterized results. And I think what really has him furious is that the originators of the idea (Feldheim and Eaton) have tried, all these years, to carry on as if nothing was wrong.

I think, though, that Franzen is having his revenge whether he realizes it or not. It's coming up on ten years now since the original RNA nanocrystal paper. If this work were going to lead somewhere, you'd think that it would have led somewhere by now. But it doesn't seem to be. The whole point of the molecular-biology-meets-materials-science aspect of this idea was that it would allow a wide variety of new materials to be made quickly, and from the looks of things, that just hasn't happened. I'll bet that if you went back and looked up the 2005 grant application for the Keck foundation that Eaton, Feldheim (and at the time, Franzen) wrote up, it would read like an alternate-history science fiction story by now.

Comments (14) + TrackBacks (0) | Category: Chemical News | Press Coverage | The Dark Side | The Scientific Literature

January 16, 2014

Should Drug Industry Research All Get Rejected For Publication?

Email This Entry

Posted by Derek

If you work in the drug industry, and for some reason you feel that your blood pressure isn't quite high enough today, a look at this debate at the British Medical Journal should fix that up for you. "Should journals stop publishing research funded by the drug industry?" is the title - there, doesn't that constrict your blood vessels already?

Taking the "Yes, they should" side are Richard Smith (former editor of the journal, now with a British organization called "Patients Know Best", and Peter C. Gøtzsche of the Nordic Cochrane Center. Here's their opening statement, and Gøtzsche's recent opinion piece in the same journal is a good harbinger, as it turns out:

The BMJ and its sibling journals have stopped publishing research funded by the tobacco industry for two main reasons: the research is corrupted and the companies publish their research to advance their commercial aims, oblivious of the harm they do. But these arguments apply even more strongly to research funded by the drug industry, and we suggest there is a better way to communicate the results of trials that would be safer for patients.

Prescribed drugs are the third leading cause of death, partly because of flaws in the evidence published in journals. We have long known that clinical trials funded by the drug industry are much more likely than publicly funded trials to produce results favourable to the company. The reason is obvious. The difference between an honest and a less than honest data analysis can be worth billions of euros, and the fraudulent trials of some cyclo-oxygenase-2 inhibitors for arthritis and selective serotonin reuptake inhibitors for depression are good examples

They're absolutely right about the financial motivations, and a first-rate moral hazard it is, too. But the comparison with the tobacco companies is a real pencil-snapper (as they no doubt intended it to be). They go on about prescription drugs being the "third largest cause of death", about "drug industry crimes", and so on. To be fair, and first let me brush these pencil fragments off my desk, the pharmaceutical companies have laid themselves wide open to these sorts of attacks, painting huge fluorescent bulls-eye targets on themselves again and again. But still.

This piece casually mentions that "olanzapine (Zyprexa), has probably caused 200 000 deaths", footnoting a book by one of the two authors. I seem to have missed that. Many antipsychotic drugs are associated with QT prolongation, which can lead to fatal heart arrythmias, but the worst of them have long been taken out of use. The FDA is investigating two deaths following injection of long-acting olanzapine, not two hundred thousand. Olanzapine has plenty of side effects, though, including weight gain (which can exacerbate Type II diabetes), and it has a warning label in the US about giving it to elderly patients under any conditions. But two hundred thousand deaths? I can't find any support for any such figure; it appears in Gøtzsche's book and apparently nowhere else, so citing it in this article as if it were a well-established fact is a nice move.

Taking the "No" side is Trish Groves of the BMJ itself. She rejects the analogy with the tobacco industry - as she should, because it's offensive and ridiculous. She goes on to detail the problems with industry-reported results and what the journal is doing about them. As opposed to the "Yes" side, it's a pretty reasonable piece. One of the things she mentions is that investigator-led trials have their own sources of bias. Very few people organizing an effort the size of a useful clinical trial will be disinterested in its results, unfortunately.

How much can we trust the evidence base for drugs in current use? It’s hard to tell, given the woeful legacy of widespread non-registration, non-publication, and selective reporting of clinical trials. Much of this reporting bias also applies to investigator led trials, and the many steps now being taken to mandate prospective trial registration, ensure reporting of all results, and access patient level data on interventions’ benefits and harms, as called for by the AllTrials campaign, must apply to them as much as to industry led trials. Moreover, new rules on transparency need to be applied retrospectively: laudable plans to provide access to data on new drugs aren’t enough.

That’s why the BMJ is keen to publish papers from the RIAT (Restoring Invisible and Abandoned Trials) initiative, through which academics who find previously unreported trials can write them up and publish them if the original investigators decline to do so. We also welcome “negative” trials that find no evidence of benefit, as long as their research questions are important and their methods robust, and we’re particularly interested in publishing trials of comparative effectiveness. Both these types of study can be much more useful to clinical practice than the placebo controlled trials that regulators demand. . .

It should be no great task to guess which side of this debate I favor - after all, I'm one of those evil drug company scientists who mow down the customers by the hundreds of thousands. I do wish that Groves' response had strayed a bit from the topic at hand and addressed those accusations of mass murder (that's what they are). I realize that it must be hard to tell a former editor to tone things down and go back for a rewrite. But still.

Comments (48) + TrackBacks (0) | Category: Clinical Trials | The Scientific Literature | Why Everyone Loves Us

January 7, 2014

How Much Is Wrong?

Email This Entry

Posted by Derek

Here's another take, from Jeff Leek at Simply Statistics, on the "How much published research is false?" topic. This one is (deliberately) trying to cut down on the alarm bells and flashing red lights.

Note that the author is a statistician, and the the arguments made are from that perspective. For example, the Amgen paper on problems with reproducibility of drug target papers is quickly dismissed with the phrase "This is not a scientific paper" (because it has no data), and the locus classicus of the false-research-results topic, the Ioannidis paper in PLoS Medicine, is seen off with the comment that "The paper contains no real data, it is purely based on conjecture and simulation."

I'll agree that we don't need to start assuming that everything is junk, as far as the eye can see. But I'm not as sanguine as Leek is, I think. Semi-anecdotal reports like the Amgen paper, the Bayer/Schering paper, and even scuttlebutt from Bruce Booth and the like are not statistically vetted scientific reports, true. But the way that they're all pointing in the same direction is suggestive. And it's worth keeping in mind that all of these parties have an interest in the answer being the opposite of what they're finding - we'd all like for the literature reports of great new targets and breakthroughs to be true.

The one report where Leek is glad about the mathematical underpinnings is the Many Labs project. But there's something about that that bothers me. The Many Labs people were trying to replicate results in experimental psychology, and while there's probably some relevance to the replications problems in biology and chemistry, there are big differences, too. I worry that everything is getting lumped together as Science, and if this part of Science is holding up, then those worries that people in other parts of Science have are probably ill-founded (after all, they don't have any real numbers, right?)

Comments (11) + TrackBacks (0) | Category: The Scientific Literature

January 2, 2014

It Just So Happens That I Have A Conference Right Here

Email This Entry

Posted by Derek

Here's a good addition to the "bogus conferences" file. The folks at Retraction Watch have the story of Navin Kabra, an Indian engineer who's blowing the whistle on a racket peculiar to that country.

There are apparently many universities in India that have a requirement that everyone attaining a certain degree has to have their work accepted at an "international conference". So. . .a number of "international conference" organizers have stepped up to fill that market niche, with hefty registration fees and talk of rigorous peer review and high standards. They do nothing of the kind, of course. People pay their cash, pay their own way to the conference, and get to present to a scattered audience of other people who've done the same thing. No one else shows up - why would anyone?

So Kabra sent them a manuscript full of gibberish and stretches of dialog from "My Cousin Vinny", and (you guessed it), the thing passed the brutal review process as soon as the cash appeared. After revealing his hoax, the paper seems to have been taken down from the conference web site, but up until then, it was available for interested scholars, or people interested in Joe Pesci and/or Marisa Tomei. As long as the universities pretend that everyone coming through their programs has done work that's fit to present, there will be people there who will pretend to hold conferences for them. The real losers are the students, many of whom apparently think that these are real meetings. How do you recognize the real thing if all you've ever been exposed to are the scams?

Comments (7) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

December 16, 2013

Top Chemical Publications of 2013

Email This Entry

Posted by Derek

From Organic Solution, here's a list of the top chemical papers of 2013. If I were to make my own list, some of the papers on this one would certainly overlap. Other people will have different ones, though, and I'd be glad to link to them as well - I'll update this post with any suggestions that come in for other selections.

Comments (8) + TrackBacks (0) | Category: The Scientific Literature

December 11, 2013

Down With the Luxury Journals?

Email This Entry

Posted by Derek

Nobel laureate Randy Schekman has stirred up a lot of controversy with his public declaration that he will send no more manuscripts to Nature, Science, Cell and such "luxury journals".

. . .The prevailing structures of personal reputation and career advancement mean the biggest rewards often follow the flashiest work, not the best. Those of us who follow these incentives are being entirely rational – I have followed them myself – but we do not always best serve our profession's interests, let alone those of humanity and society.

We all know what distorting incentives have done to finance and banking. The incentives my colleagues face are not huge bonuses, but the professional rewards that accompany publication in prestigious journals – chiefly Nature, Cell and Science.

These luxury journals are supposed to be the epitome of quality, publishing only the best research. Because funding and appointment panels often use place of publication as a proxy for quality of science, appearing in these titles often leads to grants and professorships. But the big journals' reputations are only partly warranted. While they publish many outstanding papers, they do not publish only outstanding papers. Neither are they the only publishers of outstanding research.

These journals aggressively curate their brands, in ways more conducive to selling subscriptions than to stimulating the most important research. Like fashion designers who create limited-edition handbags or suits, they know scarcity stokes demand, so they artificially restrict the number of papers they accept. The exclusive brands are then marketed with a gimmick called "impact factor". . .

The editorial staffs at these journals have been quick to point out that this is not necessarily a disinterested move, since Schekman is editor of eLife, a high-end open access journal. And no doubt some colleagues thought to themselves that it's much easier to abjure publication in the big journals after you've won your Nobel prize.

But there's a flip side to both those arguments. Scheckman is indeed editor of eLife, but that also means that he's willing to put his time and effort where his mouth is when he says that the current top-tier journals are a problem. And he's also willing to use his Nobelist profile in the service of that idea - many of the people complaining about how Scheckman is already famous would have produced no headlines at all had they announced that they would no longer publish in Nature. I seem to have taken that vow myself, at least so far in my career, without even realizing it.

Here's the response from Phillip Campbell at Nature Publishing Group. He says that he doesn't think it's helpful to mix the idea of open access with selectivity in publication, since open access is more of a business model. I see his point - there are selective open-access journals (like eLife), and there are nonselective ones, all the way down to the outright frauds. The same applies to the traditional journals, although the opportunities for fraud are not as lucrative. Campbell also decries the emphasis on impact factors, although the same objections can be made as with Scheckman, that it's easy for the editor of a high-impact-factor journal to play them down.

But impact factors really are pernicious. There will always be more- and less-presigious places to publish your research - any attempt to legislate otherwise is on a par with the creation of the New Soviet Man. But the advent of the impact factor gives everyone a lazy way to make the worst of it. It's especially tempting for people who don't even understand what someone else's research means, or might mean, but who have to judge them anyway. Why bother with the technical details? You have everything you need, to three idiotic decimal places, right there.

That actually sums up what I think about impact factors, although I've never put it quite so succinctly before: they're a moral hazard.

Update: more from Retraction Watch on the subject.

Comments (20) + TrackBacks (0) | Category: The Scientific Literature

December 6, 2013

Shop Up Some Gels For the Paper

Email This Entry

Posted by Derek

There have been many accusations over the years of people duplicating and fudging gels in biology papers. The site made quite an impression with some of these, and there are others. But as in so many other fields, manual labor is giving way to software and automation.

Nature News has the story of an Italian company that has come up with an automated way of searching images in scientific papers for duplication. The first scalp has already been claimed, but how bad is the problem?

Now midway through the analysis, he estimates that around one-quarter of the thousands of papers featuring gels that he has analysed so far potentially breached widely accepted guidelines on reproducing gel images. And around 10% seem to include very obvious breaches, such as cutting and pasting of gel bands. Some journals were more affected than others, he says. Those with a high impact factor tended to be slightly less affected. He plans to publish his results.

I'll be happy to see the paper, and glad to see this sort of technique applied more broadly. I wonder if it can be adapted to published NMR spectra?

Comments (21) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

December 3, 2013

What You Can Publish After a Shamectomy

Email This Entry

Posted by Derek

The sleazy scientific publishing racket continues to plumb new depths in its well-provisioned submarine. Now comes word of "Stringer Open" - nope, not Springer Open, that one's a real publisher of real journals. This outfit is Stringer, which is a bit like finding a list of journals published by the American Comical Society. The ScholarlyOA blog noticed that the same person appears on multiple editorial boards across their various journals. When contacted, she turned out to be a secretary who's never heard of "Stringer". Class all the way. The journals themselves will be populated by the work of dupes and/or con artists - maybe some of those Chinese papers-for-rent can be stuffed in there to make a real lasagna of larceny out of the whole effort.

Comments (12) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

December 2, 2013

Authorship For Sale. Papers For Sale. Everything For Sale.

Email This Entry

Posted by Derek

Academic publishing fraud in China has come up here before, but Science has an in-depth look at the problem. And a big problem it is:

"There are some authors who don't have much use for their papers after they're published, and they can be transferred to you," a sales agent for a company called Wanfang Huizhi told a Science reporter posing as a scientist. Wanfang Huizhi, the agent explained, acts as an intermediary between researchers with forthcoming papers in good journals and scientists needing to snag publications. The company would sell the title of co–first author on the cancer paper for 90,000 yuan ($14,800). Adding two names—co–first author and co–corresponding author—would run $26,300, with a deposit due upon acceptance and the rest on publication. A purported sales document from Wanfang Huizhi obtained by Science touts the convenience of this kind of arrangement: "You only need to pay attention to your academic research. The heavy labor can be left to us. Our service can help you make progress in your academic path!"

For anyone who cares about science and research, this is revolting. If you care a lot more about climbing that slippery ladder up to a lucrative position, though, it might be just the thing, right? There are all sorts of people ready to help you realize your dreams, too:

The options include not just paying for an author's slot on a paper written by other scientists but also self-plagiarizing by translating a paper already published in Chinese and resubmitting it in English; hiring a ghostwriter to compose a paper from faked or independently gathered data; or simply buying a paper from an online catalog of manuscripts—often with a guarantee of publication.

Offering these services are brokers who hawk titles and SCI paper abstracts from their perches in China; individuals such as a Chinese graduate student who keeps a blog listing unpublished papers for sale; fly-by-night operations that advertise online; and established companies like Wanfang Huizhi that also offer an array of above-board services, such as arranging conferences and producing tailor-made coins and commemorative stamps. Agencies boast at conferences that they can write papers for scientists who lack data. They cold-call journal editors. They troll for customers in chat programs. . .

The journal contacted 27 agencies in China, with reporters posing as graduate students or other scientists, and asked about paying to get on a list of authors or paying to have a paper written up from scratch. 22 of them were ready to help with either or both. Many of these were to be placed in Chinese-language journals, but for a higher fee you could get into more international titles as well. Because of Chinese institutional insistence on high-impact-factor journal publications, people who can deliver that kind of publication can charge as much as a young professor's salary. (Since some institutions turn around and pay a bonus for such publications, though, it can still be feasible).

Some agencies claim they not only prepare and submit papers for a client: They furnish the data as well. "IT'S UNBELIEVABLE: YOU CAN PUBLISH SCI PAPERS WITHOUT DOING EXPERIMENTS," boasts a flashing banner on Sciedit's website.

One timesaver: a ready stock of abstracts at hand for clients who need to get published fast. Jiecheng Editing and Translation entices clients on its website with titles of papers that only lack authors. An agency representative told an undercover Science reporter that the company buys data from a national laboratory in Hunan province.

The article goes on to show that there are many Chinese scientists that are trying to do something about all this. I hope that they succeed, but it's going to take a lot of work to realign the incentives. Unless this happens, though, the Chinese-language scientific literature risks finding itself devolving into a bad joke, and papers from Chinese institutions risk having to go through extra levels of scrutiny when submitted abroad.

Comments (30) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

November 19, 2013

More on the Open Access Sting Article

Email This Entry

Posted by Derek

I wrote here about the recent article by John Bohannon in Science, where he submitted a clearly substandard article to a long list of open-access publishers. (The results reflected poorly on many of them). Now here's a follow-up interview with Bohannon at The Scholarly Kitchen, where he addresses many of the critiques of the piece. Well worth a read if this issue interests you.

Q: What has been the response of editors and publishers? Have any journals ceased publication? Have any editors/editorial board members resigned in protest? Do any of them blame you, personally, for the outcomes? Have any threats (legal or otherwise) been made towards you or Science Magazine as a result of the exposé?

A couple of weeks before the story was published, I contacted editors and publishers specifically named in the story. Their responses are printed in the article, ranging from falling on their own sword and accepting responsibility to blaming the publishers and claiming they were not involved with the journal. But since the story was published, editors and publishers have largely fallen silent.

One exception is an editor based in the Middle East who says that the sting has cost him his job. It pains me to hear that. But then again, he clearly wasn’t doing his job.

As far as I can tell, it has been business as usual for the 157 publishers that got stung. I know of only one fully confirmed closure of a journal (as reported in Retraction Watch). There have been statements by publishers that they intend to close a journal, but I’ll believe it when I see it.

Of course, closing a single journal is nothing but a pinprick for many of the publishers that accepted the fake paper. Most publish dozens–some, hundreds–of titles.

I was bracing myself for lawsuits and PR blitzes from the publishers and editors that got stung. Ironically, the attacks came instead from advocates of the open access movement.

Comments (24) + TrackBacks (0) | Category: The Scientific Literature

November 12, 2013

It Doesn't Repeat? Who's Interested?

Email This Entry

Posted by Derek

Nature Biotechnology is making it known that they're open to publishing studies with negative results. The occasion is their publication of this paper, which is an attempt to replicate the results of this work, published last year in Cell Research. The original paper, from Chen-Yu Zhang of Nanjing University, reported that micro-RNAs (miRNAs) from ingested plants could be taken up into the circulation of rodents, and (more specifically) that miRNA168a from rice could actually go on to modulate gene expression in the animals themselves. This was a very interesting (and controversial) result, with a lot of implications for human nutrition and for the use of transgenic crops, and it got a lot of press at the time.

But other researchers in the field were not buying these results, and this new paper (from miRagen Therapeutics and Monsanto) reports that they cannot replicated the Nanjing work at all. Here's their rationale for doing the repeat:

The naturally occurring RNA interference (RNAi) response has been extensively reported after feeding double-stranded RNA (dsRNA) in some invertebrates, such as the model organism Caenorhabditis elegans and some agricultural pests (e.g., corn rootworm and cotton bollworm). Yet, despite responsiveness to ingested dsRNA, a recent survey revealed substantial variation in sensitivity to dsRNA in other Caenorhabditis nematodes and other invertebrate species. In addition, despite major efforts in academic and pharmaceutical laboratories to activate the RNA silencing pathway in response to ingested RNA, the phenomenon had not been reported in mammals until a recent publication by Zhang et al. in Cell Research. This report described the uptake of plant-derived microRNAs (miRNA) into the serum, liver and a few other tissues in mice following consumption of rice, as well as apparent gene regulatory activity in the liver. The observation provided a potentially groundbreaking new possibility that RNA-based therapies could be delivered to mammals through oral administration and at the same time opened a discussion on the evolutionary impact of environmental dietary nucleic acid effects across broad phylogenies. A recently reported survey of a large number of animal small RNA datasets from public sources has not revealed evidence for any major plant-derived miRNA accumulation in animal samples. Given the number of questions evoked by these analyses, the limited success with oral RNA delivery for pharmaceutical development, the history of safe consumption for dietary small RNAs and lack of evidence for uptake of plant-derived dietary small RNAs, we felt further evaluation of miRNA uptake and the potential for cross-kingdom gene regulation in animals was warranted to assess the prevalence, impact and robustness of the phenomenon.

They believe that the expression changes that the original team noted in their rodents were due to the dietary changes, not to the presence of rice miRNAs, which they say that they cannot detect. Now, at this point, I'm going to exit the particulars of this debate. I can imagine that there will be a lot of hand-waving and finger-pointing, not least because these latest results come partly from Monsanto. You have only to mention that company's name to an anti-GMO activist, in my experience, to induce a shouting fit, and it's a real puzzle why saying "DeKalb" or "Pioneer Hi-Bred" doesn't do the same. But it's Monsanto who take the heat. Still, here we have a scientific challenge, which can presumably be answered by scientific means: does rice miRNA get into the circulation and have an effect, or not?

What I wanted to highlight, though, is another question that might have occurred to anyone reading the above. Why isn't this new paper in Cell Research, if they published the original one? Well, the authors apparently tried them, only to find their work rejected because (as they were told) "it is a bit hard to publish a paper of which the results are largely negative". That is a silly response, verging on the stupid. The essence of science is reproducibility, and if some potentially important result can't be replicated, then people need to know about it. The original paper had very big implications, and so does this one.

Note that although Cell Research is published out of Shanghai, it's part of the Nature group of journals. If two titles under the same publisher can't work something like this out, what hope is there for the rest of the literature? Congratulations to Nature Biotechnology, though, for being willing to publish, and for explicitly stating that they are open to replication studies of important work. Someone should be.

Comments (20) + TrackBacks (0) | Category: Biological News | The Scientific Literature

November 11, 2013

The Past Twenty Years of Drug Development, Via the Literature

Email This Entry

Posted by Derek

Here's a new paper in PlOSOne on drug development over the past 20 years. The authors are using a large database of patents and open literature publications, and trying to draw connections between those two, and between individual drug targets and the number of compounds that have been disclosed against them. Their explanation of patents and publications is a good one:

. . .We have been unable to find any formal description of the information flow between these two document types but it can be briefly described as follows. Drug discovery project teams typically apply for patents to claim and protect the chemical space around their lead series from which clinical development candidates may be chosen. This sets the minimum time between the generation of data and its disclosure to 18 months. In practice, this is usually extended, not only by the time necessary for collating the data and drafting the application but also where strategic choices may be made to file later in the development cycle to maximise the patent term. It is also common to file separate applications for each distinct chemical series the team is progressing.

While some drug discovery operations may eschew non-patent disclosure entirely, it is nevertheless common practice (and has business advantages) for project teams to submit papers to journals that include some of the same structures and data from their patents. While the criteria for inventorship are different than for authorship, there are typically team members in-common between the two types of attribution. Journal publications may or may not identify the lead compound by linking the structure to a code name, depending on how far this may have progressed as a clinical candidate.

The time lag can vary between submitting manuscripts immediately after filing, waiting until the application has published, deferring publication until a project has been discontinued, or the code name may never be publically resolvable to a structure. A recent comparison showed that 6% of compound structures exemplified in patents were also published in journal articles. While the patterns described above will be typical for pharmaceutical and biotechnology companies, the situation in the academic sector differs in a number of respects. Universities and research institutions are publishing increasing numbers of patents for bioactive compounds but their embargo times for publication and/or upload of screening results to open repositories, such as PubChem BioAssay, are generally shorter.

There are also a couple of important factors to keep in mind during the rest of the analysis. The authors point out that their database includes a substantial number of "compounds" which are not small, drug-like molecules (these are antibodies, proteins, large natural products, and so on). (In total, from 1991 to 2010 they have about one million compounds from journal articles and nearly three million from patents). And on the "target" side of the database, there are a significant number of counterscreens included which are not drug targets as such, so it might be better to call the whole thing a compound-to-protein mapping exercise. That said, what did they find?
Here's the chart of compounds/target, by year. The peak and decline around 2005 is quite noticeable, and is corroborated by a search through the PCT patent database, which shows a plateau in pharmaceutical patents around this time (which has continued until now, by the way).

Looking at the target side of things, with those warnings above kept in mind, shows a different picture. The journal-publication side of things really has shown an increase over the last ten years, with an apparent inflection point in the early 2000s. What happened? I'd be very surprised if the answer didn't turn out to be genomics. If you want to see the most proximal effect of the human genomics frenzy from around that time, there you have it in the way that curve bends around 2001. Year-on-year, though (see the full paper for that chart), the targets mentioned in journal publications seem to have peaked in 2008 or so, and have either plateaued or actually started to come back down since then. Update: Fixed the second chart, which had been a duplicate of the first).
The authors go on to track a number of individual targets by their mentions in patents and journals, and you can certainly see a lot of rise-and-fall stories over the last 20 years. Those actual years should not be over-interpreted, though, because of the delays (mentioned above) in patenting, and the even longer delays, in some cases, for journal publication from inside pharma organizations.

So what's going on with the apparent decline in output? The authors have some ideas, as do (I'm sure) readers of this site. Some of those ideas probably overlap pretty well:

While consideration of all possible causative factors is outside the scope of this work it could be speculated that the dominant causal effect on global output is mergers and acquisition activity (M&A) among pharmaceutical companies. The consequences of this include target portfolio consolidations and the combining of screening collections. This also reduces the number of large units competing in the production of medicinal chemistry IP. A second related factor is less scientists engaged in generating output. Support for the former is provided by the deduction that NME output is directly related to the number of companies and for the latter, a report that US pharmaceutical companies are estimated to have lost 300,000 jobs since 2000. There are other plausible contributory factors where finding corroborative data is difficult but nonetheless deserve comment. Firstly, patent filing and maintenance costs will have risen at approximately the same rate as compound numbers. Therefore part of the decrease could simply be due to companies, quasi-synchronously, reducing their applications to control costs. While this happened for novel sequence filings over the period of 1995–2000, we are neither aware any of data source against which this hypothesis could be explicitly tested for chemical patenting nor any reports that might support it. Similarly, it is difficult to test the hypothesis of resource switching from “R” to “D” as a response to declining NCE approvals. Our data certainly infer the shrinking of “R” but there are no obvious metrics delineating a concomitant expansion of “D”. A third possible factor, a shift in the small-molecule:biologicals ratio in favour of the latter is supported by declared development portfolio changes in recent years but, here again, proving a causative coupling is difficult.

Causality is a real problem in big retrospectives like this. The authors, as you see, are appropriately cautious. (They also mention, as a good example, that a decline in compounds aimed at a particular target can be a signal of both success and of failure). But I'm glad that they've made the effort here. It looks like they're now analyzing the characteristics of the reported compounds with time and by target, and I look forward to seeing the results of that work.

Update: here's a lead author of the paper with more in a blog post.

Comments (22) + TrackBacks (0) | Category: Drug Development | Drug Industry History | Patents and IP | The Scientific Literature

October 22, 2013

ACSNano on Problematic Papers

Email This Entry

Posted by Derek

The editorial board at ACSNano has come out with a statement on how they'd like problematic papers to be handled. This, the article most pointedly does not say, is surely a response to the controversy over a recent (ridiculously Photoshopped) paper that appeared in the journal. That one didn't make anyone look good, and I can see why the editors felt that they had to make an effort.

The piece is superficially reasonable. They're asking that if someone sees a paper with questionable content, that they should contact the journal first, which I think is good practice in any case. But then we have this:

In the end, a decision will be made, ranging from notification that no cause was found to support the accusations made, corrections to a published article, retraction of the article, and/or to notifying the authors’ institutions of such actions. At ACS Nano, we take scientific fraud seriously and, as needed, retract articles and place sanctions on authors for set numbers of years, including bans on further submissions. The difference between this formalized accusation investigation and reports in blogs or on Twitter is that, during the investigation, the authors of the article under dispute have a fair chance to explain, and the decisions are made by known experts in the field. After we have made our decision, all are welcome to comment on it in any blog, even if they have different opinions; this is their privilege. We strongly suggest that such comments be made without the cloak of anonymity, using real names and affiliations, so that direct and open discussion of the work can be understood by others.

While we appreciate readers being critical and thus helping to weed out incorrect or fraudulent manuscripts, we still should not consider each publication from a competitor as being potentially wrong. A climate of mistrust will not help anyone and will only hamper honest scientists, which are the great majority of our community. Researchers make their reputations by publishing excellent data, not by being whistleblowers with mixed records of accuracy. It is easy to criticize the work of others, but it is substantially harder to achieve something by oneself. In other words, be critical, but never forget to be fair. One can be competitive, but still friends with colleagues, who naturally are also in some ways competitors. We are all humans, and we should never forget the human touches in our work.

So no one is supposed to comment until the editors have made a decision, no matter how long that might take? Desirable or not, I don't see that happening. Look, a scientific paper, once published out on the flippin' internet, is open to comment from whoever wishes to read it. That's what it's there for, to be made use of as its readers find appropriate. I tend to think that a more wide-open discussion of the merits of articles (or their lack of same) is actually good for the field. It should spur people on to write better papers, and put a bit more fear into those who might be tempted to fake things up.

I realize that people are afraid of libel, of character assassination, and so on. But arguing over the details of scientific publications does not lend itself to those activities very easily, although it's certainly true that there are plenty of folks out there who would not above that sort of thing if they thought they could get away with it. But these misdeeds are rather transparent, for the most part, and can just end up making the accusers themselves look foolish. They get the same kind of scrutiny as everyone else. (And besides, don't the sorts of people who really get into that stuff have a significant overlap with the sorts who would fake their papers?) I don't see this as mistrust - I see it as science. If your results are firm, they should be able to stand up to some shaking. If they can't, well, everyone should know about it. If you accuse someone mistakenly, well, you yourself should be ready to take the consequences of that, too. On the other hand, assuming (as the ACSNano piece seems to assume) that anyone with complaints about a paper must be a disgruntled competitor seems be a rather mistrustful way to look at things, too.

That second paragaph above, with its "play nice" advice, should be read while glancing at the "nanorod" photos from that recent paper. Try to reconcile the high-minded tone with what you see, and see if you have any better luck than I did.

Comments (62) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

October 17, 2013

The Reproducibility Initiative is Open

Email This Entry

Posted by Derek

Looks like the Reproducibility Initiative might be getting off the ground. This press release from the Center for Open Science says that they have a $1.3 million grant to start validating high-profile results in the oncology research literature. This will be done through the Science Exchange site, with results freely available to all comers.

I'm happy to see something like this coming together, but I don't know how far that money's going to go. The press release talks about 50 key papers that they'd like to reproduce, and I don't see how 1.3 million dollars will be enough to get through that list. (Is there a list of the papers anywhere? I can't find it). Some of the key tests will be relatively quick and cheap, but not all of them. But I agree with the COS that one of the important things here is to get this idea out into the real world, to get people used to it, and to establish it as useful. If they pick their targets carefully, the money should allow that to happen.

Comments (9) + TrackBacks (0) | Category: Cancer | The Scientific Literature

October 16, 2013

Holding Back Experimental Details, With Reason

Email This Entry

Posted by Derek

There's a lot of worry these days about the reproducibility of scientific papers (a topic that's come up here many times). And there's reason to believe that the sharing of data, protocols, and materials is not going so well, either.

. . . authors seem less willing to share these additional details about their study protocols than they have been in the past, according to a survey of 389 authors who published studies in the Annals of Internal Medicine. The findings, presented on 9 September at the International Congress on Peer Review and Biomedical Publication in Chicago, found that over the five years studied the percentage saying they would be willing to do so has dropped from almost 80% to only 60%.

A lack of incentives for sharing might be partly to blame. “There's no recognition, no promotion and no profit for scientists who share more information,” says Steven Goodman, a clinical research expert at Stanford University School of Medicine in California, who was part of the team that evaluated the survey results.

But there are two new papers out that deliberately does not share all the details, and it's not hard to see why. This NPR report has the background, but the abstract from the first paper will be enough for anyone in the field:

Clostridium botulinum strain IBCA10-7060, isolated from a patient with infant botulism, produced botulinum neurotoxin type B (BoNT/B) and another BoNT that, by use of the standard mouse bioassay, could not be neutralized by any of the Centers for Disease Control and Prevention–provided monovalent polyclonal botulinum antitoxins raised against BoNT types A–G.

That's not good. Until an antitoxin is available, the sequence of this new neurotoxin will not be published, although the fact of its existence is certainly worth knowing. The Journal of Infectious Diseases has two editorial articles on the issues that this work raises:

(The) identification of a novel, eighth botulinum neurotoxin (BoNT) from a patient with botulism expands our understanding of Clostridium botulinum and BoNT diversity, C. botulinum evolution, and the pathogenesis of botulism, but it also reveals a significant public health vulnerability. This new toxin, BoNT/H, cannot be neutralized by any of the currently available antibotulinum antisera, which means that we have no effective treatment for this form of botulism. Until anti-BoNT/H antitoxin can be created, shown to be effective, and deployed, both the strain itself and the sequence of this toxin (with which recombinant protein can be easily made) pose serious risks to public health because of the unusually severe, widespread harm that could result from misuse of either [3]. Thus, the dilemma faced by these authors, and by society, revolves around the question, should all of the information from this and similar studies be fully disseminated, motivated by the desire to realize all possible benefits from the discovery, or should dissemination of some or all of the information be restricted, with the goal of diminishing the probability of misuse?

I think they've made the right call here. (Last year's disputes about publishing work on a new strain of influenza are in just the same category.) Those studying botulin toxins need to know about this discovery, but given the molecular biology tools available to people, publishing the sequence (or making samples of the organism available) would be asking for potentially major trouble. This, unfortunately, seems to me to be an accurate reading of the world that we find ourselves in. There is a point where the value of having the knowledge out there is outweighed by the danger of. . .having the knowledge out there. This is going to be a case-by-case thing, but we should all be ready for some things to land on this side of the line.

Comments (14) + TrackBacks (0) | Category: Infectious Diseases | The Dark Side | The Scientific Literature

October 4, 2013

An Open Access Trash Heap

Email This Entry

Posted by Derek

Science magazine and writer John Bohannon have done us all a favor. There's a long article out in the latest issue that details how he wrote up a terrible, ridiculous scientific manuscript, attached a made-up name to it under the aegis of a nonexistent institution, and sent this farrago off to over three hundred open-access journals. The result?

On 4 July, good news arrived in the inbox of Ocorrafoo Cobange, a biologist at the Wassee Institute of Medicine in Asmara. It was the official letter of acceptance for a paper he had submitted 2 months earlier to the Journal of Natural Pharmaceuticals, describing the anticancer properties of a chemical that Cobange had extracted from a lichen.

In fact, it should have been promptly rejected. Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper's short-comings immediately. Its experiments are so hopelessly flawed that the results are meaningless.

I know because I wrote the paper. Ocorrafoo Cobange does not exist, nor does the Wassee Institute of Medicine. Over the past 10 months, I have submitted 304 versions of the wonder drug paper to open-access journals. More than half of the journals accepted the paper, failing to notice its fatal flaws. Beyond that headline result, the data from this sting operation reveal the contours of an emerging Wild West in academic publishing.

Well, sure, you're saying. Given the sorts of lowlife publishers out there, of course they took it, as long as the check cleared. But it's even worse than it appears:

Acceptance was the norm, not the exception. The paper was accepted by journals hosted by industry titans Sage and Elsevier. The paper was accepted by journals published by prestigious academic institutions such as Kobe University in Japan. It was accepted by scholarly society journals. It was even accepted by journals for which the paper's topic was utterly inappropriate, such as the Journal of Experimental & Clinical Assisted Reproduction.

Here's all the documentation, and it documents a sorry state indeed. You'll note from the world map in that link that India glows like a fireplace in this business. Nigeria has a prominence that it does not attain in the legitimate science publishing world, and there are exotic destinations like Oman and the Seychelles to be had as well. The editors of these "journals" tend to be people you've never heard of from universities that you didn't even know existed. And the editorial boards and lists of reviewers have plenty of those folks, mixed in with people who reviewed one paper before they didn't know better, and with people who didn't realize that their names were on the mastheads at all.

Bohannon didn't actually submit the exact same manuscript to all 304. He generated mix-and-match versions using an underlying template, giving him variations of the same crap and taking great care that the resulting papers should be obviously flawed:

he papers describe a simple test of whether cancer cells grow more slowly in a test tube when treated with increasing concentrations of a molecule. In a second experiment, the cells were also treated with increasing doses of radiation to simulate cancer radiotherapy. The data are the same across papers, and so are the conclusions: The molecule is a powerful inhibitor of cancer cell growth, and it increases the sensitivity of cancer cells to radiotherapy.

There are numerous red flags in the papers, with the most obvious in the first data plot. The graph's caption claims that it shows a "dose-dependent" effect on cell growth—the paper's linchpin result—but the data clearly show the opposite. The molecule is tested across a staggering five orders of magnitude of concentrations, all the way down to picomolar levels. And yet, the effect on the cells is modest and identical at every concentration.

One glance at the paper's Materials & Methods section reveals the obvious explanation for this outlandish result. The molecule was dissolved in a buffer containing an unusually large amount of ethanol. The control group of cells should have been treated with the same buffer, but they were not. Thus, the molecule's observed "effect" on cell growth is nothing more than the well-known cytotoxic effect of alcohol.

The second experiment is more outrageous. The control cells were not exposed to any radiation at all. So the observed "interactive effect" is nothing more than the standard inhibition of cell growth by radiation. Indeed, it would be impossible to conclude anything from this experiment.

I like this - the paper looks superficially presentable, but if you actually read it, then it's worthless. And yes, I realize that I've described a reasonable fraction of the ones that actually get published, but this is a more egregious example. I hope. The protocol was that Bohannon submitted the paper, and if it was rejected outright, that was that. If any reply came back addressing the paper's flaws in any way, he had a version ready to send back with more stuff in it, but without fixing any of the underlying problems. And if the paper was accepted, at any point in the process, he sent the journal a note that they'd discovered a serious flaw in their work and had to withdraw the manuscript.

157 journals accepted the paper, and 98 rejected it. He'd submitted it to a further 49 journals from his original list, but at least 29 of those appeared to be out of the business entirely, and the other 20 still had the paper "under review". So of those 255 decisions, 149 of them looked as if they'd occurred with little or no review. For a rejection, that's not so bad - this is a perfect example of manuscript that should not even be sent out for review. But the acceptances, well. . .

The other 106 editorial decisions made with some review are problematic, too. 70% of these were acceptances. Even in the few cases (36 times out of 304) where the paper was actually reviewed, and the reviewers realized that something was wrong with it (as they should have), the paper was accepted by 16 journals anyway.

The Elsevier journal that took this heap of junk was Drug Invention Today, in case you're wondering. I've never heard of it, and now I know why. The Sage journal was the Journal of International Medical Research, so you can strike that one off your list, too, assuming that the name wasn't enough all by itself. Another big open-access publisher, Hindawi (they advertise on the back cover of Chemistry World in the UK) rejected the paper from two of its journals, much to their relief. Jeffrey Beall's list of predatory publishers came as as pretty accurate, as well it might.

The problems with all this are obvious. These people are ripping off their authors for whatever publication fees they can scam, and some of these authors are not in a position to afford the treatment they're getting. No doubt some subset of the people who send manuscripts to these places are cynically padding their publication lists. The "editors" of these things get to reap a little unearned prestige for their "efforts" as well, so the whole enterprise just adds to the number of self-inflated jackasses with padded reputations, and the world is infested with too many of those people already. But I'm sure that there's another subset of authors who don't realize that they're submitting their results into a compost pile, and being asked to pay for the privilege. The first group are contemptible; the second group is sad. None of it's good.

Comments (59) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

October 3, 2013

RSS Readers: An Update

Email This Entry

Posted by Derek

The topic of RSS feeds for keeping up with the scientific literature has come up around here several times, especially since the demise of Google Reader. I've been trying several of these out, and wanted to report back. The Old Reader looked promising, but they seem to have had trouble dealing with the influx of users, leading to a notice a couple of months back that they were basically closing the service off to new users. This now seems to have been walked back: if others have found that they've settled things down, it's still definitely worth a look. I tried Feedly for a while, but couldn't stand the way that it dealt with graphical abstracts, and it seemed to hang on me a bit too often. So now I'm evaluating NewsBlur, and so far, it's working well, and doing what I want it to do. Keep in mind that I'm not a big social-share-this-story guy when it comes to RSS - in fact, since I use it for work, I really can't be. I just need a way to deal with the continual geyser of scientific publication.

Comments (46) + TrackBacks (0) | Category: The Scientific Literature

October 2, 2013

Results From the JACS Challenge

Email This Entry

Posted by Derek

The first results are in for the "JACS Challenge" that I mentioned here the other day. It looks like people's picks did correlate with later citation counts, overall, but with some curious exceptions. Some things that looked as if they would have been referred to a lot weren't, such as a paper on solventless polymerization. And a paper on transition-metal catalyzed boron-nitrogen bond formation was only picked by two respondents, but has amassed 258 citations since it was published. For the most part, though, people tended to judge the past pretty well (here's the PDF with all the statistics, if you're interested).

Comments (5) + TrackBacks (0) | Category: The Scientific Literature

September 30, 2013

They'll Fake the Journal if You'll Fake the Papers

Email This Entry

Posted by Derek

The Economist has a disturbing article on the extent of academic publishing fraud in China. It's disturbing that it goes on so much, and should be disturbing that it's in The Economist:

DISGUISED as employees of a gas company, a team of policemen burst into a flat in Beijing on September 1st. Two suspects inside panicked and tossed a plastic bag full of money out of a 15th-floor window. Red hundred-yuan notes worth as much as $50,000 fluttered to the pavement below.

Money raining down on pedestrians was not as bizarre, however, as the racket behind it. China is known for its pirated DVDs and fake designer gear, but these criminals were producing something more intellectual: fake scholarly articles which they sold to academics, and counterfeit versions of existing medical journals in which they sold publication slots.

As China tries to take its seat at the top table of global academia, the criminal underworld has seized on a feature in its research system: the fact that research grants and promotions are awarded on the basis of the number of articles published, not on the quality of the original research. . .

If there's one thing that economists are sure of, it's that you get what you subsidize (even if you might not have realized up front just what it was you were paying for). And if the Chinese establishment has decided that long publications lists are necessary, then long publication lists they shall have. The same thing happens in a drug research department when management is so foolish as to reward people for sheer number of compounds submitted - you get a deluge of amides, sulfonamides, and methyl-ethyl-butyl-futile coupling reactions. One half of the stockroom gets mixed with the other half, in the presence of HATU and/or palladiu, and voila, productivity on a shingle.

At least those are real componds. Apparently, many of the papers being generated under the Chinese onslaught are not just repetitious, bite-sized chunklets of stretched-out lab results, but flat-out fantasies:

The pirated medical-journal racket broken up in Beijing shows that there is a well-developed market for publication beyond the authentic SCI journals. The cost of placing an article in one of the counterfeit journals was up to $650, police said. Purchasing a fake article cost up to $250. Police said the racket had earned several million yuan ($500,000 or more) since 2009. Customers were typically medical researchers angling for promotion.

And this makes you wonder how many of the people doing the evaluating also knew, or suspected, that these journals were fakes, but had reasons of their own to pretend otherwise. Something needs to be done about all this, clearly, but that's not going to be possible without a lot of disruption. The longer it goes on, though, the worse that disruption might be. . .

Comments (21) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

September 26, 2013

An Unknown Author With Someone Else's Work. Why?

Email This Entry

Posted by Derek

Here's a bizarre one: someone apparently faked up a bunch of author names and contact information, and published results (in Biochemical and Biophysical Research Communications) that they're heard Bruce Spiegelman of Harvard talk about. The motive? Well. . .the only thing that makes sense is sheer vituperativeness, and even that doesn't make much. Here's the story - see if you can make sense of it!

Comments (17) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

September 23, 2013

What's An Important Paper? Take the JACS Challenge!

Email This Entry

Posted by Derek

Here's a darn good question: how good are we at telling (when they're published) what scientific papers are important? We were talking around here the other day about famous work that had been rejected, and this is the less dramatic aspect of the same phenomenon.

At ScienceGeist, they're putting this to an empirical test. Given a ten-year-old issue of the Journal of the American Chemical Society, could a person predict which papers in it have had the most impact? Well, here's the issue, and here's the survey to take after you've looked it over. Please don't cheat by looking up citation counts or the like - approach this one cold and use your mighty scientific intuition. I'll be putting my own answers in later this morning. I look forward to seeing the results!

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

September 17, 2013

Thoughts on the Scientific Publishing Model, And Its Inverse

Email This Entry

Posted by Derek

I mentioned the idea of an Elsevier boycott here last year. Here's someone who's thought about another course of action: if you can't beat 'em, join 'em - up to a point:

How about this for a solution:

1. We start a (say) monthly journal.

2. Set up a simple, lucrative payment for accepted articles. As a first stab, how about $10,000 paid personally to the researchers responsible. Similarly, each reviewer receives $2,000 for each article reviewed.

I imagine this would be enough to attract some serious submissions. So serious, in fact, that universities and libraries would be obliged to subscribe (at drastically reduced prices). Oh yeah, and you can only submit articles if you or your institution is a subscriber.

So every month hundreds of thousands of dollars would be distributed to scientists, and there would be a monthly prize pool for excellent research. And I bet the journal would be a cracking read (or as cracking as stiff scientific prose ever is, anyway).

But why not take it a step further.

3. Assuming, of course, you executed that first step and attracted some worthwhile research, you could simply distribute all of the profits. So if you attracted 5% of the market… say 50% penetration at half price, you would be able to distribute 250m a year (I know, I know).

I imagine a star researcher would prefer to get paid $200k rather than submit to Nature. And as the quality of the new journal improved, it could even end up becoming more prestigious.

The first problem I have with this, and it's a big one, is that if you're giving "drastically reduced" subscription prices to libraries and universities, where are all the profits coming from that you're using to pay the authors? I believe that squeezing institutional subscribers is, in fact, the business model for many of the scientific publishers. A second difficulty (there are others, too) is that our current system already encourages people to divide publications up into smaller units and get a longer list of papers to their name. If there's sill more incentive to publish (cash rewards!), I can imagine this problem only getting worse.

The general rule in nonscientific publishing, that "money flows towards the author", is based on a market that pays for the finished products voluntarily, and has the option of not reaching into its collective pocket at all. Publishers are also free to pick and choose among the many, many manuscript opportunities they're offered, trying to find those that they think will provide a return on their own investments of time and money.

Scientific publishing is a different undertaking, with a mixed set of motivations. We're publishing our results to show what we've accomplished, and to add to the storehouse of human knowledge (as opposed, say, to adding to the storehouse of human romance novels or slow-cooker chicken recipe compendia). We're also publishing to make ourselves look better, to our current employers and to potential future ones, not that such publications are a very efficient way to do that, but still. And the readers are keeping up with the published matter partly out of a desire to learn about what we have to say, and partly out of professional necessity.

Here's the thing: if it were not for the expense necessary to produce the journals, there would be no market involved at all. In the early days of science, results were distributed by personal correspondence, and journals are, in a way, just collections of such letters. Some of them, you'll notice, refer to that function by having "Letters" in their names. No one was paid to write those letters to their colleagues, and no one paid to receive them. The only expenses were the writing materials, the time it took for composition, and postage - just as the expenses now are the time and effort for composition (on the author's part) and the time and effort for editing, reviewing, and publishing (on the journal's part). These expenses have, in theory, been going down as fewer journals are published on pressed tree sheets and hauled around by mail trucks, but they cannot go to zero as long as there is editing involved, as well as commentary on the papers themselves and publicity for them, not to mention things like server maintenance, etc.

All the fights about the cost of scientific journals are fought on that narrow strip of ground. How much does it cost to do these things, and is it truly worth what people pay for them? Otherwise, we might all just as well upload everything into a central electronic archive, with a nominal fee to keep the hardware running. We'd then be back to sending letters around to let everyone else know when we've put in something that we think is hot stuff, or setting up web sites to get the news out. And some of those might become useful enough to start charging a subscription fee, and then. . .

Comments (18) + TrackBacks (0) | Category: The Scientific Literature

September 10, 2013

Great Papers That Have Been Rejected

Email This Entry

Posted by Derek

A discussion in the lab brought up this topic: there have been, so everyone hears, famous papers in the sciences that were rejected by the first journals that they were submitted to. I believe, for example, that Dan Schectman faced a lot of opposition to getting his first quasicrystal work published (and he certainly got a lot, notably from Linus Pauling, after it came out). To pick one, the original Krebs cycle paper was turned down by Nature, which a later editor called the journal's biggest single mistake. Here are some famous examples from computer science and related mathematics (update: in a parody/spoof paper!) and here's a discussion from an economist on this topic in his own field - I believe that the original Black-Scholes option pricing model paper was turned down as well.

If anyone has more examples to add from chemistry, I'd be glad to highlight them. I have some more thoughts on the subject that I'll expand into another post later on. . .

Comments (37) + TrackBacks (0) | Category: The Scientific Literature

September 9, 2013

What To Do About Chemistry Papers That Don't Work?

Email This Entry

Posted by Derek

I wanted to follow up on a point raised in the comments to this morning's post. Reader DJ writes that:

Our lab recently tried to repeat a procedure from a TL methodology paper on the amination of amines to make protected hydrazines. We noticed that the entire paper is wrong since the authors did not analyze the product NMRs carefully, and incorrectly assumed that they were forming the desired hydrazines, rather than the rearranged alkoxyamines. It's a systemic error throughout the paper that renders the entire premise and methodology essentially useless. So our quandry is this: (1) This is clearly not a case of fraud, but just bad science. (2) Our discovery is not all that interesting on its own, since the unusual reaction pathway has little practical value, and (3) the original report is from a TL paper that probably not many people will read in the first place. Still, I am tempted to do SOMETHING to correct the public record, but what? Maybe contact the original authors (somewhere in India)? Try to write a 'rebuttal style' paper in the same journal? Any suggestions?

That's a really good question. And I have a personal interest in it - no, I didn't write the hydrazine paper. I've recently tried to use a method in a paper from Chemical Communications that I cannot get to work. I gave it several tries, messing around with fresh reagents and so on, but I get nothing. In my case, I've written to the authors, and received no reply whatsoever. So, like DJ, I'm wondering what to do next. Writing to the authors is the first step that I recommend that he take, but he might well end up in the same situation I am.

So what next? Neither of these rise to a Blog-Syn level, I think, because just having half a dozen more people confirm that "Yeah, this stuff doesn't work" doesn't seem like a good use of people's time. (I think that Blog Syn efforts are better for reactions that work, but not as well as they're supposed to, because of some variables that aren't well worked out in the original papers). I'm not particularly interested in running a "Name and Shame" site for organic chemistry papers that can't be reproduced (and anyone who does will, I think, find themselves with a lot more work and many more headaches than they'd imagined). But what is there to do?

Comments (42) + TrackBacks (0) | Category: The Scientific Literature

Exposing Faked Scientific Papers

Email This Entry

Posted by Derek

Chemistry World has a good article on the problem of shaky data in journal article, and the intersecting problem of what to do about it in the chemistry blogging world. Paul Bracher's ChemBark is, naturally, a big focus of the piece, since he's been highlighting some particularly egregious examples in the last couple of months (which I've linked to from here).

The phrase "witch hunt" has been thrown around by some observers, but I don't think that's fair or appropriate. In great contrast to the number of witches around (and their effectiveness), faked information in published scientific articles is very much a real thing, and can have real consequences. Time spent looking for it and exposing it is not time wasted, not when it's at its current levels. But who should be doing the looking and the exposing?

The standard answer is "Why, journal editors and reviewers, who shouldn't be letting this stuff past in the first place". Quite true. But in many cases, they are letting it past, so what should be done once it's published? A quiet, gentlemanly note to the editorial staff? Or a big blazing row in a public forum, such as a widely-read blog? Even though I don't start many of these myself, I come down more on the side of the latter. There are problems with that stance, of course - you have to be pretty sure that there's something wrong before you go making a big deal out of it, for one thing. Hurting someone else's reputation for no reason would be a bad thing, as would damaging your own credibility by making baseless accusations.

But in some of these recent cases, there's been little doubt about the problem. Take that nanorod paper: the most common result when I showed to to people was "Oh, come on." (And the most common result when I showed the famous "Just make up an elemental" paper to people was "Oh, (expletive)", with several common words all filling in appropriately). So if there's clearly trouble with a published paper, why is it such a good thing to make a big public spectacle out of it?

Deterrence. I really think that there will be less of this if people think that there's a reasonable chance that fake science will be exposed widely and embarrassingly. Covering up half your NMR spectrum with a box of digital white-out is fraud and people committing fraud have given up their opportunity to be treated with respect. And don't forget, the whole deterrence argument applies to editors and reviewers, too. I can guarantee that many chemists looked at these recent examples and wondered if they would have let these papers go through the review process, through carelessness or lack of time, and resolved to do better the next time. I certainly did.

That said, I do not intend to make this blog the full-time scourge of the chemical literature by patrolling the literature myself. If I see something suspicious, I'll speak up about it, and if other chemistry blogs (or readers) pick up on something, I'm very glad to hear about it or link to it. But finding these examples is a perfect example of something that I think is best left to the crowd. The person best equipped to discover a fraudulent paper is the person who is interested in its subject and would like to build on its results - in other words, the person who would be most harmed by it. And if someone fakes a paper, but no one ever reads it or refers to it, well, that's the author's own reward, and I hope that they enjoy it.

Comments (24) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

August 30, 2013

Welcome to the Author's Brain. The "Fasten Seatbelts" Sign Is Illuminated

Email This Entry

Posted by Derek

Well, it's the Friday before a long holiday weekend here in the US, so I don't know if this is the day for long, detailed posts. I do have some oddities in the queue, though, so this is probably a good day to clear them out.

For starters, here's one in the tradition of the (in)famous Andrulis "gyre" paper. Another open-access publisher (SAGE) has an unusual item in their journal Qualitative Inquiry. (Some title, by the way - you could guess for days about what might appear under that category). The paper's title gets things off to a fine start: "Welcome to My Brain". And the abstract? Glad you asked:

This is about developing recursive, intrinsic, self-reflexive as de-and/or resubjective always evolving living research designs. It is about learning and memory cognition and experiment poetic/creative pedagogical science establishing a view of students ultimately me as subjects of will (not) gaining from disorder and noise: Antifragile and antifragility and pedagogy as movements in/through place/space. Further, it is about postconceptual hyperbolic word creation thus a view of using language for thinking not primarily for communication. It is brain research with a twist and becoming, ultimately valuation of knowledges processes: Becoming with data again and again and self-writing theory. I use knitting the Möbius strip and other art/math hyperbolic knitted and crocheted objects to illustrate nonbinary . . . perhaps. Generally; this is about asking how-questions more than what-questions.

Right. That's word-for-word, by the way, even though it reads as if parts of speech have been excised. Now, I do not, sadly, have access to journals with the kind of reach that Qualitative Inquiry displays, so I have not attempted to read the whole text. But the abstract sounds either like a very elaborate (and unenlightening) word game, or the product of a disturbed mind. The Neurobonkers blog, though, has some more, and it definitely points toward the latter:

This article is therefore about developing recursive intrinsic self-reflexive as de- and/or resubjective always evolving living research designs. Inquiry perhaps full stop—me: An auto-brain—biography and/or a brain theo­rizing itself; me theorizing my brain. It is thus about theo­rizing bodily here brain and transcorporeal materialities, in ways that neither push us back into any traps of biological determinism or cultural essentialism, nor make us leave bodily matter and biologies behind.

Apprarently, most of the manuscript is taken up with those "This is about. . ." constructions, which doesn't make for easy reading, either. At various points, a being/character called "John" makes appearances, as do recurring references to knitting and to Möbius strips. Brace yourselves:

Knitting John, John knitting. Knitting John Möbius. Möbius knitting John. Giant Möbius Strips have been used as conveyor belts (to make them last longer, since “each side” gets the same amount of wear) and as continuous-loop recording tapes (to double the playing time). In the 1960’s Möbius Strips were used in the design of versatile electronic resistors. Freestyle skiers have named one of their acrobatic stunts the Möbius Flip. The wear and tear of my efforts. My stunts, enthusiasm knitting. My brain and doubling and John.

OK, that's deranged. And how could anyone at SAGE have possibly reviewed it? This is the same question that came up with the MDPI journals and the Andrulis paper - five minutes with this stuff and you feel like calling up the author and telling them to adjust their dosages (or perhaps like adjusting yours). This sort of thing is interesting in a roadside-accident sort of way, but it also calls open-access publishing into disrepute. Maybe it's time for not only a list of predatory publishers, but a list of nonpredatory ones that freely admit garbage.

Comments (43) + TrackBacks (0) | Category: The Scientific Literature

August 27, 2013

Not Sent Out For Review

Email This Entry

Posted by Derek

Blogger Pete over at Fragment-Based Drug Discovery has a tale to tell about trying to get a paper published. He sent in a manuscript on alkane/water partition coefficients to the Journal of Chemical Information and Modeling, only to get back the "not sent out for review" response. That's the worst, the "We're not even going to consider this one" letter. And the odd thing is that, as he rightly put it, this does sound like a JCIM sort of paper, but the editor's response was that it was inappropriate for the journal, and that they had "limited interest" in QSAR/QSPR studies.

So off the paper went to the Journal of Computer-Aided Molecular Design. But as it was going to press, what should appear in JCIM but a paper on. . .alkane/water partition coefficients. There follows some speculation on how and why this happened, and if further details show up, I'll report on them.

But the whole "not sent out for review" category is worth thinking about. I'd guess that most papers that fall into that category truly deserve to be there - junk, junk that's written impossibly and impenetrably poorly, things that should have been sent to a completely different journal. These are the scientific equivalent of Theresa Nielsen Hayden's famous Slushkiller post, about the things that show up unsolicited at a publisher's office. If you're editing a science fiction magazine, you might be surprised to get lyric poetry submissions in another language, or biographical memoirs about growing up in Nebraska - but you'd only be surprised, apparently, if you'd never edited a science fiction magazine before (or any other kind).

But a journal editor can consign all sorts of papers to the outer darkness. At some titles, just getting a manuscript sent out to the referees is an accomplishment, because the usual response is "Stop wasting our time" (albeit not in those exact words, not usually). An author isn't going to be surprised in those cases, but getting that treatment at a less selective journal is more problematic.

Comments (17) + TrackBacks (0) | Category: The Scientific Literature

August 15, 2013

Big Pharma And Its Research Publications

Email This Entry

Posted by Derek

A longtime reader sent along this article from the journal Technological Forecasting and Social Change, which I'll freely admit never having spent much time with before. It's from a team of European researchers, and it's titled "Big Pharma, little science? A bibliometric perspective on Big Pharma's R&D decline".

What they've done is examine the publication record for fifteen of the largest drug companies from 1995 to 2009. They start off by going into the reasons why this approach has to be done carefully, since publications from industrial labs are produced (and not produced) for a variety of different reasons. But in the end:

Given all these limitations, we conclude that the analysis of publications does not in itself reflect the dynamics of Big Pharma's R&D. However, at the high level of aggregation we conduct this study (based on about 10,000 publications per year in total, with around 150 to 1500 publications per firm annually) it does raise interesting questions on R&D trends and firm strategies which then can be discussed in light of complementary quantitative evidence such as the trends revealed in studies using a variety of other metrics such as patents and, as well as statements made by firms in statutory filing and reports to investors.

So what did they find? In the 350 most-represented journals, publications from the big companies made up about 4% of the total content over those years (which comes out to over 10,000 papers). But this number has been dropping slightly, but steadily over the period. There are now about 9% few publications from Big Pharma than there were at the beginning of the period. But this effect might largely be explained by mergers and acquisitions over the same period - in every case, the new firm seems to publish fewer papers than the old ones did as a whole.

And here are the subject categories where those papers get published. The green nodes are topics such as pharmacology and molecular biology, and the blue ones are organic chemistry, medicinal chemistry, etc. These account for the bulk of the papers, along with clinical medicine.
The number of authors per publication has been steadily increasing (in fact, even faster than the other baseline for the journals as a whole), and the organizations-per-paper has been creeping up as well, also slightly faster than the baseline. The authors interpret this as an increase in collaboration in general, and note that it's even more pronounced in areas where Big Pharma's publication rate has grown from a small starting point, which (plausibly) they assign to bringing in outside expertise.

One striking result the paper picks up on is that the European labs have been in decline from a publication standpoint, but this seems to be mostly due to the UK, Switzerland, and France. Germany has held up better. Anyone who's been watching the industry since 1995 can assign names to the companies who have moved and closed certain research sites, which surely accounts for much of this effect. The influence of the US-based labs is clear:

Although in most of this analysis we adopt a Europe versus USA comparative perspective, a more careful analysis of the data reveals that European pharmaceutical companies are still remarkably national (or bi-national as a results of mergers in the case of AstraZeneca and Sanofi-Aventis). Outside their home countries, European firms have more publications from US-based labs than all their non-domestic European labs (i.e. Europe excluding the ‘home country’ of the firm). Such is the extent of the national base for collaborations that when co-authorships are mapped into organisational networks there are striking similarities to the natural geographic distribution of countries. . .with Big Pharma playing a notable role spanning the bibliometric equivalent of the ‘Atlantic’.

Here's one of the main conclusions from the trends the authors have picked up:

The move away from Open Science (sharing of knowledge through scientific conferences and publications) is compatible and consistent with the increasing importance of Open Innovation (increased sharing of knowledge — but not necessarily in the public domain). More specifically, Big Pharma is not merely retreating from publication activities but in doing so it is likely to substitute more general dissemination of research findings in publications for more exclusive direct sharing of knowledge with collaboration partners. Hence, the reduction in publication activities – next to R&D cuts and lab closures – is indicative of a shift in Big Pharma's knowledge sharing and dissemination strategies.

Putting this view in a broader historical perspective, one can interpret the retreat of Big Pharma from Open Science, as the recognition that science (unlike specific technological capabilities) was never a core competence of pharmaceutical firms and that publication activity required a lot of effort, often without generating the sort of value expected by shareholders. When there are alternative ways to share knowledge with partners, e.g. via Open Innovation agreements, these may be attractive. Indeed an associated benefit of this process may be that Big Pharma can shield itself from scrutiny in the public domain by shifting and distributing risk exposure to public research organisations and small biotech firms.

Whether the retreat from R&D and the focus on system integration are a desirable development depends on the belief in the capacities of Big Pharma to coordinate and integrate these activities for the public good. At this stage, one can only speculate. . .

Comments (14) + TrackBacks (0) | Category: Academia (vs. Industry) | Drug Industry History | The Scientific Literature

August 14, 2013

Nanorods? Or Photoshop?

Email This Entry

Posted by Derek

If you haven't seen this, which goes into some very odd images from a paper in the ACS journal Nano Letters, then have a look. One's first impression is that this is a ridiculously crude Photoshop job, but an investigation appears to be underway to see if that's the case. . .

Update: the paper has now been withdrawn. The chemistry blogs get results!

Comments (28) + TrackBacks (0) | Category: The Scientific Literature

August 8, 2013

Make Up the Elemental Analysis: An Update

Email This Entry

Posted by Derek

Chemistry Blog has more on the incident picked up first at ChemBark and noted here yesterday. This rapidly-becoming-famous case has the Supporting Information file of a paper published at Organometallics seemingly instructing a co-author to "make up" an elemental analysis to put in the manuscript.

Now the editor of the journal (John Gladysz of Texas A&M) has responded to Chemistry Blog as follows:

Wednesday 07 August
Dear Friends of Organometallics,

Chemical Abstracts alerted us to the statement you mention,which was overlooked during the peer review process, on Monday 05 August. At that time, the manuscript was pulled from the print publication queue. The author has explained to us that the statement pertains to a compound that was ”downgraded” from something being isolated to a proposed intermediate. Hence, we have left the ASAP manuscript on the web for now. We are requiring that the author submit originals of the microanalysis data before putting the manuscript back in the print publication queue. Many readers have commented that the statement reflects poorly on the moral or ethical character of the author, but the broad “retribution” that some would seek is not our purview. As Editors, our “powers” are limited to appropriate precautionary measures involving future submissions by such authors to Organometallics, the details of which would be confidential (ACS Ethical Guidelines, Our decision to keep the supporting information on the web, at least for the time being, is one of transparency and honesty toward the chemical community. Other stakeholders can contemplate a fuller range of responses. Some unedited opinions from the community are available in the comments section of a blog posting:

If you have any criticisms of the actions described above, please do not hesitate to share them with me. Thanks much for being a reader of Organometallics, and best wishes. . .

This is the first report of the corresponding author, Reto Dorta, responding about this issue (several other people have tried to contact him, with no apparent success). So much for the theory, advanced by several people in the comments section at ChemBark, that "make up" was being used in the British-English sense of "prepare". Gladysz's letter gets across his feelings about the matter pretty clearly, I'd say.

Comments (31) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

August 7, 2013

New Frontiers in Analytical Chemistry

Email This Entry

Posted by Derek

A reader sends this new literature citation along, from Organometallics. He directed my attention to the Supplementary Information file, page 12. And what do we find there?

. . .Solvent was then removed to leave a yellow residue in the vial, the remaining clear, yellow solution was concentrated to a volume of about 1ml, and diethyl ether was added in a dropwise manner to the stirred solution to precipitate a yellow solid. The vial was centrifuged so the supernatant solvent could be decanted off by Pasteur pipette. The yellow solid was washed twice more with ether and the dried completely under high vacuum to give 99mg (93% yield) of product.

Emma, please insert NMR data here! where are they? and for this compound, just make up an elemental analysis...

And don't forget to proofread the manuscript, either, while you're at it. Oops.

Update: I see that Chembark is on this one, and has gone as far as contacting the corresponding author, whose day has gotten quite a bit longer. . .

Comments (28) + TrackBacks (0) | Category: Analytical Chemistry | The Scientific Literature

July 16, 2013

Touching Up the Spectra

Email This Entry

Posted by Derek

Organic chemists have been taking NMR spectra for quite a while now. Routine use came on in the 1960s, and higher-field instruments went from exotic big-ticket items in the 1970s to ordinary equipment in the 1980s. But NMR can tell you more about your sample than you wanted to know (good analytical techniques are annoying that way). So what to do when you have those little peaks showing up where no peaks should be?

The correct answer is "Live with 'em or clean up your sample", but wouldn't it be so much easier and faster to just clean up the spectrum? After all, that's all that most people are ever going to see - right? This little line of thought has occurred to countless chemists over the years. Back In The Day, the technology needed to remove solvent peaks, evidence of isomers, and other pesky impurities was little more than a bottle of white-out and a pen (to redraw the lovely flat baseline once the extra peaks were daubed away). Making a photocopy of the altered spectrum gave you publication-ready purity in one easy step.

NMR spectra are probably the most-doctored of the bunch, but LC/MS and HPLC traces are very capable of showing you peaks you didn't want to see, either. These days there are all sorts of digital means to accomplish this deception, although I've no doubt that the white-out bottle is still deployed. In case anyone had any doubt about that, last month Amos Smith, well-known synthetic organic chemist and editor of Organic Letters, had this to say in a special editorial comment in the journal:

Recently, with the addition of a Data Analyst to our staff, Organic Letters has begun checking the submitted Supporting Information more closely. As a result of this increased scrutiny, we have discovered several instances where reported spectra had been edited to remove evidence of impurities.

Such acts of data manipulation are unacceptable. Even if the experimental yields and conclusions of a study are not affected, ANY manipulation of research data casts doubts on the overall integrity and validity of the work reported.

That it does. He went on to serve notice on authors that the journal will be checking, and will be enforcing and penalizing. And you can tell that Smith and the Org Lett staff have followed up on some of these already, because they've already had a chance to hear the default excuse:

In some of the cases that we have investigated further, the Corresponding Author asserted that a student had edited the spectra without the Corresponding Author’s knowledge. This is not an acceptable excuse! The Corresponding Author (who is typically also the research supervisor of the work performed) is ultimately responsible for warranting the integrity of the content of the submitted manuscript. . .

As the editorial goes on the say, and quite rightly, if a student did indeed alter the spectrum before showing it to the boss, it's very likely because the boss was running a group whose unspoken rule was that only perfection was acceptable. And that's an invitation to fraud, large and small. I'm glad to see statements like Smith's - the only ways to keep down this sort of data manipulation are to make the rewards for it small, increase the chances of it being found out, and make the consequences for it real.

As for those, the editorial speaks only of "significant penalties". But I have some ideas for those that might help people think twice about the data clean-up process. How about a special correction in the journal, showing the altered spectra, with red circles around the parts that had been flattened out? And a copy of the same to the relevant granting agencies and department heads? That might help get the message out, you think?

As an aside, I wanted to mention that I have seen someone stand right up and take responsibility for extra peaks in an NMR. Sort of. I saw a person once presenting what was supposed to be the final product's spectrum, only there were several other singlet peaks scattered around. "What are those?" came the inevitable question. "Water" was the answer. "Umm. . .how many water peaks, exactly?" "Oh, this one is water in solution. And this one is water complexed with the compound. And this one is water adsorbed to the inside of the NMR tube. And this one is water adsorbed to the outside of the. . ." It took a little while for order to be restored at that point. . .

Comments (38) + TrackBacks (0) | Category: Analytical Chemistry | The Dark Side | The Scientific Literature

July 9, 2013

Non-Reproducible Science: A Survey

Email This Entry

Posted by Derek

The topic of scientific reproducibility has come up around here before, as it deserves to. The literature is not always reliable, and it's unreliable for a lot of different reasons. Here's a new paper in PLOS ONE surveying academic scientists for their own experiences:

To examine a microcosm of the academic experience with data reproducibility, we surveyed the faculty and trainees at MD Anderson Cancer Center using an anonymous computerized questionnaire; we sought to ascertain the frequency and potential causes of non-reproducible data. We found that ~50% of respondents had experienced at least one episode of the inability to reproduce published data; many who pursued this issue with the original authors were never able to identify the reason for the lack of reproducibility; some were even met with a less than “collegial” interaction.

Yeah, I'll bet they were. It turns out that about half the authors who had been contacted about problems with a published paper responded "negatively or indifferently", according to the survey respondents. As to how these things make it into the literature in the first place, I don't think that anyone will be surprised by this part:

Our survey also provides insight regarding the pressure to publish in order to maintain a current position or to promote ones scientific career. Almost one third of all trainees felt pressure to prove a mentor's hypothesis even when data did not support it. This is an unfortunate dilemma, as not proving a hypothesis could be misinterpreted by the mentor as not knowing how to perform scientific experiments. Furthermore, many of these trainees are visiting scientists from outside the US who rely on their trainee positions to maintain visa status that affect themselves and their families in our country.

And some of these visiting scientists, it should be noted, come from backgrounds in authority-centered and/or shame-based cultures, where going to the boss with the news that his or her big idea didn't work is not a very appealing option. It's not for anyone, naturally, but it's especially hard if you feel that you're contradicting the head of the lab and bringing shame on yourself in the process.

As for what to do about all this, the various calls for more details in papers and better reviewing are hard to complain about. But while I think that those would help, I don't see them completely solving the problem. This is a problem of human nature; as long as science is done by humans, we're going to have sloppy work all the way up to outright cheating. What we need to do is find ways to make it harder to cheat, and less rewarding - that will at least slow it down a bit.

There will always be car thieves, too, but we don't have to make it easy for them, either. Some of our publishing practices, though, are the equivalent of habitually walking away with the doors unlocked and the keys in the ignition. Rewarding academic scientists (at all levels) so directly for the number of their publications is one of the big ones. Letting big exciting results through without good statistical foundations is another.

In this vein, a reader sends along the news that the Reproducibility Initiative is now offering grants for attempts to check big results in the literature. That's the way to get it done, and I'm glad to see some money forthcoming. This effort is concentrating on experimental psychology, which is appropriate, given that the field has had some recent scandals (follow-up here) and is now in a big dispute over the reproducibility of even its honestly-meant data. They need all the help they can get over there - but I'll be glad to see some of this done over here in the biomedical field, too.

Comments (16) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

July 8, 2013

19 Years to a Retraction. Bonus Midnight Camera Footage Included.

Email This Entry

Posted by Derek

This Retraction Watch post details the longest correction/retraction saga I've heard of yet. A 1994 paper in Nature has finally been pulled back, after years and years of wrangling. And by "wrangling" I mean multiple attempted repeats, blinded samples, fraught exchanges over scientific ethics with one of the most high-profile professors in the Czech Republic and hidden camera footage from the lab freezer. Yep, it got to that point - midnight break-ins to alter the stored samples. Read the post for more; it's really something.

Comments (4) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 24, 2013

How Robust Is That New Reaction, Anyway?

Email This Entry

Posted by Derek

Here's a paper in Nature Chemistry that addresses something that isn't explicitly targeted as often as it should be: the robustness of new reactions. The authors, I think, are right on target with this:

We believe a major hurdle to the application of a new chemical methodology to real synthetic problems is a lack of information regarding its application beyond the idealized conditions of the seminal report. Two major considerations in this respect are the functional group tolerance of a reaction and the stability of specific chemical motifs under reaction conditions. . .

Taking into account the limitations of the current methods, we propose that a lack of understanding regarding the application of a given reaction to non-idealized synthetic problems can result in a reluctance to apply new methodology. Confidence in the utility of a new reaction develops over time—often over a number of years—as the reaction is gradually applied within total syntheses, follow-up methodological papers are published, or personal experience is developed. Unfortunately, even when this information has evolved, it is often widely dispersed, fragmented and difficult to locate. To address this problem, both the tolerance of a reaction to chemical functionality and of the chemical functionality to the reaction conditions must be established when appropriate, and reported in an easily accessible manner, preferably alongside the new methodology.

This is as opposed to the current standard of one or two short tables of different substrates, and then a quick application to some natural product framework. Even those papers, I have to say, are better than some of the stuff in the literature, but we still could be doing better. This paper proposes an additional test: running the reaction in the presence of various added compounds, and reporting the % product that forms under these conditions, the % starting material remaining, and the % additive remaining as well. (The authors suggest using a simple, robust method like GC to get these numbers, which is good advice). This technique will give an idea of the tolerance of the reagents and catalysts to other functional groups, without incorporating them into new substrates, and can tell you if the reaction is just slowed down, or if something about the additive stops everything dead.

Applying this setup to a classic Buchwald amination reaction shows that free aliphatic and aromatic alcohols and amines kill the reaction. Esters and ketones are moderately tolerated. Extraneous heterocycles can slow things down, but not in all cases. But alkynes, nitriles, and amides come through fine: the product forms, and the additives aren't degraded.

I like this idea, and I hope it catches on. But I think that the only way it will is if editors and reviewers start asking for it. Otherwise, it'll be put in the "More work" category, which is easy for authors to ignore. If something like this became the standard, though, all of us synthetic chemists would be better off.

Comments (18) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

June 20, 2013

The Perfect Papers For Each Journal

Email This Entry

Posted by Derek

For the Journal of the American Chemical Society:
"Nanoscale Stuff That We Can Imagine Could Have Gone to Science or Nature, But It Went There First So It Ends Up Here"

"Another Row of Glowing Sample Vials"

For Chemical Communications:
"Wild, Out-There Amalgam of Three or Four Trendy Topics All at Once, All in Two Manuscript Pages, From a Chinese Lab You've Never Heard of"

"A Completely New Assay Technique That Looks Like It Should Need A Twelve-Page Paper, Here In Two Because We're First and Don't Forget It"

For Angewandte Chemie:
"An Actually Useful and Interesting Paper (We Reviewed This One, We Promise), Brought to You With a Wincing, Toe-Curling Pun in the Abstract"

"The First Plutonium-Plutonium Quintuple Bond. Who's Going to Say It Isn't?"

For the Journal of Organic Chemistry:
"Remember Us? Here's an Actual Full Paper About Some Reaction, With Experimental Details and Everything. Where Else Can You Find That, Eh?"

"A Total Synthesis That Would Have Been in JACS Back When, You Whippersnappers"

For Tetrahedron Letters
"Remember Us? Here's a Four-Page Paper About Some Reaction With No Experimental Whatsoever. Where Else Can You Find. . .Oh, Right. Never Mind."

"The Four Thousand And Forty-Seventh Way to Prepare Nitriles From Oximes"

For Organic Letters:
"A Four-Page Paper With No Experimental (Supplementary Info If You're Lucky), But One You Actually Might Want to Read"

"A New Metal-Catalyzed Coupling, Featuring a Catalyst You Can't Buy and Don't Want to Make"

For the Journal of Medicinal Chemistry:
"Big Pharma Here, With a Gigantic Flaming Viking Funeral for a Project That Chewed Up Eight Years, And Here's All We Have to Show For It?"

"Small Academic Lab Here, With A Series of Rhodanines and Polyphenols That Are Seriously Hot Leads for At Least Ten Diseases"

For Science:
"Don't See Much Synthetic Chemistry Over Here, Do You? That's How You Know This is Hot Stuff!"

"People Only Read One or Two Papers Out of Any Issue of This Journal, and This Isn't One of Them, is It?"

For Synthesis:
"As Long as There are Five-Membered Heterocyclic Systems, and German Labs to Make Every Possible Derivative of Them, We Will Survive"

For SynLett
"The Number of Four-Page Organic Chemistry Manuscripts Is Larger Than You Can Comprehend. Obviously."

For ACS Chemical Biology, ChemBioChem, Nature Chemical Biology, Chemistry and Biology, et very much al.:
"Look, We Have NMR Spectra and Cell Culture Conditions in the Same Article, and It Isn't Med-Chem, So Where Else Do We Publish? Right."

Update: I've left out some journals haven't I?

For Bioorganic and Medicinal Chemistry Letters:
"We Wanted to Publish This in J. Med. Chem., But It's Been So Long That We Lost Half the Analytical Data, So Here You Are"

"A Quick Resume-Building Paper, Part XVI, But Man, You Sure Need a Lot From This Journal to Build a Resume These Days"

For Bioorganic and Medicinal Chemistry:
"No One in History Has Ever Read This Journal Without Being Sent Here by a Literature Search, So It Doesn't Matter What Title We Give This. Cauliflower Bicycle Zip-Zang."

For Chemistry: A European Journal:
"German Flexibility, Italian Thoroughness, and the French Work Ethic Have Combined to Bring You This Research, Funded by a List of Euro-Acronyms That Takes Up Half a Page"

Comments (78) + TrackBacks (0) | Category: The Scientific Literature

June 13, 2013

Watching DNA Polymerase Do Its Thing

Email This Entry

Posted by Derek

Single-molecule techniques are really the way to go if you're trying to understand many types of biomolecules. But they're really difficult to realize in practice (a complaint that should be kept in context, given that many of these experiments would have sounded like science fiction not all that long ago). Here's an example of just that sort of thing: watching DNA polymerase actually, well, polymerizing DNA, one base at a time.

The authors, a mixed chemistry/physics team at UC Irvine, managed to attach the business end (the Klenow fragment) of DNA Polymerase I to a carbon nanotube (a mutated Cys residue and a maleimide on the nanotube did the trick). This give you the chance to use the carbon nanotube as a field effect transistor, with changes in the conformation of the attached protein changing the observed current. It's stuff like this, I should add, that brings home to me the fact that it really is 2013, the relative scarcity of flying cars notwithstanding.

The authors had previously used this method to study attached lysozyme molecules (PDF, free author reprint access). That second link is a good example of the sort of careful brush-clearing work that has to be done with a new system like this: how much does altering that single amino acid change the structure and function of the enzyme you're studying? How do you pick which one to mutate? Does being up against the side of a carbon nanotube change things, and how much? It's potentially a real advantage that this technique doesn't require a big fluorescent label stuck to anything, but you have to make sure that attaching your test molecule to a carbon nanotube isn't even worse.
It turns out, reasonably enough, that picking the site of attachment is very important. You want something that'll respond conformationally to the actions of the enzyme, moving charged residues around close to the nanotube, but (at the same time) it can't be so crucial and wide-ranging that the activity of the system gets killed off by having these things so close, either. In the DNA polymerase study, the enzyme was about 33% less active than wild type.

And the authors do see current variations that correlate with what should be opening and closing of the enzyme as it adds nucleotides to the growing chain. Comparing the length of the generated DNA with the FET current, it appears that the enzyme incorporates a new base at least 99.8% of the time it tries to, and the mean time for this to happen is about 0.3 milliseconds. Interestingly, A-T pair formation takes a consistently longer time than C-G does, with the rate-limiting step occurring during the open conformation of the enzyme in each case.

I look forward to more applications of this idea. There's a lot about enzymes that we don't know, and these sorts of experiments are the only way we're going to find out. At present, this technique looks to be a lot of work, but you can see it firming up before your eyes. It would be quite interesting to pick an enzyme that has several classes of inhibitor and watch what happens on this scale.

It's too bad that Arthur Kornberg, the discoverer of DNA Pol I, didn't quite live to see such an interrogation of the enzyme; he would have enjoyed it very much, I think. As an aside, that last link, with its quotes from the reviewers of the original manuscript, will cheer up anyone who's recently had what they thought was a good paper rejected by some journal. Kornberg's two papers only barely made it into JBC, but one year after a referee said "It is very doubtful that the authors are entitled to speak of the enzymatic synthesis of DNA", Kornberg was awarded the Nobel for just that.

Comments (5) + TrackBacks (0) | Category: Analytical Chemistry | Biological News | The Scientific Literature

June 11, 2013

The Overselling of Ionic Liquids

Email This Entry

Posted by Derek

Ionic liquids (molten salts at relatively low temperatures) have been a big feature of the chemical literature for the last ten or fifteen years - enough of a feature to have attracted a few disparaging comments here, from me and from readers. There's a good article out now that talks about the early days of the field and how it grew, and it has some food for thought in it.

The initial reports in the field didn't get much attention (as is often the case). What seems to have made things take off was the possibility of replacing organic solvents with reusable, non-volatile, and (relatively) non-toxic alternatives. "Green chemistry" was (and to an extent still is) a magnet for funding, and it was the combination of this with ionic liquid (IL) work that made the field. But not all of this was helpful:

The link with green chemistry during the development of the IL field, propelled both fields forward, but at times the link was detrimental to both fields when overgeneralizations eroded confidence. ILs were originally considered as green since many of these liquid salts possess a negligible vapor pressure and might replace the use of volatile organic solvents known to result in airborne chemical contamination. The reported water stability and non-volatility led to the misconception that these salts were inherently safe and environmentally friendly. This was exacerbated by the many unsubstantiated claims that ILs were ‘green’ in introductions meant to provide the motivation for the study, even if the study itself had nothing to do with green chemistry. While it is true that the replacement of a volatile organic compound (VOC) might be preferred, proper knowledge of the chemistry of the ions must also be taken into account before classifying anything as green. Nonetheless, the statement “Ionic Liquids are green” was widely published (and can still be found in papers published today). Given the number and nature of the possible ions comprising ILs, these statements are similar to “Water is green, therefore all solvents are green.”

There were many misunderstandings at the chemical level as well:

However, just as the myriad of molecular solvents (or any compounds) can have dramatic differences in chemical, physical, and biological properties based on their chemical identity, so too can ILs. With the potential for 10^18 ion combinations, a single crystal structure of one compound is not a good representation of the chemistry of the entire class of salts which melt below 100 °C and would be analogous to considering carbon tetrachloride as a model system for all known molecular solvents.

The realization that hexafluorophosphate counterions can indeed generate HF under the right conditions helped bring a dose of reality back to the field, although (as the authors point out), not without a clueless backlash that decided, for a while, that all ionic liquids were therefore intrinsically toxic and corrosive. The impression one gets is that the field has settled down, and that its practitioners are more closely limited to people who know what they are talking about, rather than having quite so many who are doing it because it's hot and publishable. And that's a good thing.

Comments (12) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

More on the GSK Shanghai Scandal

Email This Entry

Posted by Derek

The accusations of data fabrication at GlaxoSmithKline's China research site are quite real. That's what we get from the latest developments in the case, as reported by BioCentury, Pharmalot, and the news section at Nature Medicine. Jingwu Zang, lead author on the disputed paper and former head of the Shanghai research site, has been dismissed from the company. Other employees are on administrative leave while an investigation proceeds, and GSK has said it has begun the process of retracting the paper itself.

As for what's wrong with the paper in question, BioCentury Extra has this:

GSK said data in a paper published in January 2010 in Nature Medicine on the role of interleukin-7 (IL-7) in autoimmune disease characterized data as the results of experiments conducted with blood cells of multiple sclerosis (MS) patients "when, in fact, the data reported were either the results of experiments conducted at R&D China with normal (healthy donor) samples or cannot be documented at all, suggesting that they well may have been fabricated."

Pharmalot and others also report that GSK is asking all the authors of the paper to sign documents to agree that it be retracted, which is standard procedure at the Nature Publishing Group. If there's disagreement among them, the situation gets trickier, but we'll see what happens.

The biggest questions are unanswered, though, and we're not likely to hear about them except in rumors and leaks. How, for one thing, did this happen in the first place? On whose initiative were results faked? Who was supposed to check up on these results, and was there anything that could have been done to catch this problem earlier? Even more worrying - and you can bet that plenty of people inside GSK are thinking this, too - how many more things have been faked as well? You'd hope that this was an isolated incident, but if someone is willing to whip up a batch of lies like this, they might well be willing to do much more besides.

The involvement of the head of the entire operation (Jingwu Zang) is particularly troubling. Sometimes, in such cases, it turns out that the person at the top just had their name on the paper, but didn't really participate much or even know what was going on. But he's the only person so far in this mess who's been outright fired, which suggests that something larger has happened. We're not going to hear much about it, but you can bet there are some rather worried and upset people digging through this inside GlaxoSmithKline. There had better be.

Comments (30) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 7, 2013

Mutato Nomine De Te Fabula Narratur

Email This Entry

Posted by Derek

Reader may remember the sudden demise of, under threats of legal action. Its author, Paul Brookes, had a steady stream of material pointing out what very much seemed to be altered and duplicated figures in many scientific publications.

Now comes word that the Brazilian researcher (Rai Curi) whose legal threats led to that shutdown has corrected yet another one of his publications. That Retraction Watch link has the details, but I wanted to highlight the corrections involved:

After the publication of this manuscript we observed mistakes in Figures 3A, 4A, and 6A. The representative images related to pAkt (Figure 3A), mTOR total (Figure 4A), and MuRF-1 total (Figure 6A) have been revised. Please note the original raw blots are now provided with the revised Figures as part of this Correction.
In Figure 3A, pAkt panel, the C and CS bands had been duplicated.
In Figure 4A, the bands were re-arranged compared to the original blot.
In Figure 6A, the band for group D was incorrect.

The remaining Figures, results and conclusions are the same as originally reported in the article. The authors apologize for these errors and refer readers to the corrected Figures 3A, 4A, and 6A provided in this Correction.

So I'm certainly glad that Prof. Curi went after a web site that looks for rearranged blots and altered gels. We wouldn't want any of that around. Would we, now.

Comments (32) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

Five Minutes With Any Journal Article You Want?

Email This Entry

Posted by Derek

The literature access service DeepDyve has made an intriguing announcement of a new service they're offering for non-subscribers of scientific journals. For free, you can have access to the full text. . .for five minutes.

Here's more from the Information Culture blog at Scientific American. Obviously, five minutes is not enough to actually read a journal article, but it probably is enough to decide if you really want to pay to see the thing for real. (And I might note, for chemists and biologists, that five minutes is probably enough time to check a procedure in the experimental section). To that end, it's worth noting that many journals do not seem to put their Supplementary Information files behind their paywalls, and thorough experimental details seem more and more to be showing up in those, rather than the main text.

Note: DeepDyve has access to Elsevier, Wiley, and Royal Society of Chemistry journals, among many others. Nature is in there, but not ScienceBut appears to be no Journal of Biological Chemistry, to pick a heavy hitter on the bio end. And for the less-common chemistry needs, there appears to be no access to Heterocycles or the Journal of Heterocyclic Chemistry, and no Phosphorus, Sulfur, although many other out-of-the-way journals do show up. Update: note also that the American Chemical Society does not seem to be a participant at all. . .

But for people without journal access, this could be the best of a number of not-so-good options. I'll give it a try myself next time I run into some reference in a journal that my own institution doesn't subscribe to, and see how it goes. Thoughts and experiences welcome in the comments. . .

Comments (13) + TrackBacks (0) | Category: The Scientific Literature

June 6, 2013

Research Fraud at GSK Shanghai?

Email This Entry

Posted by Derek

Update: the story continues to develop. The scientist mentioned below, Jingwu Zang has been dismissed from GSK, and others are under investigation. The paper itself is in the process of being retracted. More here.

This is quite bad. Reports have been circulating that GlaxoSmithKline is investigating the scientists (and the results) behind this 2010 paper in Nature Medicine.

That first link from Pharmalot mentions this thread at the Chinese site, and similar stuff has been showing up elsewhere. The online speculation is about Jingwu Zang (sometimes appearing as Zhang, the more common transliteration of the name), who was the lead author on the paper. Various postings (from the same person?) claim that Zang has been let go from GSK, and the Biocentury link in the first paragraph says that mail to his corresponding address bounces back.

The paper is (was?) on IL-7's role in autoimmune disease, a perfectly good topic for a drug company research group to be investigating, of course. But now we're going to have to watch to see if any retraction comes out of this - GSK doesn't have to comment on their hiring (and firing) decisions, but I hope that they wouldn't let a fraudulent Nature Medicine paper stand. That's the really disturbing thing about this situation; I'll see if I can explain what I mean.

A critic from outside the drug industry might say "So what? You people publish shady junk all the time? What's another truth-stretching paper, more or less?" Now, I resent implications like that, but at the same time, there have indeed been instances of nasty publication behavior (ghostwriting, etc.), which I deplore. But those things have been driving by the desire to increase sales of approved drugs. They come from overzealous marketing departments clawing for share, trying to get physicians to write for the company's drug over the other choices.

But the further back you go from the elbow-throwing front lines of the market, the less of that stuff you should see. The paper under scrutiny is early-stage research; it could have come from any good lab (academic or industrial) studying T-cell behavior, multiple sclerosis, or autoimmune mechanisms. Frankly, most of the shady stuff (and retractions) in this kind of work come from academia: the viciously competitive front lines of their market are publications in prestigious journals (like Nature Medicine), which directly bear on funding and tenure decisions. Drug companies have an incentive to stretch the truth about how wonderful their current drug is, not about what their scientists have discovered about biochemistry and cell biology. That doesn't bring in any money.

But what a publication like that does bring in, perhaps, is internal prestige. If you're trying to show what a big deal your particular branch of the company is, and what high-quality work they do, this would be one good way to do it. Keep in mind, publications like this are not the primary goal of people in the drug business; it's not like academia. The job of a drug company research group is to increase the number of drugs the company finds, and publishing in a good journal really doesn't have much to do with that. This publication, though, is a way of telling everyone else - other drug companies, other academic and industrial scientists, other departments and higher-ups at GSK, who may or may not know much about immunology per se, that GSK's Shanghai labs do good enough work to get it into Nature Medicine.

And while we're talking about this, let's talk about another widely-held belief about pharma research branches in China. There have, of course, been a number of these opened over the last five or ten years. And there are a lot of good scientists in China, and there are a lot of research topics that are relevant to the needs of a big drug company, so why not? But it's also widely assumed - although this is certainly not written down anywhere - that the Chinese government very much encourages big foreign companies to start such operations in China itself. If you lend your company's internationally known name to an operation in Shanghai (or wherever), if you invest in getting that site going, if you hire a big group of Chinese nationals to work there and manage things. . .well, the Chinese authorities are just going to like you more. Aren't they? And while being liked by the authorities is never a bad thing in any country in the world, particularly in a heavily regulated industry like pharmaceuticals, it is a particularly good thing in some of them.

This is an unfortunate situation. I believe very strongly in a government of laws, not of men - appropriately enough for where I work, that phrase was written by John Adams into the Constitution of Massachusetts. It's an ideal very difficult to realize, particularly since both Massachusetts and the rest of the world are stocked with human beings, but ideals are supposed to be difficult to realize. I understand that personal connections matter all over the world, and that this is by no means always a bad thing. But the bigger and broader the issues, the more important should be the rule of law.

The particular problem of multinational Chinese research institutes, which this current scandal can only worsen, is that too many people can assume that they've been built mainly to satisfy the Chinese government. They suffer, in other words, from the curse of affirmative action (and other such preference programs): the ever-present suspicion that once merit and ability are made secondary, that all bets are off. (This online debate at The Economist does a good job of airing out such concerns). In other words, the government of China could well end up accomplishing the exact reverse of what it's presumably trying to do: instead of elevating Chinese research (and researchers), it could be damaging the reputations of both.

Comments (58) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 3, 2013

The Current Kudzu of Chemistry?

Email This Entry

Posted by Derek

Chemistry, like any other human-run endeavor, goes through cycles and fads. At one point in the late 1970s, it seemed as if half the synthetic organic chemists in the world had made cis-jasmone. Later on, a good chunk of them switched to triquinane synthesis. More recently, ionic liquids were all over the literature for a while, and while it's not like they've disappeared, they're past their publishing peak (which might be a good thing for the field).

So what's the kudzu of chemistry these days? One of my colleagues swears that you can apparently get anything published these days that has to do with a BODIPY ligand, and looking at my RSS journal feeds, I don't think I have enough data to refute him. There are still an awful lot of nanostructure papers, but I think that it's a bit harder, compared to a few years ago, to just publish whatever you trip over in that field. The rows of glowing fluorescent vials might just have eased off a tiny bit (unless, of course, that's a BODIPY compound doing the fluorescing!) Any other nominations? What are we seeing way too much of?

Comments (35) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

May 10, 2013

Why Not Share More Bioactivity Data?

Email This Entry

Posted by Derek

The ChEMBL database of compounds has been including bioactivity data for some time, and the next version of it is slated to have even more. There are a lot of numbers out in the open literature that can be collected, and a lot of numbers inside academic labs. But if you want to tap the deepest sources of small-molecule biological activity data, you have to look to the drug industry. We generate vast heaps of such; it's the driveshaft of the whole discovery effort.

But sharing such data is a very sticky issue. No one's going to talk about their active projects, of course, but companies are reluctant to open the books even to long-dead efforts. The upside is seen as small, and the downside (though unlikely) is seen as potentially large. Here's a post from the ChEMBL blog that talks about the problem:

. . .So, what would your answer be if someone asked you if you consider it to be a good idea if they would deposit some of their unpublished bioactivity data in ChEMBL? My guess is that you would be all in favour of this idea. 'Go for it', you might even say. On the other hand, if the same person would ask you what you think of the idea to deposit some of ‘your bioactivity data’ in ChEMBL the situation might be completely different.

First and foremost you might respond that there is no such bioactivity data that you could share. Well let’s see about that later. What other barriers are there? If we cut to the chase then there is one consideration that (at least in my experience) comes up regularly and this is the question: 'What’s in it for me?' Did you ask yourself the same question? If you did and you were thinking about ‘instant gratification’ I haven’t got a lot to offer. Sorry, to disappoint you. However, since when is science about ‘instant gratification’? If we would all start to share the bioactivity data that we can share (and yes, there is data that we can share but don’t) instead of keeping it locked up in our databases or spreadsheets this would make a huge difference to all of us. So far the main and almost exclusive way of sharing bioactivity data is through publications but this is (at least in my view) far too limited. In order to start to change this (at least a little bit) the concept of ChEMBL supplementary bioactivity data has been introduced (as part of the efforts of the Open PHACTS project,

There's more on this in an article in Future Medicinal Chemistry. Basically, if an assay has been described in an open scientific publication, the data generated through it qualifies for deposit in ChEMBL. No one's asking for companies to throw open their books, but even when details of a finished (or abandoned) project are published, there are often many more data points generated than ever get included in the manuscript. Why not give them a home?

I get the impression, though, that GSK is the only organization so far that's been willing to give this a try. So I wanted to give it some publicity as well, since there are surely many people who aren't aware of the effort at all, and might be willing to help out. I don't expect that data sharing on this level is going to lead to any immediate breakthroughs, of course, but even though assay numbers like this have a small chance of helping someone, they have a zero chance of helping if they're stuck in the digital equivalent of someone's desk drawer.

What can be shared, should be. And there's surely a lot more that falls into that category than we're used to thinking.

Comments (18) + TrackBacks (0) | Category: Drug Assays | The Scientific Literature

May 8, 2013

Total Synthesis in Print

Email This Entry

Posted by Derek

Over at the Baran group's "Open Flask" blog, there's a post on the number of total synthesis papers that show up in the Journal of the American Chemical Society. I'm reproducing one of the figures below, the percentage of JACS papers with the phrase "total synthesis" in their title.
You can see that the heights of the early 1980s have never been reached again, and that post-2000 there has been a marked drought. As the post notes, JACS seems to have begun publishing many more papers in total around that time (anyone notice this or know anything about it?), and it appears that they certainly didn't fill the new pages with total synthesis. 2013, though, already looks like an outlier, and it's only May.

My own feelings about total synthesis are a matter of record, and have been for some time, if anyone cares. So I'm not that surprised to see the trend in this chart, if trend it is.

But that said, it would be worth running the same analysis on a few other likely journal titles. Has the absolute number of total synthesis papers gone down? Or have they merely migrated (except for the really exceptional ones) to the lower-impact journals? Do fewer papers put the phrase "Total synthesis of. . ." in their titles as compared to years ago? Those are a few of the confounding variables I can think of, and there are probably more. But I think, overall, that the statement "JACS doesn't publish nearly as much total synthesis as it used to" seems to be absolutely correct. Is this a good thing, a bad thing, or some of each?

Comments (31) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

May 7, 2013

One Case of Plagiarism Down. Two Zillion to Go.

Email This Entry

Posted by Derek

You may remember this case from Chemistry - A European Journal earlier this year, where a paper appeared whose text was largely copy-pasted from a previous JACS paper from another lab. This one has finally been pulled; Retraction Watch has the details.

The most interesting part is that statement "The authors regret this approach", which I don't recall ever seeing in a situation like this. The comments at Retraction Watch build on this, and are quite interesting. There are many countries (and cultures) where it's considered acceptable (or at least a venial sin) to lift passages verbatim from other English-language papers when you're publishing in that language. I can see the attraction - I would hate to have to deliver a scientific manuscript in German, for example, which is the closest thing I have to a second language.

But I still wouldn't do it by copying and pasting big hunks of text, either. Reasons for resorting to that range from wanting to be absolutely sure that things are being expressed correctly in ones third or fourth language, all the way to "Isn't that how it's supposed to be done?" The latter situation obtains in parts of Asia, where apparently there's an emphasis in some schools on verbatim transcription of authoritative sources. There's an interesting cite to Yu Hua's China in Ten Words, where one of those ten words is "copycat" (shanzhai):

As a product of China’s uneven development, the copycat phenomenon has as many negative implications as it has positive aspects. The moral bankruptcy and confusion of right and wrong in China today, for example, and vivid expression in copycatting. As the copycat concept has gained acceptance, plagiarism, piracy, burlesque, parody, slander, and other actions originally seen as vulgar or illegal have been given a reason to exist; and in social psychology and public opinion they have gradually acquired respectability. No wonder that “copycat” has become one of the words most commonly used in China today. All of this serves to demonstrate the truth of the old Chinese saying: “The soil decides the crop, and the vine shapes the gourd.”

Four years ago I saw a pirated edition of [my novel] Brothers for sale on the pedestrian bridge that crosses the street outside my apartment; it was lying there in a stack of other pirated books. When the vendor noticed me running my eyes over his stock, he handed me a copy of my novel, recommending it as a good read. A quick flip through and I could tell at once that it was pirated. “No, it’s not a pirated edition,” he corrected me earnestly. “It’s a copycat.”

This tendency isn't a good fit with a lot of things, but it especially doesn't work out so well with scientific publication. I haven't seen it stated in so many words, but a key assumption is that every scientific paper is supposed to be different. If you take the time to read a new paper, you should learn something new and you should see something that you haven't seen before. It might be trivial, it might well be useless, but it should be at least slightly different from any other paper you've read or could find.

Now, as the Retraction Watch comments mention, some of these plagiarism cases are examples of "templating", where original (or sort of original) work was done, but the presentation of it was borrowed from an existing paper. That's not as bad as faking up results completely, of course, but you still have to wonder about the value of your work if you can lift big swaths of someone else's paper to describe it. Even when the manuscript itself has been written fresh from the ground up, there's plenty of stuff out in the literature like this. Someone gets an interesting reaction with a biphenyl and a zinc catalyst, and before you know it, there are all these quickie communications where someone else says "Hey, we got that with a napthyl", or "Hey, we got that with a boron halide catalyst". Technically, yes, these are different, but we're in the land of least publishable units now, where the salami is sliced so thinly that you can read a newspaper through it.

So the authors regret this approach, do they? So does everyone else.

Comments (9) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

April 30, 2013

Is Glyphosate Poisoning Everyone?

Email This Entry

Posted by Derek

I've had a few people send along this article, on the possible toxicological effects of the herbicide glyphosate, wondering what I make of it as a medicinal chemist. It's getting a lot of play in some venues, particularly the news-from-Mother-Nature outlets. After spending some time reading this paper over, and looking through the literature, I've come to a conclusion: it is, unfortunately, a load of crap.

The authors believe that glyphosate is responsible for pretty much every chronic illness in humans, and a list of such is recited several times during the course of the long, rambling manuscript. Their thesis is that the compound is an inhibitor of the metabolizing CYP enzymes, of the biosynthesis of aromatic amino acids by gut bacteria, and of sulfate transport. But the evidence given for these assertions, and their connection with disease, while it might look alarming and convincing to someone who has never done research or read a scientific paper, is a spiderweb of "might", "could", "is possibly", "associated with", and so on. The minute you look at the actual evidence, things disappear.

Here's an example - let's go right to the central thesis that glyphosate inhibits CYP enzymes in the liver. Here's a quote from the paper itself:

A study conducted in 1998 demonstrated that glyphosate inhibits cytochrome P450 enzymes in plants [116]. CYP71s are a class of CYP enzymes which play a role in detoxification of benzene compounds. An inhibitory effect on CYP71B1l extracted from the plant, Thlaspi arvensae, was demonstrated through an experiment involving a reconstituted system containing E. coli bacterial membranes expressing a fusion protein of CYP71B fused with a cytochrome P450 reductase. The fusion protein was assayed for activity level in hydrolyzing a benzo(a)pyrene, in the presence of various concentrations of glyphosate. At 15 microM concentration of glyphosate, enzyme activity was reduced by a factor of four, and by 35 microM concentration enzyme activity was completely eliminated. The mechanism of inhibition involved binding of the nitrogen group in glyphosate to the haem pocket in the enzyme.
A more compelling study demonstrating an effect in mammals as well as in plants involved giving rats glyphosate intragastrically for two weeks [117]. A decrease in the hepatic level of cytochrome P450 activity was observed. As we will see later, CYP enzymes play many important roles in the liver. It is plausible that glyphosate could serve as a source for carcinogenic nitrosamine exposure in humans, leading to hepatic carcinoma. N-nitrosylation of glyphosate occurs in soils treated with sodium nitrite [118], and plant uptake of the nitrosylated product has been demonstrated [119]. Preneoplastic and neoplastic lesions in the liver of female Wistar rats exposed to carcinogenic nitrosamines showed reduced levels of several CYP enzymes involved with detoxification of xenobiotics, including NADPH-cytochrome P450 reductase and various glutathione transferases [120]. Hence this becomes a plausible mechanism by which glyphosate might reduce the bioavailability of CYP enzymes in the liver.
Glyphosate is an organophosphate. Inhibition of CYP enzyme activity in human hepatic cells is a well-established property of organophosphates commonly used as pesticides [121]. In [122], it was demonstrated that organophosphates upregulate the nuclear receptor, constitutive androstane receptor (CAR), a key regulator of CYP activity. This resulted in increased synthesis of CYP2 mRNA, which they proposed may be a compensation for inhibition of CYP enzyme activity by the toxin. CYP2 plays an important role in detoxifying xenobiotics [123].

Now, that presumably sounds extremely detailed and impressive if you don't know any toxicology. What you wouldn't know from reading through all of it is that their reference 121 actually tested glyphosate against human CYP enzymes. In fact, you wouldn't know that anyone has ever actually done such an experiment, because all the evidence adduced in the paper is indirect - this species does that, so humans might do this, and this might be that, because this other thing over here has been shown that it could be something else. But the direct evidence is available, and is not cited - in fact, it's explicitly ignored. Reference 121 showed that glyphosate was inactive against all human CYP isoforms except 2C9, where it had in IC50 of 3.7 micromolar. You would also not know from this new paper that there is no way that ingested glyphosate could possibly reach levels in humans to inhibit CYP2C9 at that potency.

I'm not going to spend more time demolishing every point this way; this one is representative. This paper is a tissue of assertions and allegations, a tendentious brief for the prosecution that never should have been published in such a form in any scientific journal. Ah, but it's published in the online journal Entropy, from the MDPI people. And what on earth does this subject have to do with entropy, you may well ask? The authors managed to work that into the abstract, saying that glyphosate's alleged effects are an example of "exogenous semiotic entropy". And what the hell is that, you may well ask? Why, it's a made-up phrase making its first appearance, that's what it is.

But really, all you need to know is that MDPI is the same family of "journals" that published the (in)famous Andrulis "Gyres are the key to everything!" paper. And then made all kinds of implausible noises about layers of peer review afterwards. No, this is one of the real problems with sleazy "open-access" journals. They give the whole idea of open-access publishing a black eye, and they open the floodgates to whatever ridiculous crap comes in, which then gets "peer reviewed" and "published" in an "actual scientific journal", where it can fool the credulous and mislead the uninformed.

Comments (133) + TrackBacks (0) | Category: The Scientific Literature | Toxicology

April 26, 2013

Research Fraud, From A Master Fraud Artist

Email This Entry

Posted by Derek

A couple of years back, I wrote about the egregious research fraud case of Diederick Stapel. Here's an extraordinary follow-up in the New York Times Magazine, which will give you the shivers. Here, try this part out:

In one experiment conducted with undergraduates recruited from his class, Stapel asked subjects to rate their individual attractiveness after they were flashed an image of either an attractive female face or a very unattractive one. The hypothesis was that subjects exposed to the attractive image would — through an automatic comparison — rate themselves as less attractive than subjects exposed to the other image.

The experiment — and others like it — didn’t give Stapel the desired results, he said. He had the choice of abandoning the work or redoing the experiment. But he had already spent a lot of time on the research and was convinced his hypothesis was valid. “I said — you know what, I am going to create the data set,” he told me. . .

. . .Doing the analysis, Stapel at first ended up getting a bigger difference between the two conditions than was ideal. He went back and tweaked the numbers again. It took a few hours of trial and error, spread out over a few days, to get the data just right.

He said he felt both terrible and relieved. The results were published in The Journal of Personality and Social Psychology in 2004. “I realized — hey, we can do this,” he told me.

And that's just what he did, for the next several years, leading to scores of publications and presentations on things he had just made up. In light of that Nature editorial statement I mentioned yesterday, this part seems worth thinking on:

. . . The field of psychology was indicted, too, with a finding that Stapel’s fraud went undetected for so long because of “a general culture of careless, selective and uncritical handling of research and data.” If Stapel was solely to blame for making stuff up, the report stated, his peers, journal editors and reviewers of the field’s top journals were to blame for letting him get away with it. The committees identified several practices as “sloppy science” — misuse of statistics, ignoring of data that do not conform to a desired hypothesis and the pursuit of a compelling story no matter how scientifically unsupported it may be.

The adjective “sloppy” seems charitable. . .

It may well be. The temptation of spicing up the results is always there, in any branch of science, and it's our responsibility to resist it. That means not only resisting the opportunities to fool others, it means resisting fooling ourselves, too, because who would know better what we'd really like to hear? Reporting only the time that the idea worked, not the other times when it didn't. Finding ways to explain away the data that would invalidate your hypothesis, but giving the shaky stuff in your favor the benefit of the doubt. N-of-1 experiments taken as facts. No, not many people will go as far as Diederick Stapel (or could, even if they wanted to - he was quite talented at fakery). Unfortunately, things go on all the time that might differ from him in degree, but not in kind.

Comments (27) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

April 25, 2013

Towards Better Papers, With Real Results in Them

Email This Entry

Posted by Derek

This has to be a good thing. From the latest issue of Nature comes news of an initiative to generate more reproducible papers:

From next month, Nature and the Nature research journals will introduce editorial measures to address the problem by improving the consistency and quality of reporting in life-sciences articles. To ease the interpretation and improve the reliability of published results we will more systematically ensure that key methodological details are reported, and we will give more space to methods sections. We will examine statistics more closely and encourage authors to be transparent, for example by including their raw data. . .

. . .We recognize that there is no single way to conduct an experimental study. Exploratory investigations cannot be done with the same level of statistical rigour as hypothesis-testing studies. Few academic laboratories have the means to perform the level of validation required, for example, to translate a finding from the laboratory to the clinic. However, that should not stand in the way of a full report of how a study was designed, conducted and analysed that will allow reviewers and readers to adequately interpret and build on the results.

I hope that Science, the Cell journals at Elsevier, and other other leading outlets for such results will follow through with something similar. In this time of online supplementary info and basically unlimited storage ability, there's no reason not to disclose as much information as possible in a scientific publication. And the emphasis on statistical rigor and possible sources of error is just what's needed as well. Let's see who follows suit first, and congratulate them. And let's see who fails to respond, and treat them appropriately, too.

Comments (7) + TrackBacks (0) | Category: The Scientific Literature

April 8, 2013

Scientific Posters, Heads on Platters, and Lawsuits

Email This Entry

Posted by Derek

Yep, these all tie together. Have a look at this post at Retraction Watch for the details. It's about Colin Purrington, who has a web site on designing posters for conferences. I hadn't seen it before, but it's attained quite a bit of popularity (as it should; it seems to be full of sound advice). Purrington himself has put a lot of work into it, and has decided to protect his copyright.

That means that you have to police these things. I do a little of that myself, when I come across cheapo content-scraping blog sites that are just ripping off my posts, one after the other. What's silly about that is that I almost always grant permission to reprint things if someone goes to the trouble of asking. Colin Purrington seems to have had his hands full with people helping themselves to his work, and the latest example was from the Consortium for Plant Biotechnology Research. He sent them a please-take-this-down notice, and his notices apparently lean towards the colorful. It included a request for the head, on a platter, of whoever it was that decided to rip him off without attribution. He did offer to pay for shipping.

That didn't go over too well. He's received one of those the-sky-shall-fall-upon-you letters from CPBR's expensive lawyers, quoting copyright law to him and accusing him of taking his information from them. (There are archives of Purrington's material going back to 1997, so that should be fun to dispose of). And he was also informed that the staff took his head/platter request as a physical threat, worth contacting authorities about if repeated.

I'm sure there will be more to this story. But so far, I think that we can conclude that no matter how expensive your legal counsel, you're going to have to pay them even more if you expect them to exhibit a sense of humor.

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

April 1, 2013

Fake Journals - But They'd Like Real Money

Email This Entry

Posted by Derek

I wish that this were an April Fool's entry - and a number of scientists would like for that to have been the case, too. Nature reports that at least two journals (Archives des Sciences from Geneva and Wulfenia, a botany journal from Austria) have had their names hijacked by scam artists. Neither journal really had a web presence, so some enterprising sleazeball(s) decided to give each of them a convincing one. They were convincing enough to fool Thomson Reuters for months, and enough to get an unnamed number of authors to think that they'd published papers - after, I should add, sending publication fees to banks in Armenia. That last detail might (or should) have caused some worry, but who would have imagined a top-to-bottom counterfeit journal operation?

The journal "sites" even include editorial board members, some of whom seem to be fictitious, and some of whom are very much not (and were very much not aware that their names were being used). So if you're looking for evidence of how profitable scientific publishing can be, look no further: it's valuable enough to fake.

Comments (8) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

March 29, 2013

The Price of Publishing

Email This Entry

Posted by Derek

So, how much does it cost to publish a scientific paper, anyway? I'm not only talking about how much it costs you. That varies from journal to journal, and from type of journal to type of journal. One aspect of most open-access publishing models is that the author defrays editorial costs. (Which model is, of course, open to abuse by the sleazy). How much does it cost the publishers themselves, and how much do they make on it? There's an excellent overview at Nature that tries to put some real numbers on these questions:

Data from the consulting firm Outsell in Burlingame, California, suggest that the science-publishing industry generated $9.4 billion in revenue in 2011 and published around 1.8 million English-language articles — an average revenue per article of roughly $5,000. Analysts estimate profit margins at 20–30% for the industry, so the average cost to the publisher of producing an article is likely to be around $3,500–4,000.

In case you were wondering why we have so many journals. And if you're still wondering, for some reason, try these numbers on:

Scientists pondering why some publishers run more expensive outfits than others often point to profit margins. Reliable numbers are hard to come by: Wiley, for example, used to report 40% in profits from its science, technology and mathematics (STM) publishing division before tax, but its 2013 accounts noted that allocating to science publishing a proportion of 'shared services' — costs of distribution, technology, building rents and electricity rates — would halve the reported profits. Elsevier's reported margins are 37%, but financial analysts estimate them at 40–50% for the STM publishing division before tax. (Nature says that it will not disclose information on margins.) Profits can be made on the open-access side too: Hindawi made 50% profit on the articles it published last year. . .

Might I add that scientific publishing, for all the upheavals in it, is probably a slightly less risky bet than drug discovery? I keep planning to do a big post on pharmaceutical profit margins - and when I do, it's going to sound like an accounting seminar - and this makes me want to move it closer to the top of the list.

Comments (10) + TrackBacks (0) | Category: The Scientific Literature

March 25, 2013

Advertising in the Supplementary Information?

Email This Entry

Posted by Derek

Here's a publication concern I'd never come across before. A reader sends word that an ACS journal asked him and his co-authors to remove the names of vendors and manufacturers in their Supporting Information, over concerns that this might be seen as some form of advertising. I think they were specifically thinking of whether the authors might have had academic discounts, etc., that influenced their selection of reagents and equipment.

But while I can see that point, I also think it's important to name suppliers. Any experienced chemist knows that a palladium catalyst from one supplier may well not be the same as one from another supplier, for example (unpaid, unsolicited endorsement: stick with Strem). To pick another issue, HPLC columns come in as many varieties as there are manufacturers - how are you supposed to honestly list your experimental details if you can't say whose columns you used? I don't see how you can have a complete writeup without these details, and I think that this outweighs the concerns about discounts.

My correspondent suggests a compromise: list all the brands, but also state whether any discounts were received. Has anyone else run into this issue?

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

March 14, 2013

Scientists and Google Reader's Demise

Email This Entry

Posted by Derek

I suspect that many people follow this blog through its RSS feed. And I feel sure that many readers here follow the current scientific literature that way. Journals are updated constantly, and that's the most concentrated way to get all the new information in one place for flipping through. (No more "new journal table" in the library, is there?)

Well, as you've probably heard, the site that many of us have been using to do all this is closing. Google Reader is to be shut down on July 1. Problems had been apparent for some time now, but this still took me by surprise. And now the search is on for a replacement.

Feedly is apparently trying to clone the service on their own, so that's a possibility. And The Old Reader seems to be an effort to recreate the service as well, going back to some sharing functionality that Google stripped out a while back in the interest of promoting Google+. I'll be evaluating these and others.

What I already know is this: many RSS-based services seem to be colorful-picture-tile things (like Flipboard), and for the chemical literature, they're of no use to me. I am feeling more like a dinosaur every time I say this, but I don't own a tablet (or not yet), and I wish that web sites would find a way to deliver their content both ways: in concentrated blasts of scrolling info for people using a more conventional desktop (or who just like it that way) and in big, flippy, roomy, tablet-screen-sized chunks for those who like it that way instead. One size doesn't fit all.

And that's where Google Reader will be missed, unless someone else can step up. The scientific literature needs a tool like this - we have hundreds and hundreds of new articles coming along all the time, and while scrolling through them in RSS isn't ideal, it's a lot better than any other solution I've come across. Looking at the various comments around the web about Reader's demise, I see that it's hard-core information nerds that are mourning it most - well, if scientists don't fit that description, they should. We're industrial consumers of information, and we need industrial-strength tools.

Update: here's the best list of alternatives I've seen so far.

Comments (35) + TrackBacks (0) | Category: The Scientific Literature

March 6, 2013

Anonymity, Fakery, et al.

Email This Entry

Posted by Derek

I wanted to link to this piece at C&E News on the whole question of anonymity when it comes to comments on the chemical literature. This was brought on by the advent of Blog Syn, but it applied before that, and will continue to apply to other situations.

Its author, Fredrik von Kieseritzky, also calls for synthetic details to make it back into the body of papers, rathe than being relegated to the Supporting Information (which is never as carefully refereed as the manuscript itself). That would be a good thing, but I despair of it happening, at least until the major journals break down and admit that their page count restrictions for submissions are, in large part, relics of the days when everyone read them in print. (They serve another useful function, thought, which is getting people to tighten up their writing. "There wasn't enough time to make it shorter" is a real phenomenon).

But the rest of the commentary grew out of this piece by Rich Apodaca, whose morning it is around here. He wonders about the use of pseudonyms in science, where author recognition has long been a big motivating factor. von Kieseritzky's take is that he can see why people go anonymous (and Rich lists some very plausible reasons, too), but that he's never regretted using his own name online.

That goes for me, too. The topic of anonymity has come up here several times over the years: in chem-blogging, and in peer review of publications and grants. I'm glad that I've used my real name over the years on this blog (although it hasn't always been a smooth ride), but I also think that anonymity is a necessary option, although it certainly can be abused.

That opinion is not shared by the (pseudonymous) author of this piece in the Journal of Cell Science. It's a bit of dystopian what-if, an agitated response to the (now taken down) Science Fraud site. "Mole VIII" relates how some people (an extremely small percentage) did indeed fake scientific papers, and how this embittered other people who had been unable to make the careers in science that they wished to. So they started web sites where they cried "Fake!" about papers of all kinds, which forced the authors to spend all their time defending themselves. Many of them were driven out of doing science, whereupon they turned to exposing their former colleagues as the next best thing. And then, in one generation, science was done - stopped forever, in a hurricane of finger-pointing and snide remarks.

What a load. For one thing, I think that fakery, while not rampant, is more widespread than many people think. And even if it isn't, I think that legitimate results stand up to challenges of this sort, while the shady ones collapse at a push. Furthermore, I find the whole cycle-of-bitterness conceit ridiculous. A look back at the history of science will show that accusations of fakery and bad faith have been with us forever, and often in much more vitriolic form than today.

One problem might be that the author is a bit too academic. Try this part:

Soon, there were very few scientists left. And then fewer. Public confidence for publicly funded research disappeared. The only research that was done any more was kept secret and in the corporations. And while this gave us many new package designs for the sale of established drugs, the actual idea of ‘doing science’, of making discoveries to share with a community of interested and devoted researchers, dwindled, and finally, vanished.

Yep, that's about the size of it - package designs. I try to stay alert to threats to the scientific endeavor, and I try not to take it for granted. But I'm willing to put my real name on the opinion that the author of this stuff is being foolish.

Comments (14) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

Open Access For ACS Articles?

Email This Entry

Posted by Derek

Rich Apodaca investigates something that I didn't know, either: that the ACS provides the corresponding authors of papers with links to their articles, which (1) allow for fifty free downloads during the first year after publication, and (2) allow for unlimited free downloads after that. I thought about that for a while, and couldn't remember any examples of such a link, not that I'd noticed, at any rate.

Apodaca's having trouble reducing it to practice, too. He has been trying to get such a link for one of his own papers in J. Med. Chem., and . . .well, his post will tell you about all the places he's looked so far. Let's just say that the ACS does not make it obvious where a corresponding author is supposed to obtain such a URL. Has anyone out there tried this, and has anyone had any success?

Comments (12) + TrackBacks (0) | Category: The Scientific Literature

February 26, 2013

Phil Baran at Blog Syn

Email This Entry

Posted by Derek

I linked recently to the latest reaction check at Blog Syn, benzylic oxidation by IBX. Now Prof. Baran (a co-author on the original paper, from his Nicoloau days) has written See Arr Oh with a detailed repeat of the experiment. He gets it to work, so I think it's fair to say that (1) the reaction is doable, but (2) it's not as easy to reproduce right out of the box as it might be.

I'd like to congratulate him for responding like this. The whole idea of publicly rechecking literature reactions is still fairly new, and (as the comments here have shown), there's a wide range of opinion on it. Getting a detailed, prompt, and civil response from the Baran lab is the best outcome, I think. After all, the point of a published procedure - the point of science - is reproducibility. The IBX reaction is now better known than it was, the details that could make it hard to run are now there for people who want to try it, and Prof. Baran's already high reputation as a scientist actually goes up a bit among the people who've been following this story.

Public reproducibility is an idea whose time, I think, has come, and Blog Syn is only one part of it. When you think about the increasingly well-known problems with reproducing big new biological discoveries, things that could lead to tens and hundreds of millions being spent on clinical research, reproducing organic chemistry reactions shouldn't be controversial at all. As they say to novelists, if you're afraid of bad reviews, there's only one solution: don't show anyone your book.

Comments (59) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

February 22, 2013

What If the Journal Disappears?

Email This Entry

Posted by Derek

Hmm, here's a question I hadn't considered. What happens when an online-only journal quits publishing and (apparently) deletes its archives? That's what seems to have happened with the "Journal of Advances in Developmental Research".

Now, to a first approximation, the loss of many of the papers in this journal will not, in all likelihood, be much of a setback. Here is (was?) its stated focus:

The Journal of Advances in Developmental Research is a peer-reviewed multidisciplinary journal that publishes research articles, general articles, research communications, review article and abstracts of theses from the fields of science, social sciences, sports science, humanities, medical, education, engineering, technology, biotechnology, home science, computer, history, arts and other fields which participates in overall development of society.

It provides a platform to discuss current and future trends of research and their role in development of society.

Now, that doesn't sound like anything anyone would want to read. But as long as your check cleared, you could publish in it - it was one of those bottom-of-the-barrel predatory publishing venues. What happens now, though? If there was something worthwhile in any of those papers, we'll never have any way of knowing, because they're all gone. Can (or should) the authors resubmit the papers somewhere else where they can be seen?

Here, for reference are Jeffrey Beall's current criteria for a predatory publisher. One of them is that they "(Have) no policies or practices for digital preservation". Although these guys seem to have had a policy, if you count "wipe the hard drive" as a policy.

Tip via Ivan Oransky and Jeffrey Beall on Twitter.

Comments (15) + TrackBacks (0) | Category: The Scientific Literature

February 21, 2013

An Incentive For Hype

Email This Entry

Posted by Derek

Here's an article illustrating what goes into high-profile journal publications, and why you should always read past the title and the abstract. Björn Brembs noticed this paper coming out in Current Biology on fruit fly receptors and behavior, whose abstract claims that "blocking synaptic output from octopamine neurons inverts the valence assigned to CO2 and elicits an aversive response in flight". As Brembs puts it:

We currently have a few projects in our lab that target these octopamine neurons, so this was a potentially very important finding. It was my postdoc, Julien Colomb, who spotted the problem with this statement first. In fact, if it wasn't for Julien, I might have never looked at the data myself, as I know the technique and I know and trust the lab the paper was from. I probably would just have laid the contents of the abstract to my memory and cited the paper where appropriate, as the results confirmed our data and those in the literature (a clear case of confirmation bias on my part).

When you look harder, you find that yes, the genetically manipulated flied do seem averse to carbon dioxide plumes. But when you check the control experiments, you find that the two transgenes added to the flies (independent of the change to the octopamine system that's the subject of the paper) both decrease the tropism for CO2. So there's really no way of knowing what the effect of both of them might be, octopamine signaling or not, and you might well suspect that the two of them together could hose up the carbon dioxide response without invoking the receptor pathways at all.

As Brembs says, though, the authors aren't trying to hide this. It's in the body of their paper. Abstract be damned, the paper itself states:

"We note that the Tdc2-GAL4/+ driver line does not spend a significantly greater amount of time in the CO2 plume by comparison to air, but this line, as well as the UAS-TNT/+ parent line, spends significantly more time in the CO2 plume in comparison to their progeny. Therefore, this experimental result cannot be fully attributable to the genetic background."

No, not fully attributable at all, especially if the progeny show some sort of additive effect of the two transgenes. Of course, if you water down your conclusions too much, you might not get the paper into as good a journal as you'd like. I'll let Brembs sum up:

To make this unambiguously clear: I can't find any misconduct whatsoever in this paper, only clever marketing of the sort that occurs in almost every 'top-journal' paper these days and is definitely common practice. On the contrary, this is exactly the behavior incentivized by the current system, it's what the system demands, so this is what we get. It's precisely this kind of marketing we refer to in our manuscript, that is selected for in the current evolution of the scientific community. If you don't do it, you'll end up unemployed. It's what we do to stay alive.

If there's anyone out there who thinks that this doesn't go on in the chemistry literature, my advice is to please look around you a bit. This sort of thing goes on all the time, and I'd guess that most of us automatically dial down the statements in paper titles and abstracts as we read them, without even realizing any more that we're doing so. But in a case like this (and there are many others), even that process will still let erroneous conclusions into your head. And we all have enough of those already.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

February 19, 2013

The Wages of Copy-Pasting

Email This Entry

Posted by Derek

A few weeks ago I mentioned this situation regarding work by Prof. Xi Yan. Two recent papers seem to have been substantially copy-pasted from earlier work published by completely different groups. Now See Arr Oh has some details on what happens to you when you have the nerve to do that in a journal of the Royal Society of Chemistry: why, you have to publish a note regretting that you didn't cite the paper you copied from, that's what. "The authors apologize for this oversight."

There, that should square things up. Right? See Arr Oh is not very happy about this response, and I don't blame him for a minute. The RSC editors seem to be ignoring the word-for-word aspect of a substantial part of the new paper; it really is a paste job, and you're not supposed to do that. And the only problem they have is that the paper being stolen from wasn't cited? Oversight, my various body parts.

Comments (20) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

More From Blog Syn

Email This Entry

Posted by Derek

I wanted to mention that there are two more entries up on Blog Syn: one of them covering this paper on alkenylation of pyridines. (It's sort of like a Heck reaction, only you don't have to have an iodo or triflate on the pyridine; it just goes right into the CH bond). The short answer: the reaction works, but there are variables that seem crucial for its success that were under-reported in the original paper (and have been supplied, in part, by responses from the original author to the Blog Syn post). Anyone thinking about running this reaction definitely needs to be aware of this information.

The latest is a re-evaluation of a older paper on the use of IBX to (among many other things) oxidize arylmethane centers. It's notable for a couple of reasons: it's been claimed that this particular reaction completely fails across multiple substrates, and the reaction itself is from the Nicolau lab (with Phil Baran as a co-author). Here's the current literature situation:

A day in the library can save you a week in the lab, so let’s examine this paper’s impact using SciFinder: it's been cited 179 times from 2002-2013. Using the “Get Reactions" tool, coupled with SciFinder’s convenient new “Group by Transformation” feature, we identified 54 reactions from the citing articles that can be classified as “Oxidations of Arylmethanes to Aldehydes/Ketones" (the original reaction's designation). Of these 54 reactions, only four (4) use the conditions reported in this paper, and all four of those come from one article: Binder, J. T.; Kirsch, S. F. Org. Lett. 2006, 8, 2151–2153, which describes IBX as “an excellent reagent for the selective oxidation to generate synthetically useful 5-formylpyrroles.” Kirsch's yields range from 53-79% for relatively complex substrates, not too shabby.

I'll send you over to Blog Syn for the further details, but let's just say that not many NMR peaks are being observed around 10 ppm. Phil Baran himself makes an appearance with more details about his recollection of the work (to his credit). Several issues remain, well, unresolved. (If any readers here have ever tried the reaction, or have experience with IBX in general, I'm sure comments would be very welcome over there as well).

Comments (73) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

February 8, 2013

All Those Drug-Likeness Papers: A Bit Too Neat to be True?

Email This Entry

Posted by Derek

There's a fascinating paper out on the concept of "drug-likeness" that I think every medicinal chemist should have a look at. It would be hard to count the number of publications on this topic over the last ten years or so, but what if we've been kidding ourselves about some of the main points?

The big concept in this area is, of course, Lipinski criteria, or Rule of Five. Here's what the authors, Peter Kenny and Carlos Montanari of the University of São Paulo, have to say:

No discussion of drug-likeness would be complete without reference to the influential Rule of 5 (Ro5) which is essentially a statement of property distributions for compounds taken into Phase II clinical trials. The focus of Ro5 is oral absorption and the rule neither quantifies the risks of failure associated with non-compliance nor provides guidance as to how sub-optimal characteristics of compliant compounds might be improved. It also raises a number of questions. What is the physicochemical basis of Ro50s asymmetry with respect to hydrogen bond donors and acceptors? Why is calculated octanol/water partition coefficient (ClogP) used to specify Ro50s low polarity limit when the high polarity cut off is defined in terms of numbers of hydrogen bond donors and acceptors? It is possible that these characteristics reflect the relative inability of the octanol/water partitioning system to ‘see’ donors (Fig. 1) and the likelihood that acceptors (especially as defined for Ro5) are more common than donors in pharmaceutically-relevant compounds. The importance of Ro5 is that it raised awareness across the pharmaceutical industry about the relevance of physico- chemical properties. The wide acceptance of Ro5 provided other researchers with an incentive to publish analyses of their own data and those who have followed the drug discovery literature over the last decade or so will have become aware of a publication genre that can be described as ‘retrospective data analysis of large proprietary data sets’ or, more succinctly, as ‘Ro5 envy’.

There, fellow med-chemists, doesn't this already sound like something you want to read? Thought so. Here, have some more:

Despite widespread belief that control of fundamental physicochemical properties is important in pharmaceutical design, the correlations between these and ADMET properties may not actually be as strong as is often assumed. The mere existence of a trend is of no interest in drug discovery and strengths of trends must be known if decisions are to be accurately described as data-driven. Although data analysts frequently tout the statistical significance of the trends that their analysis has revealed, weak trends can be statistically significant without being remotely interesting. We might be confident that the coin that lands heads up for 51 % of a billion throws is biased but this knowledge provides little comfort for the person charged with predicting the result of the next throw. Weak trends can be beaten and when powered by enough data, even the feeblest of trends acquires statistical significance.

So, where are the authors going with all this entertaining invective? (Not that there's anything wrong with that; I'm the last person to complain). They're worried that the transformations that primary drug property data have undergone in the literature have tended to exaggerate the correlations between these properties and the endpoints that we care about. The end result is pernicious:

Correlation inflation becomes an issue when the results of data analysis are used to make real decisions. To restrict values of properties such as lipophilicity more stringently than is justified by trends in the data is to deny one’s own drug-hunting teams room to maneuver while yielding the initiative to hungrier, more agile competitors.

They illustrate this by reference to synthetic data sets, showing how one can get rather different impressions depending on how the numbers are handled along the way. Representing sets of empirical points by using their average values, for example, can cause the final correlations to appear more robust than they really are. That, the authors say, is just what happened in this study from 2006 ("Can we rationally design promiscuous drugs?) and in this one from 2007 ("The influence of drug-like concepts on decision-making in medicinal chemistry"). The complaint is that showing a correlation between cLogP and median compound promiscuity does not imply that there is one between cLogP and compound promiscuity per se. And the authors note that the two papers manage to come to opposite conclusions about the effect of molecular weight, which does make one wonder. The "Escape from flatland" paper from 2009 and the "ADMET rules of thumb" paper from 2008 (mentioned here) also come in for criticism on this point - binning averaged data from a large continuous set and then treated those as real objects for statistic analysis. Ones conclusions depend strongly on how many bins one uses. Here's a specific take on that last paper:

The end point of the G2008 analysis is ‘‘a set of simple interpretable ADMET rules of thumb’’ and it is instructive to examine these more closely. Two classifications (ClogP<4 and MW<400 Da; ClogP>4 or MW>400 Da) were created and these were combined with the four ionization state classifications to define eight classes of compound. Each combination of ADMET property and compound class was labeled according to whether the mean value of the ADMET property was lower than, higher than or not significantly different from the average for all compounds. Although the rules of thumb are indeed simple, it is not clear how useful they are in drug discovery. Firstly, the rules only say whether or not differences are significant and not how large they are. Secondly, the rules are irrelevant if the compounds of interest are all in the same class. Thirdly, the rules predict abrupt changes in ADMET properties going from one class to another. For example, the rules predict significantly different aqueous solubility for two neutral compounds with MW of 399 and 401 Da, provided that their ClogP values do not exceed 4. It is instructive to consider how the rules might have differed had values of logP and MW of 5 and 500 Da (or 3 and 300 Da) had been used to define them instead of 4 and 400 Da.

These problems also occur in graphical representations of all these data, as you'd imagine, and the authors show several of these that they object to. A particular example is this paper from 2010 ("Getting physical in drug discovery"). Three data sets, whose correlations in their primary data do not vary significantly, generate very different looking bar charts. And that leads to this comment:

Both the MR2009 and HY2010 studies note the simplicity of the relationships that the analysis has revealed. Given that drug discovery would appear to be anything but simple, the simplicity of a drug-likeness model could actually be taken as evidence for its irrelevance to drug discovery. The number of aromatic rings in a molecule can be reduced by eliminating rings or by eliminating aromaticity and the two cases appear to be treated as equivalent in both the MR2009 and HY2010 studies. Using the mnemonic suggested in MR2009 one might expect to make a compound more developable by replacing a benzene ring with cyclohexadiene or benzoquinone.

The authors wind up by emphasizing that they're not saying that things like lipophilicity, aromaticity, molecular weight and so on are unimportant - far from it. What they're saying, though, is that we need to be aware of how strong these correlations really are so that we don't fool ourselves into thinking that we're addressing our problems, when we really aren't. We might want to stop looking for huge, universally applicable sets of rules and take what we can get in smaller, local data sets within a given series of compounds. The paper ends with a set of recommendations for authors and editors - among them, always making primary data sets part of the supplementary material, not relying on purely graphical representations to make statistical points, and a number of more stringent criteria for evaluating data that have been partitioned into bins. They say that they hope that their paper "stimulates debate", and I think it should do just that. It's certainly given me a lot of things to think about!

Comments (14) + TrackBacks (0) | Category: Drug Assays | Drug Development | In Silico | The Scientific Literature

February 4, 2013

A Word We Didn't Know We Needed

Email This Entry

Posted by Derek

I've written here about the strange-sounding conferences that keep sending out invitations to all and sundry. They tend to be in provincial Chinese cities, have grandiose names like the "First International Summit Meeting of Advanced Medical Science", and feature so many sections, sessions, tracks, and breakouts that you wonder if anyone attends who isn't giving a talk. And you get invitations to submit invited talks in fields you've hardly even touched on in your career; that's another reliable sign.

Well, I've had one this morning whose title really rises to the top of the list. I present to you the "1st Annual Symposium of Drug Designology". No, I did not make that up - if I had, I wouldn't tell anyone, believe me. And I'm not about to provide a link to the conference site. If you want more, I'm willing to bet that a search for "drug designology" will yield only highly relevant hits, since I'm not aware of that phrase ever appearing in English until this morning. Here's hoping it submerges again.

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

January 23, 2013

Coincidence, No Doubt. Well, Some Doubt.

Email This Entry

Posted by Derek

Has anyone happened to read this paper, from 2009, or this one, from this year? Well, Shawn Burdette of WPI has, and he noticed that (to a significant extent) they're the same paper. Prof. Valerie Pierre of Minnesota, author of the first paper, is reportedly not too amused, and I don't blame her. But hey, the 2013 authors did at least cite her paper. . .in reference 14d. So at least there's that.

Update: but wait, there's more!

Comments (12) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

January 16, 2013

Blog Syn: An Idea Whose Time Has Come?

Email This Entry

Posted by Derek

Here's something I've been following for the last couple of weeks in the chemical blogging world, and now it's up on its own site: "Blog Syn", an initiative of the well-known chemblogger See Arr Oh. The idea here is to take interesting new reactions that appear in the literature, and. . .well, see if they actually work. (A radical concept, I know, but stick with me here).

The first example is this recent paper in JACS, which shows an unusual iron-sulfur reaction that ends up generating a benz-azole directly from an active methyl group in one pot. There are now three repeats of the reaction, and the verdict (so far) is that it works, but not quite as well as hoped for. You probably have to be careful to exclude oxygen (the paper itself just says "under argon"), and the yield of the test reaction is not has high as reported. As you'll see, there are spectral data, sources of reagents, photos of experimental setups - everything you'd need to see how this reaction is actually being (re)run.

I like this idea very much, and I look forward to seeing it applied to new reactions as they appear (and I hope to contribute the occasional run myself, when possible). They're accepting nominations for the next reaction to test, so if you have something you've seen that you're wondering about, put it into the hopper.

Comments (19) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

January 4, 2013

Science Fraud and Legal Action

Email This Entry

Posted by Derek

There have been occasional links here in the comments to, although I'm not sure if I ever linked them directly or not. Note the use of the past tense: as detailed here at Retraction Watch, the site has suddenly gone (mostly) dark under threats of legal action. Nothing appears on the Wayback Machine at, either.

I'm not all that surprised. I've said unkind things about people and organizations on this blog, but Science Fraud seemed to have that pedal pushed down to the floor the entire time. And while I've had threats of legal action, I think that I've managed to stay just this side of defamation, although with some people that's hard to do. (I mean for that to be interpreted both ways - both that it's hard to avoid saying nasty things about some people, but also that in such cases, it's hard to think of things that are nasty enough to be defamatory). But which side of that line Paul Brookes, the now-public U. Rochester scientist behind the Science Fraud site, has landed on is still up for debate. More as this story develops. . .

Comments (4) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

An Article That Shows What Med-Chem Is Like?

Email This Entry

Posted by Derek

Here's a query that I received the other day that I thought I'd pass on to the readership: "What's the one journal article or book chapter that you'd assign to a class to show them what medicinal chemistry and drug discovery are really like?"

That's a tricky one, because (as in many fields) the "what it's really like" aspect doesn't always translate to the printed page. But I'd be interested in seeing some suggestions.

Comments (15) + TrackBacks (0) | Category: Life in the Drug Labs | The Scientific Literature

December 21, 2012

The Last Thing a Professor Wants to Hear

Email This Entry

Posted by Derek

This can't be good. A retraction in PNAS on some RNA-driven cell death research from a lab at Caltech:

Anomalous experimental results observed by multiple members of the Pierce lab during follow-on studies raised concerns of possible research misconduct. An investigation committee of faculty at the California Institute of Technology indicated in its final report on this matter that the preponderance of the evidence and the reasons detailed in the report established that the first author falsified and misrepresented data published in this paper. An investigation at the United States Office of Research Integrity is ongoing.

As that link from Retraction Watch notes, the first author himself was not one of the signees of that retraction statement - as one might well think - and he now appears to be living in London. He appears to have left quite a mess behind in Pasadena.

Comments (14) + TrackBacks (0) | Category: Biological News | The Dark Side | The Scientific Literature

December 14, 2012

The Merck Index

Email This Entry

Posted by Derek

So the Royal Society of Chemistry has bought the Merck Index, and plans to try to raise its profile, especially online. I wish them luck, but I'm not sure how well that's going to work out. I have a copy, but it's an old one that I got for free when a library turned over its stock. There are years that go by that its pages stay undisturbed.

I think that the chemical substance entries on Wikipedia, among other things, have moved into the space once occupied by reference works like this. Now, it's true that many people would rather point to a standard reference work like the Merck Index than to Wikipedia, and that may well be the market right there. Is there, or can there be, more of one? An advertiser-supported online substance reference might have a niche, but it would have been a bigger niche if it had been colonized ten years ago.

Comments (14) + TrackBacks (0) | Category: The Scientific Literature

November 16, 2012

Part Eleven? Really?

Email This Entry

Posted by Derek

And here's another item, sent in by a reader, who noted this publication in Bioorganic and Medicinal Chemistry Letters. I have no problem with the work at all, and certainly no problem with the people who did it (some of whom I know), but Part Eleven? I'm trying to figure out why this would be sliced quite so thinly - the only thing that comes to mind is to scatter a wide group of co-authors across several publications, so as to give everyone something on their CV. But how much does a multipart BOMCL count for - heck, while we're on the subject, how much does most any publication count for in today's hiring environment? Update: note that this is not one of those multiyear series things - most of these appear to be in press right now.

Comments (7) + TrackBacks (0) | Category: The Scientific Literature

Your Press Release Has Already Been Written

Email This Entry

Posted by Derek

Alan Dove has it right here:

In a groundbreaking new study, scientists at Some University have discovered that a single molecule may drive people to perform that complex behavior we’ve all observed. Though other researchers consider the results of the small, poorly structured experiment misleading, a well-written press release ensures that their criticisms will be restricted to brief quotes buried near the bottoms of most news stories on the work, if they’re included at all.

There's more at the link, and believe me, you've seen releases that conform to this template so perfectly, it's eerie. I'm reminded of this famous BBC news report. . .

Comments (3) + TrackBacks (0) | Category: Business and Markets | The Scientific Literature

Hexacyclinol Round-Up

Email This Entry

Posted by Derek

For those wanting a timeline of the whole hexacyclinol business, with links to the articles, blogs, and commentary that's surrounded it, allow me to recommend Carmen Drahl's "History of the Hexacyclinol Hoo-Hah". (And no, the whole thing is not written in alliteration; for that, you'll be wanting this).

Comments (4) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

November 14, 2012

Hexacyclinol Retracted. It Only Took Six Years.

Email This Entry

Posted by Derek

Via Chemjobber's Twitter feed comes news of this: the formal retraction of the LaClair hexacyclinol synthesis.

The retraction has been agreed due to lack of sufficient Supporting Information. In particular, the lack of experimental procedures and characterization data for the synthetic intermediates as well as copies of salient NMR spectra prevents validation of the synthetic claims. The author acknowledges this shortcoming and its potential impact on the community

Potential? After six years? There were people taking their first undergraduate organic course when this controversy hit who are now thinking about how to start tying together their PhD dissertations. It seems that Angewandte Chemie is very loath to go the full-retraction route (there haven't been many), but that retraction notice doesn't bring up anything that wasn't apparent after the first ten minutes of reading the paper.

Update: Wavefunction isn't too impressed, either.

Comments (18) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

November 6, 2012

How Much Fraud, As Opposed to Plain Old Error?

Email This Entry

Posted by Derek

How many retracted papers, would you say, are due to honest error rather than fraud and other misconduct? We now can put a number on that, thanks to this paper. The authors have looked over all 2,047 paper listed on PubMed from the life sciences as "retracted" (better them than me), with the earliest going back to 1977. The authors are careful to point out that this is absolutely an underestimate, though, with several examples of papers which are known to be fraudulent but have never been officially retracted. But they find that:

. . .only 21.3% of retractions were attributable to error. In contrast, 67.4% of retractions were attributable to misconduct, including fraud or suspected fraud (43.4%), duplicate publication (14.2%), and plagiarism (9.8%).

They blame incomplete and outright misleading retraction notices for confusing the issue about these numbers. (I've always liked, in a teeth-gritting way, the idea of a dubious retraction notice - it gives these things the full surround-sound sensory experience). Many published retractions that blame things like "flaws in the data analysis" turn out, on follow-up, to have been the subject of investigations that strongly suggested fraud.

Other trends: the US, Germany, Japan, and China accounted for the majority of papers pulled because of fraud, but China and India each stand out a bit in a crowded plagiarism field (China also stand out in the "duplicate publication" category). Higher-impact journals were significantly more likely to have papers retracted because of outright fraud rather than plagiarism (a result that makes sense, and squares with my own experience as a reader). And retractions have definitely been increasing over time, probably with several factors operating at once (greater incentives to fraud, coupled with increased detection). The paper sums up this way:

Given that most scientific work is publicly funded and that retractions because of misconduct undermine science and its impact on society, the surge of retractions suggests a need to reevaluate the incentives driving this phenomenon. We have previously argued that increased retractions and ethical breaches may result, at least in part, from the incentive system of science, which is based on a winner-takes-all economics that confers disproportionate rewards to winners in the form of grants, jobs, and prizes at a time of research funding scarcity.

Fixing this, though, will not be easy. There are recommendations for an increased focus on ethics training (which will do nothing at all, I think, to stop the sort of person who would do these sorts of things). But they also call for some standardization of retraction notices, with minimum standards of disclosure, which sounds like a good idea, and also for trying to find some way to reward scientists that doesn't involve publishing a ton of papers. I like that idea, too, although the implementation will be tricky. . .

Comments (10) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

November 1, 2012

Hype, Malpractice, and Scientific Misconduct in Organic Synthesis

Email This Entry

Posted by Derek

That's the word-for-word title of a provocative article by Rolf Carlson and Tomas Hudlicky in Helvetica Chimica Acta. That journal's usually not quite this exciting, but it is proud of its reputation for compound characterization and experimental accuracy. That probably helped this manuscript find a home there, where it's part of a Festschrift issue in honor of Dieter Seebach's 75th birthday.

The authors don't hold back much (and Hudlicky has not been shyabout these issues, either, as some readers will know). So, for the three categories of malfeasance described in the title, the first (hype) includes the overblown titling of many papers:

As long as the foolish use of various metrics continues there is little hope of return to integrity. Young scientists entering academia and competing for resources and recognition are easily infected with the mantra of importance of
publishing in 'high-impact journals' and, therefore, strive to make their work as noticeable as possible by employing excess hype.

It is the reader, not the author, of papers describing synthetic method who should evaluate its merits. Therefore, self-promoting words like 'novel', 'new', 'efficient', 'simple', 'high-yielding', 'versatile', 'optimum' should not be used in the title of the paper if such qualities are not covered by the actual content of the paper.

It also includes the inflation of reaction yields (see that link in the second paragraph above for more on that topic). This is another one that's going to be hard to fix:

Unfortunately, the community has chosen and continues to choose the yield values in submitted manuscripts as a measure of overall quality and/or utility of the report. This, of course, encourages the 􏰛'adjustment' in the values in order to avoid critique. An additional problem in the reported values is the fact that synthesis is performed on small scales, thanks to advances in NMR and other techniques available for structure determination. On milligram scales it is extremely difficult to accurately determine weight and content of a sample, given the equipment available in typical academic laboratory.

The second category, malpractice, is sloppy work, but not outright fraud:

Malpractice, as explained above, is usually not deliberate and derives primarily from ignorance or professional incompetence. The most frequent cases involve improper experimental protocols, improper methods used in characterization of compounds, and the lack of correct citations to previous work.

For example, the authors point out that very, very rarely are any new synthetic methods given a proper optimization. One-variable one-at-a-time changes are worthwhile, but they're not sufficient to explore a reaction manifold, not when these changes can interact with each other. As process chemists in industry know, the only way to explore such landscapes is with techniques such as Design of Experiments (DoE), which try to find out what factors in a multivariate system produce the greatest change in results. Here's an example; the process chemistry literature furnishes many more.

And finally, you have outright scientific misconduct - fraud, poaching of ideas from grant applications and the like, plagiarism in publications, etc. It's hard to get a handle on these - they seem to be increasing, but the techniques to find and expose them are also getting better. Over time, thought, these techniques might just have the effect of making fraud more sophisticated; that would be in line with human behavior as I understand it, and with selection pressure as well. The motives for such acts are with us still, and do not seem to be abating much, so I tend to think that determined miscreants will find ways to do what they want to do.

Thoughts? Some of this paper's points could be put in the "grumblings about the good old days" category, but I think that a lot of it is accurate. I'm not sure how good the old days were, myself, since they were also filled with human beings, but the pressures found today do seem to be bringing on a lot of behaviors we could do without.

Comments (71) + TrackBacks (0) | Category: Chemical News | The Dark Side | The Scientific Literature

October 31, 2012

Oops. We Didn't Mean to Publish That.

Email This Entry

Posted by Derek

Here's an interesting situation, courtesy of Retraction Watch - trying to pull back a paper because it disclosed something that was supposed to be the subject of your patent. Say the authors of the paper in the Korean Journal of Physiology and Pharmacology:

We regret to inform that the published paper included a few parts that disclosed confidential information which should have been protected under patent law. We admit that the request for retraction is due to the indiscretion of the authors, and confirmed that editorial committee of KJPP have not conducted any fault in publishing the paper.

I would think that if you've disclosed, you've disclosed, so this will all come down to timing. Shouldn't matter much whether the paper has been retracted or not. . .

Comments (8) + TrackBacks (0) | Category: Patents and IP | The Scientific Literature

October 24, 2012

Too Many Med Chem Journals?

Email This Entry

Posted by Derek

The recent discussions on awful molecules in the med-chem literature got me to thinking the other day. There are many more med-chem journals now than there used to be. Back in the day, you had the Journal of Medicinal Chemistry, and that was that. Then in the early 1990s came Bioorganic and Medicinal Chemistry and Bioorganic and Medicinal Chemistry Letters. Now we've got ACS Medicinal Chemistry Letters and MedChemComm and ChemMedChem, among others. (And as we were seeing the other day, there are levels and levels as you move down, ranging from obscure-but-respectable, to just plain obscure, all the way to disreputable crap).

This at a time when industrial medicinal chemistry has been getting cut back. Now, on the other side of the ledger, you have all the folks who are cleaning out the file cabinets for publishable material to have on their C.V.s (and don't think that's not a real force). But in general, I have to wonder if the amount of medicinal chemistry being published from industry hasn't been going down.

Here, then is the question: is there enough decent medicinal chemistry being done to usefully fill all the pages of all these journals? My answer, as you might guess, is "Probably not". But I'd welcome other opinions.

Comments (20) + TrackBacks (0) | Category: The Scientific Literature

October 22, 2012

The Absolute Bottom of the Publishing Barrel

Email This Entry

Posted by Derek

Every so often, you come across scientific journals that you've absolutely, completely never heard of. Back in graduate school (mid-1980s for me), I used to keep track of the weirdest references that came up - Journal of the Siberian Oil Chemist's Society, or Bulletin of the Kentucky Academy of Sciences (1954), which I think you'd have a hard time laying your hands on even in Kentucky. Then there are all the obscure "flag carrier" journals. One that shows up fairly often in searches for odd heterocyclic systems in the Egyptian Journal of Chemistry, but there are others that I have never seen a reference to in nearly 30 years of looking at the chemical literature, such as the Revista Colombiana de Quimica. Europe used to be covered with national chemistry titles, most of which have ceased publication or were merged into Chem. Eur. J. or the like. But some of the newly independent countries were glad to start up their own literature, so you have (for example) the Journal of the Serbian Chemical Society.

Now, I have no wish to offend any Serbian readership I may have, but I will not be bringing any unexpected news if I point out that JSCS is not the most prestigious venue available. In the old days, such a title would be full of local papers, and to be sure, there are plenty of manuscripts from Belgrade. But there are also plenty from places like Brazil, Iran, Egypt, Pakistan, and (naturally) the further corners of India and China. I suspect that some authors from these countries get to count such papers as having been published in a European scientific journal, as opposed to the less-impactful venues closer to home. There is, for example, an Iranian Journal of Chemistry and Chemical Engineering, as there is a Journal of the Brazilian Chemical Society.

But these days, there's a much larger and fuzzier category of obscure journal, and we have the internet and the idea of open access to thank for them. Well, those and greed. If I had to pick, I'd say that greed is the main factor. I'm talking about scam publishing, the dozens upon dozens of "open access" journals that have sprung up that (1) accept everything, and (2) charge a significant publication fee. That money is supposed to cover the costs of editorial work and open access on publication - and such fees can be completely legitimate, of course. But in the case of these publishers, it's a scam, since there are very, very few costs involved. No one edits these papers to any significant degree, and to a good approximation, no one ever accesses the papers, either. Bandwidth charges are thus held down to manageable levels.

Here's a good resource on these outfits, Beall's List of Predatory Open-Access Publishers. Jeffrey Beall of the University of Colorado-Denver has compiled a list of shady operations, most of which are characterized by suspiciously vast lists of titles and hefty publication charges. The one publisher on the list that you might have heard of is Bentham Open, the "open-access" arm of Bentham Publishing. I've always considered their regular list of journals to be pretty borderline stuff, although they have published some useful reviews. But Beall characterizes Bentham Open as "a scholarly vanity press", and that seems pretty accurate.

Take, for example, their Open Medicinal Chemistry Journal. It appears to have published two papers so far this year. Last year, it put out a special issue on "Medicinal Chemistry of Novel Anti-Diabetic Drugs", which sounds interesting until you note that there are three papers therein: a leadoff editorial (from an author at the University of the United Arab Emirates), a paper from that editorial writer and several collaborators (four authors, four countries), and still another paper from him and one of the authors of the first paper. Hmm.

Now, the scholarly worth of such things can be debated. They're of little immediate interest, but if the results contained are real, then they are, presumably, tiny bricks in the great edifice of scientific knowledge, and might conceivably be useful to someone, someday. From that standpoint, I don't have much room to criticize them. But since I've said many unkind things about the established scientific publishing houses and their business models, it's only fair that I point out that some of the untraditional ones are just as rapacious. The sorts of "journals" on Beall's list are not even pretending to add anything to the store of human knowledge: they're publication mills, turning anything you want to pay for into a "scientific paper". Some (not all) of the authors may deserve sympathy, by virtue of their obscure, unfunded origins (although they must have enough funds to pay for these papers), but the publishers deserve none at all for taking advantage of them. And when they're not taking advantage of ignorance and/or desperation, then the transaction is a cynical one indeed, reminding me of the old joke from the Soviet Union that went "As long as they pretend to pay me, I'll pretend to work".

Will they really publish anything? Why, yes, they will, as a mathematician proved by submitting a paper full of incoherent gibberish and getting it accepted. He used MathGen, a modified version of the random-paper generator SciGen that I've written about here. You'd think that the institutional address of "University of Southern North Dakota at Hoople" would tip someone off, but there are no P.D.Q. Bach fans in that audience.

Comments (27) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

October 1, 2012

Six Rule of Reproducibility

Email This Entry

Posted by Derek

Bruce Booth has a look at some rules suggested by Glenn Begley of Amgen, who's been involved in trying to reproduce published data. He's had enough bad experiences in that line (and he's not alone) that he's advocating these standards for evaluating something new and exciting:

1) Were studies blinded?

2) Were all results shown?

3) Were experiments repeated?

4) Were positive and negative controls shown?

5) Were reagents validated?

6) Were the statistical tests appropriate?

Applying these tests would surely scythe down an awful lot of the literature - but a lot of the stuff that would be weeded out would deserve it. I really wonder, for example, how many n=1 experiments make it into print; I'm sure it's far more than anyone would be comfortable with if we knew the truth. As I've mentioned here before, different fields have different comfort levels with what needs to be done to assure reproducibility, but I think that everyone would agree that complex biology experiments need all the backing up that they can get. The systems are just too complex, and there are too many places were things can go silently wrong.

That "Were all results shown" test is a tough one, too. Imagine a synthetic paper where each reaction has a note next to it, like "4/5", to show the number of times the reaction worked out of the total number of times it was tried. There would be a lot of "2/2", which would be fine, and (in total synthesis papers) some "1/1" stuff in the later stages, which readers could take or leave. But wouldn't it be instructive to see the "1/14"s in print? We never will, though. . .

Comments (27) + TrackBacks (0) | Category: The Scientific Literature

September 20, 2012

Various Links Of Stuff

Email This Entry

Posted by Derek

Swamped with all sorts of stuff today - when science marches on, you have to make sure that it's not leaving its bootprints on your back. But I do have some interesting links:

The bluest of blue-sky brain research, funded by Paul Allen. Fascinating stuff, on several levels - here's a big publication that came out this week. I find the phenomenon of tech-billionaire funding for things like this, asteroid mining, low-cost orbital access and the like very encouraging. (And of course, the Gates Foundation is doing a lot in more earthbound pursuits).

The Wall Street Journal reveals what is apparently a rather ill-kept secret: most firms funded by venture capital fail. "Most", as in about 3 out of 4. That's a loose definition, though - as the article says, if you're talking total wipeout of capital, then that's about one third of them. If you're talking about failing to see the projected return in the projected time, well, that's over 90%. But it's all about the ones that succeed, just like the drug business.

The Royal Society of Chemistry, in a rather self-congratulatory press release, pledges money to help authors publish their work open-access in RSC journals. The UK government is putting money into this, but no one's sure if it'll be enough.

Do you want to make this compound? No? Neither do I. Especially not when they turn around and stick three more nitro groups onto it.

Comments (12) + TrackBacks (0) | Category: Business and Markets | The Central Nervous System | The Scientific Literature

September 14, 2012

Walking Away From the ACS

Email This Entry

Posted by Derek

Here's a look at the scientific journal situation from the viewpoint of the people who have to shell out the money for institutional subscriptions. This librarian (from SUNY-Potsdam) ended up deciding that the American Chemical Society journal package just cost too much, especially for an institution of her size:

We also learned that their base price and pricing model, when applied to much larger institutions, did not produce the same unsustainable pricing – I cannot provide numbers, as they are marked SUNY Confidential, but I can easily say that what our ARL peers pay for ACS in support of their doctoral programs is, in my estimation, in no way fair or reflective of the usage, FTE, or budgets of those institutions as compared to the pricing offered my institution for my usage, FTE, and budgets. It seems to me that the tiered increases may be fair and be reflective, but the problem lies with the base price underlying their pricing model. That base price is unsustainable for small institutions. And, unfortunately, the ACS sales team is not currently interested in negotiating on that fact. In response to any suggestions of ways that SUNY or campuses might collaborate or negotiate to reach a place where we could sustain our subscriptions – one which might well be applied to other campuses, other consortia by ACS – we were repeatedly told “but that’s not our pricing model.”

Wouldn't want to upset the pricing model, that's for sure. The slow earthquake in scientific publishing continues to rumble on.

Comments (50) + TrackBacks (0) | Category: The Scientific Literature

September 13, 2012

ENCODE And What It All Means

Email This Entry

Posted by Derek

You'll have heard about the massive data wave that hit (30 papers!) courtesy of the ENCODE project. That stands for Encyclopedia of DNA Elements, and it's been a multiyear effort to go beyond the bare sequence of human DNA and look for functional elements. We already know that only around 1% of the human sequence is made up of what we can recognize as real, traditional genes: stretches that code for proteins, have start and stop codons, and so on. And it's not like that's so straightforward, either, what with all the introns and whatnot. But that leaves an awful lot of DNA that's traditionally been known by the disparaging name of "junk", and sure it can't just be that - can it?

Some of it does its best to make you think that way, for sure. Transposable elements like Alu sequences, which are repeated relentlessly hundreds of thousands of times throughout the human DNA sequence, must either be junk, inert spacer, or so wildly important that we just can't have too many copies of them. But DNA is three-dimensional (and how), and its winding and unwinding is crucial to gene expression. Surely a good amount of that apparently useless stuff is involved in these processes and other epigenetic phenomena.

And the ENCODE group has indeed discovered a lot of this sort of thing. But as this excellent overview from Brendan Maher at Nature shows, it hasn't discovered quite as many as the headlines might lead you to think. (And neither has it demolished the idea that all the 99% of noncoding DNA is junk, because you can't find anyone who believed that one, either). The figure that's in all the press writeups is that this work has assigned functions for 80% of the human genome, which would be an astonishing figure on several levels. For one thing, it would mean that we'd certainly missed an awful lot before, and for another, it would mean that the genome is a heck of a lot more information-rich than we ever thought it might be.

But neither of those quite seem to be the case. It all depends on what you mean by "functional", and opinions most definitely vary. See this post by Ed Yong for some of the categories. which range out to some pretty broad, inclusive definitions of "function". A better estimate is that maybe 20% of the genome can directly influence gene expression, which is very interesting and useful, but ain't no 80%, either. That Nature post provides a clear summary of the arguments about these figures.

But even that more-solid 20% figure is going to keep us all busy for a long time. Learning how to affect these gene transcription mechanisms is going should be a very important route to new therapies. If you remember all the hype about how the genome was going to unlock cures to everything - well, this is the level we're actually going to have to work at to make anything in that line come true. There's a lot of work to be done, though. Somehow, different genes are expressed at different times, in different people, in response to a huge variety of environmental cues. It's quite a tangle, but in theory, it's a tangle that can be unraveled, and as it does, it's going to provide a lot of potential targets for therapy. Not easy targets, mind you - those are probably gone - but targets nonetheless.

One of the best ways to get a handle on all this work is this very interesting literature experiment at Nature - a portal into the ENCODE project data, organized thematically, and with access to all the papers involved across the different journals. If you're interested in epigenetics at all, this is a fine place to read up on the results of this work. And if you're not, it's still worth exploring to see how the scientific literature might be presented and curated. This approach, it seems to me, potentially adds a great deal of value. Eventually, the PDF-driven looks-like-a-page approach to the literature will go extinct, and something else will replace it. Some of it might look a bit like this.

Note, just for housekeeping purposes - I wrote this post for last Friday, but only realized today that it didn't publish, thus the lack of an entry that day. So here it is, better late, I hope, than never. There's more to say about epigenetics, too, naturally. . .

Comments (16) + TrackBacks (0) | Category: Biological News | The Scientific Literature

September 6, 2012

Graphical Abstract Tedium

Email This Entry

Posted by Derek

Scrolling through my journal RSS feeds, a question occurs to me. What's the biggest cliché, the most overused trope in graphic abstracts? My nomination is the "row of glowing vials" to illustrate some new fluorescent/luminescent sensor molecule. Nothing wrong with a row of glowing vials per se, but man, has that image ever been done to death. (I'm just glad that I'm not working on anything of the kind, so I don't have to figure out what to show instead). Your nominations? Just think of what makes you grit your teeth as you glance over the journal table of contents, even before you've read the title of the paper, and you'll have it.

Update: as mentioned in the comments, if you want the TOC graphics that are each. . .special. . .in their own way, then look no further than TOCROFL.

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

August 28, 2012

A Synthetic Retraction

Email This Entry

Posted by Derek

There's an odd retraction in the synthetic chemistry literature. A synthesis of the lundurine alkaloid core from the Martin group at Texas was published last year, and its centerpiece was a double-ring-closing olefin metathesis reaction. (Coincidentally, that reaction was one of the "Black Swan" examples in the paper I blogged about yesterday - the initial reports of it from the 1960s weren't appreciated by the synthetic organic community for many years).

Now the notice says that the paper is being retracted because that RCM reaction is "not reproducible". (The cynical among you will already be wondering when that became a criterion for retraction in the literature - if it works once, it's in, right?)

There are more details at The Heterocyclist, a blog by the well-known synthetic chemist Will Pearson that I've been remiss in not highlighting before now. While you're there, fans of the sorts of chemicals I write about in "Things I Won't Work With" might enjoy this post on the high explosive RDX, and the Michigan chemist (Werner Bachmann) who figured out how to synthesize it on scale during World War II.

Comments (18) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

August 27, 2012

Chemistry's Mute Black Swans

Email This Entry

Posted by Derek

What's a Black Swan Event in chemistry? Longtime industrial chemist Bill Nugent has a very interesting article in Angewandte Chemie with that theme, and it's well worth a look. He details several examples of things that all organic chemists thought they knew that turned out not to be so, and traces the counterexamples back to their first appearances in the literature. For example, the idea that gold (and gold complexes) were uninteresting catalysts:

I completed my graduate studies with Prof. Jay Kochi at Indiana University in 1976. Although research for my thesis focused on organomercury chemistry, there was an active program on organogold chemistry, and our perspective was typical for its time. Gold was regarded as a lethargic and overweight version of catalytically interesting copper. More- over, in the presence of water, gold(I) complexes have a nasty tendency to disproportionate to gold(III) and colloidal gold(0). Gold, it was thought, could provide insight into the workings of copper catalysis but was simply too inert to serve as a useful catalyst itself. Yet, during the decade after I completed my Ph.D. in 1976 there were tantalizing hints in the literature that this was not the case.

One of these was a high-temperature rearrangement reported in 1976, and there was a 1983 report on gold-catalyzed oxidation of sulfides to sulfoxides. Neither of these got much attention, as the Nugent's own chart of the literature on the subject shows. (I don't pay much attention when someone oxidizes a sulfide, myself). Apparently, though, a few people had reason to know that something was going on:

However, analytical chemists in the gold-mining industry have long harnessed the ability of gold to catalyze the oxidation of certain organic dyes as a means of assaying ore samples. At least one of these reports actually predates the (1983) Natile publication. Significantly, it could be shown that other precious metals do not catalyze the same reactions, the assays are specific for gold. It is safe to say that the synthetic community was not familiar with this report.

I'll bet not. It wasn't until 1998 that a paper appeared that really got people interested, and you can see the effect on that chart. Nugent has a number of other similar examples of chemistry that appeared years before its potential was recognized. Pd-catalyzed C-N bond formation, monodentate asymmetric hydrogenation catalysts, the use of olefin metathesis in organic synthesis, non-aqueous enzyme chemistry, and many others.

So where do the black swans come into all this? Those familiar with Nasim Taleb's book
will recognize the reference.

The phrase “Black Swan event” comes from the writings of the statistician and philosopher Nassim Nicholas Taleb. The term derives from a Latin metaphor that for many centuries simply meant something that does not exist. But also implicit in the phrase is the vulnerability of any system of thought to conflicting data. The phrase's underlying logic could be undone by the observation of a single black swan.

In 1697, the Dutch explorer Willem de Vlamingh discovered black swans on the Swan River in Western Australia. Not surprisingly, the phrase underwent a metamorphosis and came to mean a perceived impossibility that might later be disproven. It is in this sense that Taleb employs it. In his view: “What we call here a Black Swan (and capitalize it) is an event with the following three attributes. First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact. Third, in spite of its outlier status, human nature makes us concoct an explanation for its occurrence after the fact, making it explainable and predictable.”

Taleb has documented this last point about human nature through historical and psychological evidence. His ideas remain controversial but seem to make a great deal of sense when one attempts to understand the lengthy interludes between the literature antecedents and the disruptive breakthroughs shown. . .At the very least, his ideas represent a heads up as to how we read and mentally process the chemical literature.

I have no doubt that unwarranted assumptions persist in the conventional wisdom of organic synthesis. (Indeed, to believe otherwise would suggest that disruptive break- throughs will no longer occur in the future.) The goal, it would seem, is to recognize such assumptions for what they are and to minimize the time lag between the appearance of Black Swans and the breakthroughs that follow.

One difference between Nugent's examples and Taleb's is the "extreme impact" part. I think that Taleb has in mind events in the financial industry like the real estate collapse of 2007-2008 (recommended reading here
), or the currency events that led to the wipeout of Long-Term Capital Management in 1998. The scientific literature works differently. As this paper shows, big events in organic chemistry don't come on as sudden, unexpected waves that sweep everything before them. Our swans are mute. They slip into the water so quietly that no one notices them for years, and they're often small enough that people mistake them for some other bird entirely. Thus the time lag.

How to shorten that? It'll be hard, because a lot of the dark-colored birds you see in the scientific literature aren't amazing black swans; they're crows and grackles. (And closer inspection shows that some of them are engaged in such unusual swan-like behavior because they're floating inertly on their sides). The sheer size of the literature now is another problem - interesting outliers are carried along in a flood tide of stuff that's not quite so interesting. (This paper mentions that very problem, along with a recommendation to still try to browse the literature - rather than only doing targeted searches - because otherwise you'll never see any oddities at all).

Then there's the way that we deal with such things even when we do encounter them. Nugent's recommendation is to think hard about whether you really know as much as you think you do when you try to rationalize away some odd report. (And rationalizing them away is the usual reponse). The conventional wisdom may not be as solid as it appears; you can probably put your foot through it in numerous places with a well-aimed kick. As the paper puts it: "Ultimately, the fact that something has never been done is the flimsiest of evidence that it cannot be done."

That's worth thinking about in terms of medicinal chemistry, as well as organic synthesis. Look, for example, at Rule-Of-Five type criteria. We've had a lot of discussions about these around here (those links are just some of the more recent ones), and I'll freely admit that I've been more in the camp that says "Time and money are fleeting, bias your work towards friendly chemical space". But it's for sure that there are compounds that break all kinds of rules and still work. Maybe more time and money should go into figuring out what it is about those drugs, and whether there are any general lessons we can learn about how to break the rules wisely. It's not that work in this area hasn't been done, but we still have a poor understanding of what's going on.

Comments (16) + TrackBacks (0) | Category: Chemical News | Drug Industry History | The Scientific Literature | Who Discovers and Why

August 15, 2012

More On Reproducing Scientific Results: Organic Chemistry Edition

Email This Entry

Posted by Derek

Yesterday's post on reproducing scientific results got me to thinking about the application of this to organic chemistry. How much of this are we going to see, compared to biology?

Not as much, is my guess. Some of the barriers to reproducibility are too low to bother with, while others are too high. In the "too low" category are many new synthetic method papers. People try these things out, if they look useful at all, and they either work or they don't. Most of the time, you end up finding the limits of the reported method - your substrate failed dismally, but when you look, you realize that you had a basic tertiary amine in your molecule, and none of the examples in the paper have one. Ah-hah.

It's rare that a useful-looking reaction turns out to be completely non-reproducible across multiple structures (although it has happened). Here's a paper from 2000, by one Vincent C. O. Njar, claiming that carbonyl diimidazole reacted with hydroxy groups to give direct N-alkylation of imidazole. Two years later, Walter Fischer from Ciba Specialty Chemicals took this paper apart in detail, showing that it did not work and could not have worked. The products were carbamates instead - not surprising - and the original author should have realized this (as should the referees of the paper).

Then you have total synthesis. And here, the barrier is too high: no one is going to reproduce these things after a certain point. A 48-step synthesis of Shootmenowicene could appear tomorrow, and the odds are overwhelming that no one will ever explore its heights again. There have been total syntheses that have been received with grave doubts (hexacyclinol!), but no one, to the best of my knowledge, has gone back over every step of one of these. The return on the investment of time and money is just too low - which, to be frank, is a sentence that sums up my opinion of a lot of total synthesis work these days.

Where the Reproducibility Initiative could come in handy inside organic chemistry, though, would be for unusual things of wide applicability that are still hard to believe. The famous "NMR chirality" scandal at the University of Bonn in the 1990s would be a good example of this. This was a startling result - that the chirality of organic reactions could be measurably influenced by the handedness of an applied magnetic field - and many people had trouble believing it on physical grounds. They were right, too, because it all turned out to be faked by an individual inside the group, a fact that was only discovered after much effort and embarrassment. Having immediate access to third-party reproducibility testing would have sped things up quite a bit - and perhaps if that access is more widely known, used, and appreciated, we might see fewer bizarre cases like this in general.

Comments (16) + TrackBacks (0) | Category: The Scientific Literature

August 14, 2012

Reproducing Scientific Results - On Purpose

Email This Entry

Posted by Derek

We've spoken several times around here about the problems with reproducing work in the scientific literature. You have to expect some slippage on cutting-edge work, just because it's very complex and is being looked at for the first time. But at the same time, it's that sort of work that we're depending on to advance a field, so when it turns out to be wrong, it causes more damage than something older and more obscure that falls apart.

There's a new effort which is trying to attack the problem directly. Very directly. The Reproducibility Initiative is inviting people to have their work independently confirmed by third-party researchers. You'll be responsible for the costs, but at the end of it, you'll have a certification that your results have been verified. The validation studies themselves will be published in the new PLOS ONE Reproducibility Collection, and several leading publishers have agreed to link the original publications back to this source.

I very much hope that this catches on. The organizers have rounded up an excellent advisory committee, with representatives from academia and industry, both of whom would be well served by more accurate scientific publication. I can especially see this being used when someone is planning to commercialize some new finding - going to the venture capital folks with independent verification will surely count for a lot. Granting agencies should also pay attention, and reward people accordingly.

Here's an article by Carl Zimmer with more on the idea. I'll be keeping a close eye on this myself, and hope to highlight some of the first studies to make it through the process. With any luck, this can become the New Normal for groundbreaking scientific results.

Comments (27) + TrackBacks (0) | Category: General Scientific News | The Scientific Literature

July 19, 2012

Quick Links

Email This Entry

Posted by Derek

I'm pressed for time this morning, so I wanted to put up a quick link to Adam Feuerstein's thoughts on media embargoes of scientific results (and how they're becoming increasingly useless).

And I also wanted to note this odd bit of news: I'll bet you thought that fluorine, elemental gaseous fluorine, wasn't found in nature. Too reactive, right? But we're all wrong: it's found in tiny cavities in an unusually stinky mineral. And part (or all) of that smell is fluorine itself, which I'll bet that very few people have smelled in the lab. I hope not, anyway.

Comments (6) + TrackBacks (0) | Category: Business and Markets | Chemical News | The Scientific Literature

July 3, 2012

The Papers In This Journal Are Just So Darn Relevant

Email This Entry

Posted by Derek

Thomson Reuters is out with their lists of impact factors for journals, and these come with the usual cautions: too much is made of the impact factor in general, and the very fact that the tiniest variations are seized on gleefully by journal publishers should be enough to set off alarms.

This time a record number of journals were taken off the list for excessive self-citation. And as that Nature News article notes, somewhat gleefully, one of the journals had recently been profiled by Thomson Reuters as a "Rising Star". (All that profiling and interviewing has made me wonder in the past, and I'm not surprised at all that this has happened. The company measures the impact factors, promotes them as meaningful, interviews journal editors who have found ways to raise theirs, which makes that important news because the people who sell impact factors say that it's important, and they have the press releases to prove it. I'm standing by my earlier comparison to the Franklin Mint. (And in case you're wondering, the fact that I'm citing my own blog on the topic of self-referentiality has not escaped me).

At any rate, I don't believe that any chemistry journals were on the banned list. The most interesting case was a group of journals that were deliberately citing each other, but I'll freely admit that I'd never heard of any of them, despite their best efforts to rise in the world. If anyone does have any evidence of citation oddities in the chemistry world, though, I'd be happy to help publicize them. . .

Comments (13) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

July 2, 2012

If They Pretend to Read It, He'll Pretend to Write It

Email This Entry

Posted by Derek

Is this the record? At least 172 faked publications from a Japanese anaesthesiology researcher. He doesn't seem to have been a particularly high-impact person in the field, but that makes you wonder, too. Sitting around all day, making up data for papers that no one reads. . .what a life! I don't how anything on quite this scale could happen in chemistry, but perhaps that's wishful thinking.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

June 20, 2012

The Cream Rises to the Top, But So Does the Pond Scum

Email This Entry

Posted by Derek

Here's a rather testy letter to the editors of The Lancet about some recent work published there by Novo Nordisk and collaborators.

Both trials produce the same finding. . .Each focuses its main conclusion not on this primary outcome, but on one of several secondary measurements: nocturnal hypoglycaemia in the first paper and overall hypoglycaemia in the second. In both, the difference was of marginal significance and no mention is made of adjustment for multiple testing. These lower hypoglycaemia rates in unblinded studies should be considered, at best, hypothesis generating. At worst they are spurious. . .

The Lancet's reprints are a major source of revenue for the journal, and a major part of drug company marketing. These trials were written and analysed by NovoNordisk statisticians and NovoNordisk-funded professional writers. We applaud their skill, but regret the lack of editorial effort deployed to balance it. . .

"What are editors for?", asks the letter. This brings up something that we all may have to contend with if the scientific publishing model continues to change and erode. The publishers themselves make much of their status as gatekeepers, citing their coordination of the peer review process and their in-house editing. (The counterarguments are that the peer review is being done by free labor, and not always very effectively, and that the quality of the in-house editing varies from "pretty good" to "surely you jest").

These papers are a case in point. What if they are, as the letter writers contend, largely just vehicles for marketing? That sort of thing certainly does happen. Will it happen even more under some new scientific publishing system? You'd have to think that the marketing folks are wondering the same thing, but from the standpoint of a feature rather than a bug.

Marketing, though, would rather have papers to point at that are published in a prestigious journal, which is one reason that letter is being sent to The Lancet. And no matter what sort of publishing model comes along, I don't think that we're ever going to get rid of prestige as a factor, human nature being what it is. (And beyond that, having a stratum of recognizably prestigious journals does have its uses, although its abuses can outweigh them). It is, in fact, the prestige factor that's keeping the current system afloat, as far as I can see.

The only thing I can think of to replace it that wouldn't be as vulnerable to the same abuses would be one where papers float to the top through reader comments and interest. Upvotes, downvotes, number of comments and replies, number of downloads and page views - these might end up as what people point to when they want to show the impact of their papers, along with the traditional measures based on citations in other papers. But while that might avoid some of the current problems, it would be open to new ones, various ways of gaming the system to boost papers beyond where they naturally would end up (and to send rival work down the charts as well?) There's also the problem that the most-discussed papers aren't a perfect proxy for the most important ones. A harder-to-comprehend paper, made that way either through its presentation or through its intrinsic subject matter, will make less headway. And deliberately buzzy, controversial stuff will rise faster and higher, even if it's not so worthwhile on closer inspection.

It's probably impossible to come up with a system that can't be gamed or abused. I won't miss the current one all that much, but we'll have to be careful not to replace it with something worse.

Comments (18) + TrackBacks (0) | Category: The Scientific Literature

May 24, 2012

Publishing Without Consent

Email This Entry

Posted by Derek

Here's a note on an ugly situation: when a post-doc publishes a paper without the permission of the principal investigator. Now, this is a fairly rare situation, but still not as rare as you might imagine - the article itself has several citations, and it quotes a journal editor who's seen it happen a number of times.

In most of these cases, there seems to be a more fundamental confusion about ownership of data, with publishing as the sequel. People leave a research group with their piles of results, and decide that since it's theirs, that it's time to get it out into the literature with their name on it. But as the article points out, if work is done under NIH funding, then the results belong to the institution, and the grantee/PI is the person who decides when and where things are published. You may, as a grad student or post-doc, feel that the data you worked so hard to generate are rightfully yours, but most of the time that's just not the case.

In industry we have our own disputes, but this isn't one of them. There's rarely any argument about ownership of data: that's all company property, and you sign documents when you're hired that explicitly spell that out. And publication is rarely as bitter a business as it is in academia (where it's the coin of the realm). We argue about whether a particular project is advanced enough (or dead enough, more likely) to be written up for a journal, but these are secondary questions.

Who gets on the patent is a slightly bigger question, but it's not like you get a cut of the profits based on whether your name is on the list. That's as opposed to Germany, where that's exactly what happens, and I've often wondered if we should try that here. That system leads to some elbow-throwing when it comes to inventorship on a hot project, but it also leads to everyone having a clear idea of the legal requirements to be an inventor. Ownership is, naturally, not in dispute at all. Every invention realized at the company is company property, too (those same documents take care of that back when you're hired on).

So while rogue academic publishing is a known phenomenon, rogue industrial patenting isn't. Well, as far as I know it isn't - anyone have an example of someone who tried to get away with it?

Comments (35) + TrackBacks (0) | Category: Patents and IP | The Scientific Literature

May 23, 2012

How Come?

Email This Entry

Posted by Derek

I'm baffled by this abstract. Why would you go to the trouble of putting an unusual group (a ferrocene) on a molecule, and then show that putting it on seems to do little or nothing to the properties and activity of the parent compound? "We put a ferrocene on and it didn't kill the molecule" doesn't seem to be enough grounds for a full J. Med. Chem. paper. Does it?

Comments (21) + TrackBacks (0) | Category: The Scientific Literature

May 17, 2012

The Breslow Chirality Paper Mess, Resolved

Email This Entry

Posted by Derek

Just a note: the Breslow origins-of-chirality paper, also known more widely (thanks to some bizarre PR work at the ACS) as the "alien dinosaur" paper, has been withdrawn, and on the correct grounds. The pun in the headline is intended.

Comments (16) + TrackBacks (0) | Category: The Scientific Literature

May 10, 2012

The UK Goes Open-Access

Email This Entry

Posted by Derek

More disruption in the scientific publishing model: the UK government has announced that it will set up an open-access system for papers that are generated through its funding, similar to the system in the US. The details are still being worked out, and the government is still making noises about not "ruining the value provided by academic publishers", but it's that value that's at issue, isn't it?

A statement from Wiley said that "Publishers enable content digitisation, rigorous peer review, strong editorial infrastructure and support and investment in an effective online platform for dissemination." And yes, they do those things. But how well do they do them? And how well do they do them for the prices they charge? I'm glad that these arguments are finally out on the table.

Comments (6) + TrackBacks (0) | Category: The Scientific Literature

May 7, 2012

You're A Peer, Too, You Know

Email This Entry

Posted by Derek

Over at The Curious Wavefunction, there's a great post looking back at the infamous "negative rate constant" affair (Breslow, Menger, Haim). If you're not familiar with that one, give it a look. I remember this one while it was going on, and in retrospect, you have to imagine what it would have been like if there had been a chemical blog world at the time. It's an extraordinary chapter in chemical (and chemical literature) history.

To that end, there's this opinion piece from yesterday's New York Times. Author Jack Hitt is talking about the tail of comments that now follows any notable article, in any field:

Almost any article worth reading these days generates some version of this long tail of commentary. Depending on whether they are moderated, these comments can range from blistering flameouts to smart factual corrections to full-on challenges to the very heart of an article’s argument. . .

. . .the comments section of any engaging article is almost as necessary a read as the piece itself — if you want to know how insider experts received the article and how those outsiders processed the news (and maybe to enjoy some nasty snark from the trolls).

Should this part of every contemporary article be curated and edited, almost like the piece itself? Should it have a name? Should it be formally linked to the original article or summarized at the top? By now, readers understand that the definitive “copy” of any article is no longer the one on paper but the online copy, precisely because it’s the version that’s been read and mauled and annotated by readers. (If a book isn’t read until it’s written in — as I was always told — then maybe an article is not published until it’s been commented upon.) Writers know this already. The print edition of any article is little more than a trophy version, the equivalent of a diploma or certificate of merit — suitable for framing, not much else.

I think this is exactly what science is about, and exactly what it needs. People should be able to read the latest results, add their opinions and criticisms to them, and those comments in turn should also be available for everyone to see. There's going to be noise in there, but I'll take some noise as the price that gets paid for figuring things out more quickly and more completely than we ever could before. As far as I'm concerned, the "peer" in "peer review" means "Everyone who can read and understand the paper".

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

May 1, 2012

Flip That Glucose, Please

Email This Entry

Posted by Derek

Just another nitpicking note: if you're going to publish a paper on glucose conjugates of drugs (aspirin, in this case), you might want to be sure to draw the glucose as the correct enantiomer. I had to do a little head-scratching with this one, since the sugar ring is drawn from a perspective that no carbohydrate chemist would ever use, but as far as I can tell, that's L-glucose instead of D. . .

This is not as big a deal in the grand scheme of things, but it particularly gets to me, as someone who worked with sugars as chiral starting materials for 4 1/2 years. And then, every chemical drawing program extant will draw you the correct stereochemistry. . .

Comments (20) + TrackBacks (0) | Category: The Scientific Literature

April 27, 2012

Different Worlds: A Last DHFR Paper Thought

Email This Entry

Posted by Derek

Inspired by a discussion with a colleague, I'm going to take one more crack at the recent discussion here about the J. Med. Chem. DHFR paper. Those of you with an interest in the topic, read on. Those whose interest has waned, or who never had much interest to start with, take heart: other topics are coming.

It's clear that many people were disappointed with my take on this paper, and my handling of the whole issue. Let me state again that I mishandled the biology aspects of this one thoroughly, through carelessness, and I definitely owe this apology to the authors of the paper (and the readers of this site) for that.

Of course, that's not the only arguable thing about the way I handled this one. As I spent paragraphs rambling on about in yesterday's post, there's a chemical aspect to the whole issue as well, and that's what caught my eye to start with. I think one of the things that got me into trouble with this one is two different ways of looking at the world. I'll explain what I mean, and you can judge for yourself if I'm making any sense.

The authors of the paper (and its reviewer who commented here) are interested in D67 dihydrofolate reductase, from a biological/enzymological perspective. From this viewpoint - and it's a perfectly tenable one - the important thing is that D67 DHFR is an unusual and important enzyme, a problem in bacterial resistance, interesting in its own right as a protein with an odd binding site, and for all that, still has no known selective inhibitors. Anything that advances the understanding of the enzyme and points toward a useful inhibitor of it is therefore a good thing, and worth publishing in J. Med. Chem., too.

I come in from a different angle. As someone who's done fragment-based drug discovery and takes a professional interest in it, I'll take a look at any new paper using the technique. In this case, I gave the target much too cursory a look, and filed it as "DHFR, bacterial enzyme, soluble, X-ray structures known". In other words, a perfectly reasonable candidate for FBDD as we know it. Once I'd decided that this was a mainstream application of something I already have experience with, I turned my attention to how the fragment work was done. By doing so, I missed out on the significance of the DHFR enzyme, which means, to people in the first camp, that I whiffed on the most important part of the entire thing. I can understand their frustration as I brushed that off like a small detail and went on to what (to them) were secondary matters.

But here's where my view of the world comes in. As a drug discovery guy, when I read a paper in J. Med. Chem., I'd like to see progress in, well, the medicinal chemistry of the topic. That was the thrust of my blog post yesterday: that I found the med-chem parts of the paper uncompelling, and that the application of fragment-based techniques seemed to me to have gone completely off track. (I havne't mentioned the modeling and X-ray aspects of the paper, as Teddy Z did at Practical Fragments, but I also found those parts adding nothing to the worth of the manuscript as a whoel). The most potent compounds in the paper seem, to me, to be the sort that are very unlikely to lead to anything, and are unlikely to show selectivity in a cellular environment. If the paper's starting fragment hits are real (which is not something that's necessarily been proven, as I mentioned in yesterday's post), then it seems to me that everything interesting and useful about them is being thrown away as the paper goes on. From the other point of view, things are basically the opposite - the paper gets better and better as the compounds get more potent.

But here's where, perhaps, the two viewpoints I spoke of earlier might find something in common. If you believe that the important thing is that selective inhibitors of D67 DHFR have finally been discovered, then you should want these to be as potent and selective as possible, and as useful as possible in a variety of assays. This, I think, is what's in danger of being missed. I think that a fragment-based effort should have been able to deliver much more potent chemical matter than these compounds, with less problematic structures, which are more likely to be useful as tools.

I'll finish up by illustrating the different angles as starkly as I can. The authors of this paper have, in one view of the world, completed the first-ever fragment screen against an important enzyme, discovered the first-ever selective inhibitors of it, and have published these results in a prestigious journal: a success by any standard. From my end, if I were to lead a drug discovery team against the same enzyme, I might well see the same fragment hits the authors did, since I know that some of these are in the collections I use. But if I proceeded in the same fashion they did, prosecuting these hit compounds in the same way, I would, to be completely honest about it, face some very harsh questioning. And if I persevered in the same fashion, came up with the same final compounds, and presented them as the results of my team's work, I would run the serious risk of being fired. Different worlds.

Update: Prof. Pelletier sends the following:

I certainly have been following this with interest, and learning much from it – not just science.

Throughout the week, I have appreciated your civil tone – many thanks. I willingly accept your apology, just as I accept the constructive criticism that will improve our future work. I think your ‘two-worlds’ point of view smacks of truth. The bottom line from my point of view is that I’m open to collaboration with a real fragment library: if anyone is interested in making this better, they should contact me. I’d be delighted to work with more than what can be scavenged from neighbouring labs in an academic setting.

Your bloggers’ response to this come-and-go was fascinating: the process was admired to an extent that surprised me. A number of responders point out that there are currently few occurrences of open exchange on these blogs and – sorry to disappoint hard-core bloggers – it does not endear me to the blogging process. I don’t blog because I can’t stand anonymous, frequently disrespectful and sometimes poorly researched comments. I nonetheless hope that this will open the door to a more transparent blogging process in the long run.

For any who care, I am brave, not at all desperate, and definitely a woman. ; )

If you feel any of this would be of interest for your blog, please feel free to post. Thanks for seeing this through rather than shaking it off.

Comments (21) + TrackBacks (0) | Category: Academia (vs. Industry) | The Scientific Literature

April 26, 2012

Elsevier Picks Up the Pace

Email This Entry

Posted by Derek

So perhaps I should rethink all those nasty things I've been saying about Elsevier journals. Here's someone who submitted a paper to Nuclear Instruments and Methods on a Friday evening, and got it accepted - with two referee reports, yet - on Monday morning. How is that possible, you say? That's what this author is wondering, too. . .

Comments (8) + TrackBacks (0) | Category: The Scientific Literature

April 25, 2012

DHFR Inhibitors Revisited: A Word From the Authors (and Reviewers)

Email This Entry

Posted by Derek

The other day, I had some uncomplimentary things to say about a recent J. Med. Chem. paper on fragment-based dihydrofolate reductase inhibitors. Well, I know that I don't say these things into a vacuum, by any means, but in this case the lead author has written me about the work, and a reviewer of the paper has showed up in the comments. So perhaps this is a topic worth revisiting?

First, I'll give Prof. Joelle Pelletier of U. Montreal the floor to make the case for the defense. Links added are mine, for background; I take responsibility for those, and I hope they're helpful.

I was informed of your recent blog entitled ‘How do these things get published’. I am corresponding author of that paper. I would like to bring to your attention a crucial point that was incorrectly presented in your analysis: the target enzyme is not that which you think it is, i.e.: it is not a DHFR that is part of ‘a class of enzymes that's been worked on for decades’.

Indeed, it would make no sense to report weak and heavy inhibitors against ‘regular’ DHFRs (known as ‘type I DHFRs’), considering the number of efficient DHFR inhibitors we already know. But this target has no sequence or structural homology with type I DHFRs. It is a completely different protein that offers an alternate path to production of tetrahydrofolate (see top of second page of the article). It has apparently evolved recently, as a bacterial response to trimethoprim being introduced into the environment since the ‘60’s. Because that protein is evolutionarily unrelated to regular DHFRs, it doesn’t bind trimethoprim and is thus intrinsically trimethoprim resistant; it isn’t inhibited by other inhibitors of regular DHFRs either. There have been no efforts to date to inhibit this drug resistance enzyme, despite its increasing prevalence in clinical and veterinary settings, and in food and wastewater (see first page of article). As a result, we know nothing about how to prevent it from providing drug resistance. Our paper is thus the first foray into inhibiting this new target – one which presents both the beauty and the difficulty of complex symmetry.

Regular (type I) DHFRs are monomeric enzymes with an extended active-site cleft. They are chromosomally-encoded in all living cells where they are essential for cellular proliferation. Our target, type II R67 DHFR, is carried on a plasmid, allowing rapid dissemination between bacterial species. It is an unusual homotetrameric, doughnut-style enzyme with the particularity of having a single active site in the doughnut hole. That’s unusual because multimeric enzymes typically have the same number of active sites as they do monomers. The result is that the active site tunnel, shown in Figure 4 a, has 222 symmetry. Thus, the front and back entrances to the active site tunnel are identical. And that’s why designing long symmetrical molecules makes sense: they have the potential of threading through the tunnel, where the symmetry of the inhibitor would match the symmetry of the target. If they don’t string through but fold up into a ‘U”, it still makes sense: the top and bottom of the tunnel are also alike, again allowing a match-up of symmetry. Please note that this symmetry does create a bit of a crystallographer’s nightmare at the center of the tunnel where the axes of symmetry meet; again, it is an unusual system.

You have referred to our ‘small, poorly documented library of fragment compounds’. As for the poor documentation, the point is that we have very little prior information on the ligands of this new target, other than its substrates. We cast as wide a net as we could within a loosely defined chemical class, using the chemicals we have access to. Unfortunately, I don’t have access to a full fragment library, but am open to collaboration.

As a result of extending the fragments, the ligand efficiency does take a beating… so would it have been better not to mention it? No, that would have been dishonest. In addition, it is not a crucial point at this very early stage in discovery: this is a new target, and it IS important to obtain information on tighter binding, even if it comes at the cost of heavier molecules. In no way do we pretend that these molecules are ripe for application; we have presented the first set of crude inhibitors to ‘provide inspiration for the design of the next generation of inhibitors’ (last sentence of the paper).

Your blog is widely read and highly respected. In this case, it appears that your analysis was inaccurate due to a case of mistaken identity. I did appreciate your calm and rational tone, and hope that you will agree that there is redeeming value to the poor ligand efficiency, because of the inherent novelty of this discovery effort. I am appealing to you to reconsider the blog’s content in light of the above information, and respectfully request that you consider revising it.

Well, as for DHFRs, I'm guilty as charged. The bacterial ones really are way off the mammalian ones - it appears that dihydro/tetrahydrofolate metabolism is a problem that's been solved a number of different ways and (as is often the case) the bacteria show all kinds of diversity compared to the rest of the living world. And there really aren't any good D67 DHFR inhibitors out there, not selective ones, anyway, so a molecule of that type would definitely be a very worthwhile tool (as well as a potential antibiotic lead).

But that brings us to the fragments, the chemical matter in the paper. I'm going to stand my my characterization of the fragment library. 100 members is indeed small, and claiming lack of access to a "full fragment collection" doesn't quite cover it. Because of the amount of chemical space that can be covered at these molecular weights, a 200-member library can be significantly more useful than a 100-member one, and so on. (Almost anything is more useful than a 100-member library). There aren't more compounds of fragment size on the shelves at the University of Montreal?

More of a case could be made for libraries this small if they covered chemical space well. Unfortunately, looking over the list of compounds tested (which is indeed in the Supplementary Material), it's not, at first glance, a very good collection. Not at all. There are some serious problems, and in a collection this small, mistakes are magnified. I have to point out, to start with, that compounds #59 and #81 are duplicates, as are compounds #3 and #40, and compounds #7 and #14. (There may be others; I haven't made a complete check).

The collection is heavily biased towards carboxylic acids (which is a problem for several reasons, see below). Nearly half the compounds have a COOH group by my quick count, and it's not a good idea to have any binding motif so heavily represented. I realize that you intentionally biased your screening set, but then, an almost featureless hydrophobic compound like #46 has no business in there. Another problem is that some of the compounds are so small that they're unlikely to be tractable fragment hits - I note succinimide (#102) and propyleneurea (#28) as examples, but there are others. At the other end of the scale, compounds such as the Fmoc derivative #25 are too large (MW 373), and that's not the only offender in the group (nor the only Fmoc derivative). The body of the manuscript mentions the molecular weights of the collections as being from 150 to 250, but there are too many outliers. This isn't a large enough collection for this kind of noise to be in it.

There are a number of reactive compounds in the list, too, and while covalent inhibitors are a very interesting field, this was not mentioned as a focus of your efforts or as a component of the screening set. And even among these, compounds such as carbonyldiimidazole (#26), the isocyanate #82, and disuccinimidylcarbonate (#36) are really pushing it, as far as reactivity and hydrolytic stability. The imine #110 is also very small and likely to have hydrolytic stability problems. Finally, the fragment #101 is HEPES, which is rather odd, since HEPES is the buffer for the enzyme assays. Again, there isn't room for these kinds of mistakes. It's hard for me to imagine that anyone who's ever done fragment screening reviewed this manuscript.

The approach to following up these compounds also still appears inadequate to me. As Dan Erlanson pointed out in a comment to the Practical Fragments post, small carboxylic acids like the ones highlighted are not always legitimate hits. They can, as he says, form aggregates, depending on the assay conditions, and the most straightforward way of testing that is often the addition of a small amount of detergent, if the assay can stand it. The behavior of such compounds is also very pH-dependent, as I've had a chance to see myself on a fragment effort, so you need to make sure that you're as close to physiological conditions as you can get. I actually have seen some of your compounds show up as hits in fragment screening efforts, and they've been sometimes real, sometimes not.

But even if we stipulate that these compounds are actually hits, they need more work than they've been given. The best practice, in most cases when a fragment hit is discovered and confirmed, is to take as many closely related single-atom changes into the assay as possible. Scan a methyl group around the structure, scan a fluoro, make the N-for-C switches - at these molecular weights, these changes can make a big difference, and you may well find an even more ligand-efficient structure to work from.

Now, as for the SAR development that actually was done: I understand the point about the symmetry of the enzyme, and I can see why this led to the idea of making symmetrical dimer-type compounds. But, as you know, this isn't always a good idea. Doing so via flexible alkyl or alkyl ether chains is not a good idea, though, since such compounds will surely pay an entropic penalty in binding.

And here's one of the main things that struck both me and Teddy Z in his post: if the larger compounds were truly taking advantage of the symmetry, their ligand efficiency shouldn't go down. But in this case it does, and steeply. The size of the symmetical inhibitors (and their hydrophobic regions, such as the featureless linking chains, make it unsurprising that this effort found some micromolar activity. Lots of things will no doubt show micromolar activity in such chemical space. The paper notes that it's surprising that the fragment 4c showed no activity when its structural motif was used to build some of the more potent large compounds, but the most likely hypothesis is that this is because the binding modes have nothing to do with each other.

To be fair, compounds 8 and 9 are referred to as "poorly optimized", which is certainly true. But the paper goes on to say that they are starting points to develop potent and selective inhibitors, which they're not. The fragments are starting points, if they're really binding. The large compounds are dead ends. That's why Teddy Z and I have reacted as strongly as we have, because the path this paper takes is (to our eyes) an example of how not to do fragment-based drug discovery.

But still, I have to say that I'm very glad to hear a direct reply to my criticism of this paper. I hope that this exchange has been useful, and that it might be of use for others who read it.

Comments (24) + TrackBacks (0) | Category: Drug Assays | Infectious Diseases | The Scientific Literature

Breslow's Chirality Paper: More Than Just Alien Dinosaurs

Email This Entry

Posted by Derek

Update: the paper has, for now, been pulled by JACS. More to come, no doubt.

I didn't written anything about the Breslow origins-of-chirality paper that mentioned, as a joking aside, the possibility of intelligent alien dinosaurs. As most readers will know, the ACS publicity office, most cluelessly, decided to make that throwaway line the focus of their press release, and much confusion ensued.

But things have gotten weirder. Stu Cantrill read the Breslow paper, and realized that he'd already read a lot of it before. See these three pictures (1, 2, 3) and realize the extent to which this latest paper was apparently a cut-and-paste job.

I've met Breslow many times (he used to consult for one of my former employers), and I've enjoyed reading much of his work. But this really shouldn't be acceptable - we wouldn't put up with it from some unknown chemist, and we shouldn't put up with it from someone famous. Chembark has an excellent summary of the situation, with recommendations about what the ACS should do next. These range from fixing that idiotic press release, to retracting the paper, to barring Breslow from publishing in JACS for a period.

In retrospect, the alien dinosaurs are becoming my favorite part of the whole paper.

Update: Breslow defends himself to Nature News.

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

April 24, 2012

That's Some Fine Editorial Work There

Email This Entry

Posted by Derek

Since I've mentioned the scientific publication business today, I thought I'd include this story from Retraction Watch as an example of what you're paying for when you pay Elsevier for quality control. When's the last time you saw a paper withdrawn because it "contains no scientific content"? Or noticed that the lead author's email contract address was at the domain ""?

Comments (12) + TrackBacks (0) | Category: The Scientific Literature

Harvard's Had Enough

Email This Entry

Posted by Derek

Several readers sent along this memo from Harvard's library: they see the current price structure of scientific journals as unsustainable, and they're asking the faculty to help them do something about it.

The Faculty Advisory Council to the Library, representing university faculty in all schools and in consultation with the Harvard Library leadership, reached this conclusion: major periodical subscriptions, especially to electronic journals published by historically key providers, cannot be sustained: continuing these subscriptions on their current footing is financially untenable. Doing so would seriously erode collection efforts in many other areas, already compromised.

They're asking faculty members to try to "move prestige" to open-access journals by favoring them for new publications, and in general to do whatever they can to get away from the current scientific publishing model. And that ties in with this post over at the Guardian, saying that not only are the current publishers causing a financial burden, but other burdens that may be even more of a problem:

Research, especially scientific research, thrives in an atmosphere that allows the free exchange of ideas and information: open discussion and debate are essential if the scientific method is to operate properly. Before the arrival of the internet, academic publishers provided a valuable service that was a real benefit to the scientific community. Not any more. . .

. . .But open access isn't just about the end products of research. It's the entire process of scientific enquiry, including the collection and processing of data, scrutiny of the methods used in the analysis, questioning of assumptions, and discussion of alternative interpretations. In particular, it's about access to scientific data.

I believe all data resulting from publicly funded research should be in the public domain, for two reasons. First, it's public money that funds us, so we scientists have a moral responsibility to be as open as possible with the public. Second, the scientific method only works when analyses can be fully scrutinised and, if necessary, replicated by other researchers. In other words, to seek to prevent your data becoming freely available is plain unscientific.

We'll see how far this gets. But this is already the biggest upheaval that I can remember in the scientific literature, and it show no signs of slowing down. . .

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

April 18, 2012

How Do These Things Get Published?

Email This Entry

Posted by Derek

Update: I've heard from both the lead author of this paper and one of its reviewers, and I've written a follow-up post on this subject, as well as revising this one where shown below.

I've been saved the trouble of demolishing this J. Med. Chem. paper - the Practical Fragments blog has done it for me. I really hate to say such things, but this appears to be one of the worst papers that journal has published in quite a while.

The authors start out with a small, poorly documented (update: the compounds are, in fact, in the paper's supplementary information, but see the follow-up post) library of fragment compounds. They screen these against dihydrofolate reductase, and get a few possible hits - mind you, there's not much correlation between the numbers and any potency against the enzyme, but these aren't potent compounds, and fragment-level hits don't always perform in high-concentration enzyme assays. But what happens next? The authors string these things together into huge dimeric molecules, apparently because they think that this is a good idea, but they get no data to support this hypothesis at all.

Well, their potency goes from low millimolar to low micromolar, but as Teddy Z at Practical Fragments points out, this actually means taking a terrible beating in ligand efficiency. All that extra molecular weight should buy you a lot more potency than this. There's some hand-waving docking of these structures - which the authors themselves refer to as "poorly optimized" - and some inconclusive attempts at X-ray crystallography, leading to uninterpretable data.

And that's it. That's the paper. This on a class of enzymes that's been worked on for decades, yet. (Update: this characterization is completely wrong on my part - see the follow-up post linked to above for more). Again, I hate to be unkind about this, but I cannot imagine what this is doing in J. Med. Chem., or how it made it through the editorial process. When you submit a scientific manuscript for publication, you open yourself to comments from all comers, and those are mine.

Comments (29) + TrackBacks (0) | Category: Infectious Diseases | The Scientific Literature

April 13, 2012

More on the Federation of Independent Scientists: Journal Access

Email This Entry

Posted by Derek

Update: just to make things clear, this is only one aspect of the whole problem, but perhaps an easier one to tackle. More on the rest of the proposals to come!

Yesterday's post brought in a lot of welcome comment, and I want to follow up on the ideas in it. The first problem I wanted to tackle was journal access for entrepreneurs, the recently unemployed, and small shops. Here are some of the comments from the first post, consolidated:

A better bet would be to negotiate a group agreement with DeepDyve. "You can rent the article and view it at DeepDyve for 24 hours or more" for $1. The catch? No printing- you have to read it online (hence the renting not purchasing model).

Something like that might be necessary, because others pointed out that:

I like the idea of the journal access, HOWEVER, you'll pay handsomely for such access. It's no different than a large corp or large library. You have 300 people in your organization? You need to buy a group license.

And also:

OK, how you going to limit access? Is it just single, non-employed people who can get access? Perhaps you'd like to expand it to small companies? What size cut off? 10? 100? 500? Perhaps the mid-size biotechs should join up, and drop their own subscriptions. And maybe the smaller colleges?

And from "Mrs. McGreevy" herself, who kicked off the discussion:

Alternatively, if enough people think this is something the ACS should be doing, and start demanding it loudly and frequently, perhaps the ACS will get around to doing it. Maybe they don't know what to do, either. Cheaper access to ACS publications? That would be nice. Perhaps even negotiating group payment rates or even (*gasp*) subsidies for access to other publishers' papers? I wonder if that's even possible, but I'll bet the ACS hasn't even considered it up until now. Perhaps there wouldn't need to be an independent library if the ACS were willing to take on the job. We should ask them. (In fact, why stop at the ACS? Maybe the unemployed biologists would be willing to pony up some time, money and lobbying power as well.)

Another idea would be to subsidize journal access with website ads as well as membership dues. Perhaps start a paid online directory of consultants or CROs, sort of the Ye Olde Yellowe Pages model. Perhaps there could be a small fee for posting an RFP or a project up for bid, sort of a classified ad business model to help clients, CROs and consultants find each other. Various small fees for various small services ==> money for subsidized journal access.

OK, those are the journal thoughts so far (other than a number of people who agree that it's a major problem, as do I!) Any more ideas on this aspect to add to the pile? I'd never heard of DeepDyve myself, and they sound interesting: anyone have any experience with them, and is there anyone else in that market niche? We'll put together some action points after this round of ideas. . .

Comments (25) + TrackBacks (0) | Category: Business and Markets | The Scientific Literature

April 12, 2012

A Federation of Independent Researchers?

Email This Entry

Posted by Derek

I've had an interesting e-mail from a reader who wants to be signed as "Mrs. McGreevy", and it's comprehensive enough that I'm going to reproduce it in full below.

As everyone but the editorial board of C&E News has noticed, jobs in chemistry are few and far between right now. I found your post on virtual biotechs inspiring, but it doesn't look like anyone has found a good solution for how to support these small firefly businesses until they find their wings, so to speak. Lots of editorials, lots of meetings, lots of rueful headshaking, no real road map forward for unemployed scientists.

I haven't seen this proposed anywhere else, so I'm asking you and your readership if this idea would fly:

What about a voluntary association of independent research scientists?

I'm thinking about charging a small membership fee (for non-profit administration and hard costs) and using group buying power for the practical real-world support a virtual biotech would need:

1. Group rates on health and life insurance.

How many would-be entrepreneurs are stuck in a job they hate because of the the health care plan, or even worse, are unemployed or underemployed and uninsurable, quietly draining their savings accounts and praying no one gets really sick? I have no idea how this would work across state lines, or if it is even possible,but would it hurt to find out? Is anyone else looking?

2. Group rates on access to journals and library services.

This is something I do know a bit about. My M.S. is in library science, and I worked in the Chemistry Library in a large research institution for years during grad school. What if there were one centralized virtual library to which unaffiliated researchers across the country could log in for ejournal access? What if one place could buy and house the print media that start-ups would need to access every so often, and provide a librarian to look things up-- it's not like everyone needs their own print copy of the Canada & US Drug Development Industry & Outsourcing Guide 2012 at $150 a pop. (But if 350 people paid $1 a year for a $350/yr online subscription . . . )

Yes, some of you could go to university libraries and look these things up and print off articles to read at home, but some of you can't. You're probably violating some sort of terms of service agreement the library and publisher worked out anyway. It's not like anyone is likely to bust you unless you print out stacks and stacks of papers, but still. It's one more hassle for a small company to deal with, and everyone will have to re-invent the wheel and waste time and energy negotiating access on their own.

3. How about an online community for support and networking-- places for blogs, reviews, questions, answers, exchanges of best practices, or even just encouragement for that gut-wrenching feeling of going out on your own as a new entrepreneur?

4. What sort of support for grantwriting is out there? Is there a hole that needs to be filled?

5. How about a place to advertise your consulting services or CRO, or even bid for a contract? Virtual RFP posting?

6. Would group buying power help negotiate rates with CROs? How about rates for HTS libraries, for those of you who haven't given up on it completely?

Is there a need for this sort of thing? Would anyone use it if it were available? How much would an unaffiliated researcher be willing to pay for the services? Does anyone out there have an idea of what sort of costs are involved, and what sort of critical mass it would take to achieve the group buying power needed to make this possible?

I'd be happy to spark a discussion on what a virtual biotech company needs besides a spare bedroom and a broadband connection, even if the consensus opinion is that the OP an ill-informed twit with an idea that will never fly. What do you need to get a virtual biotech started? How do we make it happen? There are thousands of unemployed lab scientists, and I refuse to believe that the only guy making a living these days from a small independently-funded lab is Bryan Cranston.

A very worthy topic indeed, and one whose time looks to have come. Thoughts on how to make such a thing happen?

Comments (59) + TrackBacks (0) | Category: Business and Markets | Drug Development | General Scientific News | The Scientific Literature

April 11, 2012

A New Journal (With Bonus Elsevier-Bashing)

Email This Entry

Posted by Derek

You know, I think that we really are seeing the breakup of the current model of scientific publishing. Spät kommt er, doch er kommt: it's coming on slowly, and in stages, but it's looking more and more inevitable. Take this news from the Wellcome Trust, one of the largest funding agencies in the world for medical research:

Sir Mark Walport, the director of Wellcome Trust, said that his organisation is in the final stages of launching a high calibre scientific journal called eLife that would compete directly with top-tier publications such as Nature and Science, seen by scientists as the premier locations for publishing. Unlike traditional journals, however, which cost British universities hundreds of millions of pounds a year to access, articles in eLife will be free to view on the web as soon as they are published. . .

Walport, who is a fellow of the Royal Society, Britain's premier scientific academy, said the results of public and charity-funded scientific research should be freely available to anyone who wants to read it, for whatever purpose they need it. His comments echo growing concerns from scientists who baulk at the rising costs of academic journals, particularly in a time of shrinking university budgets.

That journal is being launched with the Max Planck Gesellschaft and the Howard Hughes Medical Institute, so it's definitely something worth taking seriously. And outside of that new journal, they're going as far as considering sanctions in funding for any researchers who don't adhere to their open-access policies (basically, free within six months of publication in a journal). I was looking up some papers from back in the 1980s the other day, and was reminded, by contrast, of the policies of some of the commercial publishers: never free, ever, no matter how old it is, or who funded it. Gold, those journal archives are: the costs are long gone; it's all been digitized for years and sits on the servers. But anyone who wants to look at a thirty-year-old paper in Tet Lett had better get ready to pony up. Patents expire, as they should, but copyright? Hah!

Elsevier says in the article that they're committed to offering their customers "choice", and that gosh, the subscription model is just so darn popular that they don't see how they can go against the wishes of their customers by getting rid of it. I particularly enjoyed this quote:

A spokesperson for Elsevier said the company was open to any "mechanism or business model, as long as they are sustainable and maintain or improve existing levels of quality control".

This is the Elsevier that can't manage to fix their own RSS feeds for months, that set up a whole fake-but-real journal division for the advertising revenue, that solicits good textbook reviews on Amazon in exchange for $25 gift cards, that charged people $4500 a year to read a journal stuffed with the editor's own nonsensical papers, and whose chemistry titles repeatedly let through howlers that even undergraduates could have spotted?

That level of quality control is going to be quite a strain to keep up. And yes, I know that the other publishers hardly have clean records, but they managed not to be quoted about their quality control. This time, anyway.

Comments (28) + TrackBacks (0) | Category: The Scientific Literature

March 29, 2012

Sloppy Science

Email This Entry

Posted by Derek

Nature has a comment on the quality of recent publications in clinical oncology. And it's not a kind one:

Glenn Begley and Lee Ellis analyse the low number of cancer-research studies that have been converted into clinical success, and conclude that a major factor is the overall poor quality of published preclinical data. A warning sign, they say, should be the “shocking” number of research papers in the field for which the main findings could not be reproduced. To be clear, this is not fraud — and there can be legitimate technical reasons why basic research findings do not stand up in clinical work. But the overall impression the article leaves is of insufficient thoroughness in the way that too many researchers present their data.

The finding resonates with a growing sense of unease among specialist editors on this journal, and not just in the field of oncology. Across the life sciences, handling corrections that have arisen from avoidable errors in manuscripts has become an uncomfortable part of the publishing process.

I think that this problem has been with us for quite a while, and that there are a few factors making it more noticeable: more journals to publish in, for one thing, and increased publication pressure, for another. And the online availability of papers makes it easier to compare publications and to call them up quickly; things don't sit on the shelf in quite the way that they used to. But there's no doubt that a lot of putatively interesting results in the literature are not real. To go along with that link, the Nature article itself referred to in that commentary has some more data:

Over the past decade, before pursuing a particular line of research, scientists. . .in the haematology and oncology department at the biotechnology firm Amgen in Thousand Oaks, California, tried to confirm published findings related to that work. Fifty-three papers were deemed 'landmark' studies. . . It was acknowledged from the outset that some of the data might not hold up, because papers were deliberately selected that described something completely new, such as fresh approaches to targeting cancers or alternative clinical uses for existing therapeutics. Nevertheless, scientific findings were confirmed in only 6 (11%) cases. Even knowing the limitations of preclinical research, this was a shocking result.

Of course, the validation attempts may have failed because of technical differences or difficulties, despite efforts to ensure that this was not the case. Additional models were also used in the validation, because to drive a drug-development programme it is essential that findings are sufficiently robust and applicable beyond the one narrow experimental model that may have been enough for publication. To address these concerns, when findings could not be reproduced, an attempt was made to contact the original authors, discuss the discrepant findings, exchange reagents and repeat experiments under the authors' direction, occasionally even in the laboratory of the original investigator. These investigators were all competent, well-meaning scientists who truly wanted to make advances in cancer research.

So what leads to these things not working out? Often, it's trying to run with a hypothesis, and taking things faster than they can be taken:

In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set. For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups. Investigators frequently presented the results of one experiment, such as a single Western-blot analysis. They sometimes said they presented specific experiments that supported their underlying hypothesis, but that were not reflective of the entire data set. . .

This can rise, on occasion, to the level of fraud, but it's not fraud if you're fooling yourself, too. Science is done by humans, and it's always going to have a fair amount of slop in it. The same issue of Nature, as fate would have it has a good example of irreproducibility this week. Sanofi's PARP inhibitor iniparib already wiped out in Phase III clinical trials not long ago, after having looked good in Phase II. It now looks as if the compound was (earlier reports notwithstanding) never much of a PARP1 inhibitor at all. (Since one of these papers is from Abbott, you can see that doubts had already arisen elsewhere in the industry).

That's not the whole story with PARP - AstraZeneca had a real inhibitor, olaparib, fail on them recently, so there may well be a problem with the whole idea. But iniparib's mechanism-of-action problems certainly didn't help to clear anything up.

Begley and Ellis call for tightening up preclinical oncology research. There are plenty of cell experiments that will not support the claims made for them, for one thing, and we should stop pretending that they do. They also would like to see blinded protocols followed, even preclinically, to try to eliminate wishful thinking. That's a tall order, but it doesn't mean that we shouldn't try.

Update: here's more on the story. Try this quote:

Part way through his project to reproduce promising studies, Begley met for breakfast at a cancer conference with the lead scientist of one of the problematic studies.

"We went through the paper line by line, figure by figure," said Begley. "I explained that we re-did their experiment 50 times and never got their result. He said they'd done it six times and got this result once, but put it in the paper because it made the best story. It's very disillusioning."

Comments (38) + TrackBacks (0) | Category: Cancer | Drug Assays | The Scientific Literature

March 16, 2012

Elsevier's RSS Feeds: Maybe a Not-So-Minor Complaint

Email This Entry

Posted by Derek

As has been noted here in the comments sections, the RSS feeds of Elsevier's journals have been hosed, in various ways, for some time now. Things don't come through, or they don't come through correctly, or they're duplicated, or you get abstracts from journals that you never heard of. How many people - those of you who read journals via RSS - are experiencing these problems? And has anyone gotten Elsevier to respond to any complaints?

Comments (27) + TrackBacks (0) | Category: The Scientific Literature

March 5, 2012

Department of Lots of Nerve (Patent Applications Desk)

Email This Entry

Posted by Derek

When you file a patent application, there are plenty of things that the PTO wants you to include. One of the big ones is prior art: you're supposed to disclose all the relevant inventions close to yours that you're aware of, in order to show how your discovery is different. Prior art is naturally to be found in other previous patent filings, and it's also to be found in journal articles and other such public disclosures. If you don't submit relevant prior art that is known to you, your patent application gets into a lot of trouble eventually (and the more worthwhile your invention, the greater the chance becomes of that catching up with you).

So in light of this, you might find it interesting that some of the large scientific publishers are suing over all this. Why? Well, these lawsuits (filed by Wiley and by the American Institute of Physics) allege that the accused law firms violated copyright by submitting unauthorized copies of journal articles with their patent applications.

As that post at PatentlyO goes on to show, the plaintiffs seem to also be very interested in the internal copies of articles that the law firms are making. But I don't really see how they're going to make either of these stick. I mean, I tend to think that a lot of things are "fair use", but aren't these? This really looks like an act of desperation - the traditional scientific publishing model must be in even worse shape than I thought.

Comments (15) + TrackBacks (0) | Category: Patents and IP | The Scientific Literature

February 29, 2012

Bias in Industry-Funded Trials in Rheumatoid Arthritis?

Email This Entry

Posted by Derek

The title of this one says it all: "Association of industry funding with the outcome and quality of randomized controlled trials of drug therapy for rheumatoid arthritis". Any number of critics of the drug business will tell you what that association is: we publish the good stuff and bury the bad news, right?

Well, not so much in arthritis, apparently. The authors identified 103 recent clinical trials in the area, over half of them industry-funded. But when it came to outcomes, things were pretty much the same. Trials from the three largest classes of funding (industry, nonprofit, and "unspecified") all tended to strongly favor the tested drug, although the small number (six) of mixed-funding trials ended up with two favoring and four against. The industry-run trials tended to have more subjects, while the nonprofit ones tended to run longer. The industrial trials also tended to have a more complete description of their intent-to-treat and workflow. As you'd figure, the industrial trials tended to be on newer agents, while the others tended to investigate different combinations or treatment regimens with older ones. But the take-home is this:

No association between funding source and the study outcome was found after adjustment for the type of study drug used, number of study center, study phase, number of study subject, or journal impact factor. . .

. . .Though preponderance of data in medical literature shows that industry funding leads to higher chances of pro-industry results and conclusions, we did not find any association between the funding source and the study outcome of "published" (randomized clinical trials) of RA drug therapies.

The one worrying thing they did find was a trend towards publication bias - the industry-sponsored studies showed up less often in the literature. The authors speculate as to whether these were trials with less favorable outcomes, but didn't have enough data to say one way or another. . .

Comments (5) + TrackBacks (0) | Category: Clinical Trials | The Dark Side | The Scientific Literature

February 28, 2012

Yes, The Research Works Act Is Dead

Email This Entry

Posted by Derek

The sponsors of the Research Works Act (Representatives Carolyn Maloney, D-N.Y., and Darrell Issa, R-California) have announced that they will not be bringing it forward. Elsevier's backtrack was indeed the sign.

Comments (2) + TrackBacks (0) | Category: The Scientific Literature

February 27, 2012

Is the Research Works Act Dead?

Email This Entry

Posted by Derek

It may well be. This morning comes news that Elsevier has dropped support for the RWA, which makes one think that they're feeling the pressure:

We have heard expressions of support from publishers and scholarly societies for the principle behind the legislation. However, we have also heard from some Elsevier journal authors, editors and reviewers who were concerned that the Act seemed inconsistent with Elsevier’s long-standing support for expanding options for free and low-cost public access to scholarly literature. That was certainly not our intention in supporting it. . .

While we continue to oppose government mandates in this area, Elsevier is withdrawing support for the Research Work Act itself. We hope this will address some of the concerns expressed and help create a less heated and more productive climate for our ongoing discussions with research funders. . .

You can smell the smoke from the brake pads, and hear the reverse gears engaging. Maybe now the American Chemical Society will make a public statement - I haven't heard anything from them yet, although their default position (as a member of the AAP) is to support it. (They supported a previous version of the bill in 2008).

Comments (5) + TrackBacks (0) | Category: The Scientific Literature

February 16, 2012

Imidazole-1-Sulfonyl Azide HCl: Look Out

Email This Entry

Posted by Derek

I don't know how many of you out there like to form azides, but if you do, you've probably used (or thought about using) imidazole-1-sulfonyl azide hydrochloride. This reagent appeared in Organic Letters a few years ago as a safe-to-handle shelf-stable azide transfer reagent, and seems to have found popularity. (I've used it myself).

So it was with some alarm that I noted this new paper on the stability and handling characteristics of the reagent. It's a collaboration between the University of Western Australia (where the reagent was developed, partly by the guy whose lab bench I took over in grad school back in 1983, Bob Stick), the University of British Columbia, and the Klapötke group at Munich. That last bunch is known to readers of "Things I Won't Work With", as experts in energetic materials, and when I saw that name I knew I'd better read the paper pronto.

As it turns out, the hydrochloride isn't quite as optimal as thought. It's impact-sensitive, for one thing, and not shelf-stable. The new paper mentions that it decomposes with an odor of hydrazoic acid on storage - you don't want odors of hydrazoic acid, believe me - and I thought while reading that, "Hmm. My bottle of the stuff is white crystalline powder; that's strange." But then I realized that I hadn't looked at my bottle for a few months. And as if by magic, there it was, turning dark and gooey. I had the irrational thought that the act of reading this paper had suddenly turned my reagent into hazardous waste, but no, it's been doing that slowly on its own.

So if you have some of this reagent around, take care. The latest work suggests that the hydrogensulfate salt, and especially the fluoroborate, are less sensitive and more stable alternatives to the hydrochloride, and I guess I'll have to make some at some point. (They also made the perchlorate - just for the sake of science, y'know - and report, to no one's surprise, that it "should not be prepared by those without expertise in handling energetic materials"). But it needs no ghost come from the grave to tell us this.

So, back to my lab and my waste-disposal problem! And here's a note on the literature. We have the original prep of the reagent, a follow-up note on stability problems, and this latest paper on alternatives. But when you go back to the original paper, there is no mention of the later hazard information. Shouldn't there be a note, a link, or something? Why isn't there? Anyone at Organic Letters or the ACS care to comment on that?

Update: I've successfully opened my bottle, with tongs and behind a blast shield, just to be on the safe side, and defanged the stuff off by dilution.

Comments (17) + TrackBacks (0) | Category: Chemical News | Safety Warnings | The Scientific Literature

February 14, 2012

An Elsevier Boycott

Email This Entry

Posted by Derek

There's been a movement afoot to boycott Elsevier journals. It's started over in the mathematics community, led by Timothy Gowers, a serious mathematician indeed. The objections to Elsevier are the ones you'd think: high prices, unsplittable bundles of journal subscriptions for institutions, and their strong support for legislation like the Research Works Act.

Writing about this is tricky, since I'm on the editorial board of an ACS journal that competes with Elsevier titles. Of course, as that link in the first paragraph shows, Nature Publishing Group has no problem talking about the issue themselves, and they're competing tooth and claw with Elsevier. At any rate, there's now a central website for the boycott movement, and it continues to gain publicity. There are, of course, some field where Elsevier is more prominent than others - biomedically, the Cell Press journals (and The Lancet) are heavy hitters, so a real test of this movement will be to see how many people from these fields it can attract.

Personally, I think that the current system of scientific publishing is increasingly outmoded, although I'm loath to forecast what will replace it. But we could be looking at another step in its demise.

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

February 8, 2012

Every Methods Paper Has a Table

Email This Entry

Posted by Derek

Here's a helpful translation, and there's more truth in it than there should be. My rule of thumb is to be extremely suspicious of a methods paper that doesn't have at least a couple of low-yielding or "NR" entries. If they aren't there, it means that someone didn't do enough experiments (or, worse, that they're not telling you about those little details).

Comments (21) + TrackBacks (0) | Category: The Scientific Literature

February 3, 2012

How the Andrulis Paper Got Published

Email This Entry

Posted by Derek

The editor of the journal Life has published an attempt at detailing how the notorious Andrulis paper managed to make its way into print. See how convincing you find it. In the course of explaining that it can be hard to find reviewers for interdisciplinary topics, and how the journal tries to find reputable people in each field (and carefully checks author suggestions for reviewers), we have this:

Life is a new journal that deals with new and sometime difficult interdisciplinary matters. Consequently, the journal will occasionally be presented with submitted articles that are controversial and/or outside conventional scientific views. Some papers recently accepted for publication in Life have attracted significant attention. Moreover, members of the Editorial Board have objected to these papers; some have resigned, and others have questioned the scientific validity of the contributions. . .

. . .In the case of the Dr. Andrulis’s long paper, the two reviewers were both faculty members of reputable universities different than the author’s and both went to considerable trouble presenting lengthy review reports. Dr. Andrulis revised his manuscript as requested, and the paper was subsequently published.

Really? Is that how it really went? I know what I would have said if they'd sent the paper to me: that it was a perfect example of what happens when an active, learned mind begins to slip loose from its moorings, and that while the paper appeared to have no scientific merit at all, it was quite useful as a diagnostic sign of oncoming psychosis.

If you only read the Life editor's remarks without reading any of the original paper, you might find them reasonable. But that's because you haven't been exposed to a theory that purports to explain the abiotic origins of life, the underlying principles of biochemistry, the formation of the solar system, the expansion of the universe, global weather patterns, the structure of cellular membranes, the distributions of comets and asteroids, the origins of riboviruses, the protein folding problem, the nature of biological aging, and the unification of quantum mechanics with general relativity. I have not made any of that up, it's all in the paper, and I would very much like to see a reviewer who could let all that go past. "Publish with revisions", sure.

Comments (23) + TrackBacks (0) | Category: The Scientific Literature

January 31, 2012

The Andrulis Paper's Fallout

Email This Entry

Posted by Derek

The fallout from the bizarre Andrulis paper continues. Carl Zimmer reports that editorial board members are resigning from the journal, having had no idea that their names would wind up over something like this.

Naturally, that brings up the question of just who did let this thing through the review process, but my bet is that we'll never know. Whoever signed off on it is no doubt running for cover.

Another useful feature this affair has had is the chance to see who just posts press releases for fun and profit, and who has some tiny residual bit of editorial discretion. In the former category, apparently, are and (the latter has taken down their post. But then again, the Times of India bit for it as well. . .

Comments (16) + TrackBacks (0) | Category: The Scientific Literature

January 30, 2012

The Key to Everything? Not Quite.

Email This Entry

Posted by Derek

Here's one of the strangest things I've ever seen in the scientific literature. A new journal, Life, apparently solicited papers for their inaugural issues, and one of them was from Erik Andrulis at Cast Western's School of Medicine. The manuscript came in at 105 printed pages, which should have rung at least a tiny alarm bell, you'd think. And if that wasn't a bit concerning, perhaps the title ("Theory of the Origin, Evolution, and Nature of Life") might have seemed a bit sweeping? Or the abstract, which promises that "The theoretical framework unifies the macrocosmic and microcosmic realms, validates predicted laws of nature, and solves the puzzle of the origin and evolution of cellular life in the universe." No? Nothing to worry about yet?

But editors aren't supposed to just look at page counts, titles, and abstracts. Just a riffle through the actual manuscript should have been enough to convince anyone that, rather than a Theory of Everything, that this work is, most unfortunately, the product of a disordered mind. P. Z. Myers has excerpts from the paper on his blog - take a look and see what you think. Here's a sample, and it should really be sufficient:

The ultimate state of gyromnemesis is the stably adapted particle or gyronexus in the gyrobase. . .Finally, although a diquantal IEM (X'') undergoes gyrognosis as the gyrobase of a primary majorgyre, it undergoes gyromnemesis as the gyrapex of an alternagyre.

Right. The paper ranges through the origins of life, organic chemistry, cosmology, geology, astronomy, and who knows what else, all of it explained in language exactly like the above. And yes, there is a multi-page glossary of all those gyro-terms, and no, it does not help. As Myers points out, the spectacularly weird thing is that not only did this paper get published, it got press-released by Case Western. Here, check it out. Whoever put this thing together has gamely attempted to summarize the paper, and not only that, to highlight its importance for the greater glory of Case Western:

To test his paradigm, Dr. Andrulis designed bidirectional flow diagrams that both depict and predict the dynamics of energy and matter. While such diagrams may be foreign to some scientists, they are standard reaction notation to chemists, biochemists, and biologists.

Dr. Andrulis has used his theory to successfully predict and identify a hidden signature of RNA biogenesis in his laboratory at Case Western Reserve University School of Medicine. He is now applying the gyromodel to unify and explain the evolution and development of human beings.

Oh, go take a look and tell me if you see any standard notation. (Update: I see from RetractionWatch that the university has pulled the release from their own sites, saying that they're "evaluating our processes regarding media outreach". I'll bet they are(. Now, I realize that picking up a text on, say, quantum electrodynamics could lead to the same what-is-this-stuff feeling. But any text on QED starts with a grounding in the physical world and the connections of the theory to known physics. And this sort of thing is different in both degree and kind (for one thing, QED has nothing to say about lunar craters). There's a difference between a work that makes you think "Boy, I don't understand this" and one that makes you think "Boy, this person has lost it". The near-infallible signs of scientific derangement include the "Why, this explains everything" aspect, the "Everything you thought you knew is wrong" one, and the intricate details-within-details style, almost always taken to unbearable lengths.

What the Andrulis paper reminds me of, actually, is Alfred Lawson and his Lawsonomy. That one also explains everything from bacteria to the composition of the moon, and brings in "zig-zag and swirl" motions to do so, at excruciating length. No, if you've had any exposure to the fill-the-margins-with-green-ink thinkers, you'll recognize Andrulis' problem, and hope that he can get some sort of help for it. Here's a book-length collection of such, very interesting for what it shows you about the ways that human reason can go off the rails.

That's something I've thought about for a long time - in fact, here's an entry on this blog from ten years ago on that very subject. It's interesting to me that there are a limited number of relatively defined mental illnesses; I think that says something about the deeper structures of human consciousness. The Andrulis paper is a flawless example of one of those categories - the wildly intricate, over-systematized Key to the Universe. I've just never seen one in a scientific journal.

Comments (68) + TrackBacks (0) | Category: The Scientific Literature

January 19, 2012

The Research Works Act: One (Two!) Against and One For

Email This Entry

Posted by Derek

I noted that the Nature Publishing Group has come out against the proposed Research Works Act, which would roll back the requirement that research funded by the US government be made freely available after (at most) one year. They are, I believe, the largest and most prominent journal publisher to take such a stand (although I'll be glad to be wrong about that):

NPG and Digital Science do not support the Research Works Act.

NPG and Digital Science exist to support the creation and dissemination of human knowledge on a sustainable commercial basis. We seek to enable the open exchange of ideas, especially in scientific communities, in line with the requirements and objectives of relevant stakeholders.

Update: from the comments, the AAAS (publishers of Science) have also come out against the RWA, saying that they're fine with the current system, and that their membership in the AAP does not mean that the organization speaks for them on this issue. How about the American Chemical Society? As far as I can tell, the ACS has made no statement, and silence speaks loudly.

Meanwhile, Rich Apodaca at Depth-First surprised me with this post coming out in favor of the RWA. But read the whole thing. He is, as the Marxists used to say, interested in "heightening the contradictions", and sees the scientific publishing industry bringing down the roof on its head even faster if the act passes. And the sooner that happens, he says, the sooner we can get rid of an outmoded system:

Any scientist who has been an active participant in scientific publication as an author, reviewer, and consumer recognizes that the only remaining value added by scientific publishers today is imprimatur. Imprimatur is the implied endorsement received by authors who publish in certain scientific journals, particularly in those that earned a high level of prestige during the pre-digital period of publication scarcity.

Ironically, imprimatur remains so valuable in science that it has kept numerous publishers afloat despite wave upon wave digital destruction being visited on sister industries such as book publication and newspapers.

But imprimatur can lose its luster, particularly in an environment in which fewer and fewer scientist can actually read the publications appearing in ‘high-impact’ journals. Prestige counts for nothing in science if your peers can’t read your papers. Nevertheless, that’s where scientific publication is heading.

I'm not sure which way is faster, myself. But we agree that the current scientific publishing model is being eroded, and that this is an opportunity, not a disaster that has to be repaired with legislation.

Comments (9) + TrackBacks (0) | Category: The Scientific Literature

January 17, 2012

Down With the Research Works Act

Email This Entry

Posted by Derek

Back in December, a short bill was introduced in the House called the "Research Works Act". Its backers, Darrell Issa (R-CA) and Carolyn Maloney (D-NY), describe it as something that will maintain the US's standing in scientific publishing. After looking over its language and reading a number of commentaries on it, I have to disagree: this looks to me like shameless rent-seeking by the commercial scientific publishers.

And it pains me to say that, because I know several people in that business. But it's a business whose long-term model has problems. (See the Addendum below if you're not in the field and want a brief summary of how scientific publishing works). The problem is, the work of the editorial staff has changed a good deal over the years. Back when everyone sent in hard copies of papers, in who knows what sort of format, there was a good deal of work to do just turning the good ones into a consistent journal. Electronic submission has ironed a lot of the grunt work out - it's still work, but it's not what it used to be.

That leaves the higher editorial functions themselves, and here's where the arguing starts. Most, and in some cases all editing of content is done by unpaid peer reviewers. There are journals whose editors exist mainly to keep the flow of submissions moving to the reviewers, and from them back into the official journal, while hardly ever laying a finger on the copy itself. They function as Peer Review Mailroom Managers. And while that's a necessary job, it's the center of the argument about scientific publishing today. How much, exactly, is it worth?

Scientific journal are expensive. I mean, really, really expensive to subscribe to. And if you're not a subscriber, access to individual papers is pretty steep, too - typically in the $15 to $50 range. This is the business model for commercial scientific publishing: create a space with value (reputation, name recognition) and charge the maximum that that traffic will bear. And that's fine; there are a lot of businesses that work the same way - if they can.

The problem is, the information-sharing capabilities of the Internet blow a large hole in some of the traditional publishing model. And another problem is that a large number of papers that come into the journals from US academic researchers have had some (or all) of that work paid for by government grants (NIH, NSF, DOE and so on). As it stands, articles funded by the NIH are available in PubMed Central for free access, no later (by law) than 12 months from the initial journal publication. Researchers can also submit their work to "open access" journals (such as those from the Public Library of Science), which charge a fee to authors to defray editorial costs, but then allow immediate unlimited access to all comers once a paper is accepted. (I should note that some commercial journals get away with "page charges" as well, and some have a model where the authors can pay extra to bring their paper out from behind the paywall).

And here's where we have the Research Works Act. It would forbid any publication in an open access journal for anything funded in academia by US government grants, and it would forbid any public-access repository for such work. That's its purpose. Well, to be more accurate, its purpose, as described by the head of the Association of American Publishers, is that it "ensures the sustainability of the industry". Yep, make my business model part of statutory law, and beggar my competition: what else is a government for, anyway?

Update: see the comments section. I'm interpreting the text of the law to mean the above, but another way to read it - probably the correct one - is that it's mainly rolling back the 2008 law that mandates that NIH-funded papers go open-access after a year. But that's bad enough as it stands.

To their credit, the MIT Press looks like the first big academic publisher to defect from this position. But the commercial publishers (Elsevier, Wiley, and so on) will never give up on this goal. Yes, the RWA, according to them, is aimed at "preventing regulatory interference with private-sector research publishers". Here's Congresswoman Maloney using Elsevier's own press release language, sentence by sentence, as detailed by Michael Eisen, co-founder of PLoS. (He also has an op-ed in the New York Times on this issue).

I see no reason why we should make the current scientific publishing system a matter of law. I think it should change - and be allowed to change - as new technology allows it to. And I think that the Research Works Act is nothing more a blatant attempt to hold on to a profitable business plan.

Addendum: For those outside the scientific world, here's a brief summary of how things have traditionally worked. As a scientist (academic or industrial), you take the time and effort to write up your results for a journal. You have to pick your journal at the start of the process, since each of them have their own ways of organizing a paper, their own preferred way of citing other papers as references, and so on. Anyone can send anything to any journal they feel like, although you'd be well advised to target your paper to the ones that (a) have the best chance to actually accept it and (b) will do you good to have a paper published in. The overlap between those two may not be large, or may not exist at all, depending on your paper. These days, most journals have templates for Word or the like, which standardizes the submissions, and have some sort of automatic PDF generation during the submission step step so you can see how the paper will look when formatted in the journal's style and page layout.

Your manuscript is given a quick check to make sure that it's appropriate for review. A few of of the higher-end journals make this a key step, because they can afford to turn down even rather interesting papers as not necessarily worth their time to go on checking. But in most journals, unless there's something obviously off, your paper goes out for peer review. Two or more scientists from Out There Somewhere look it over and send in comments. Those comment forms have a section for the original authors to see, and a section for remarks that go just to the editorial staff, and you can use those as you see fit. (I, for example, once used the latter forum to ask the editors to please stop sending me papers from a certain author, because I'd done three of them and couldn't stand to see any more. They honored my request.)

As an author, you see the comments when they come back and get a recommendation from the journal - usually it's "Publish as is", "Publish after minor revisions", "Publish after major revisions", or "Go away". That last one usually isn't expressed in quite those words. The middle two are the most common, since most stuff eventually gets published somewhere if the authors are persistent enough (and are willing enough to have their work appear in the Zambodian Journal of Chemistry or what have you).

Now at this point, traditionally, the work of assembling the accepted papers into a printed journal kicks in on the editorial side. And it still does, but that process is becoming less and less important. I honestly can't tell you when I last saw a hard copy of any of the journals I read regularly. Even the idea of separate issues is becoming antiquated, since new papers (in the case of many journals) just plop out onto the web site (and into the RSS feeds) as they emerge from the review process.

Comments (35) + TrackBacks (0) | Category: The Scientific Literature

January 16, 2012

Defending Das' Resvertrol Research. Oh, Come On.

Email This Entry

Posted by Derek

I'm getting all the press releases from Bill Sardi, of Resveratrol Partners, as he does damage control from the Das scandal at UConn. And I have to say, he's putting in the hours getting these together. Problem is, on some key points, he doesn't know what his biggest problems are.

The latest one is titled "World Without Resveratrol: Researcher Falsely Accused", and claims that this may all be a plan to "send a message" to any academic who collaborates with the makers of resveratrol pills. The release goes on about how these are old accusations, which Das has refuted since then, and asks why these ancient concerns are coming up now, eh? The phrase "orchestrated hit job" is used. But that glosses over the times of the whole investigation, which has been a very detailed and involved one, and glosses over the amount of due process involved as well. There are a lot of problems with the publications from the Das lab, as detailed in the report that I linked to the other day, and tying them together has involved a lot of work.

But here comes my favorite part of the latest Sardi release:

". . .I asked Dr. Das directly, did he altered (sic) western blot images, or directed others in his lab to do so. While his initial answer was no, meaning he had not fabricated or altered any scientific finding, altering western blot images are a common practice in laboratories for reasons other than deception. The university chose to present their findings in a derogatory manner. Dr. Das explains that editors at scientific publications commonly request researchers enhance faded images of western blot tests so they can be duplicated in their publications. Western blot tests are frequently altered to remove backgrounds, enhance contrast and increase dots-per-inch resolution so they are suitable for publication. This had been fully explained to university officials long before. . .

No, no, no. The problems with the Das papers have nothing to do with enhancing the contrast on Western blots. They have to do with cutting and pasting sections of them, rearranging them, reusing them, and creating them out of pieces of other experiments. Look at that report. These people appear to have spent a ridiculous amount of time assembling "Western blots" out of miscellaneous digitized chunks. The resulting figures purport, in many cases, to represent particular experiments, but they do no such thing. They represent a bunch of previous bands from other experiments entirely, sliced and diced in a way that would seem to have no other possible motive than to deceive. Come on.

Oh, but there's more. Here, according to the press release, is how a cutting-edge academic lab works these days:

"As I drilled Dr. Das’ former students with questions, I found that lead researchers like Dr. Das do not do any lab bench experiments. Students do all the work and submit their results to him via e-mail or by directly downloading data into his computer. Dr. Das says when he is not traveling his office is open and students can enter and download data directly onto his computer. I had previously visited Dr. Das at the University of Connecticut and noticed his office door was left open and anyone could have access to his computer.

One former student told me that typically lead researchers like Dr. Das write the introduction and conclusion of experiments and the students enter all the data, before publication in scientific journals. Dr. Das, who is busy lecturing all over the globe because of his groundbreaking studies, does not directly oversee tests that are performed, and neither do most other lead researchers. The University of Connecticut report says the university holds Dr. Das responsible for all of the data. Probably most lead researchers in scientific laboratories around the globe are vulnerable to errors or even fabrication of data by their students."

Where to start? What the heck is this "download data directly into his computer" stuff? And what about all the doctored files found on other machines in the group? And yes, while lead authors are indeed vulnerable to errors and fabrication, this sort of thing typically does not involved years of work spread out across dozens of papers in multiple journals. Even the busiest and most distracted principal investigator might be expected to take the time to notice, eventually, that his group's work is a tower of fraud. And yes, the University should hold Dr. Das responsible for the data in his papers. His name is on the grants, his name is on the office door, he's the one with a high-paying tenured position while the students are cranking away under low salaries and stipends, and it's his name with an asterisk next to it on all those papers, as the contact person for any questions about them. Damn right he's responsible. He's responsible for making sure that anything going out into the literature with his name on it is something that he can stand behind.

Ah, but not to worry. It's all being taken care of:

"Dr. Das says many editors at scientific journals don’t believe the University of Connecticut report. They full-well know that editing of western blot tests is common practice and that the tests in question in no way invalidate his work and were only one part of the evidence provided in his papers from which Dr. Das drew conclusions. This is the case of scientific fraud that wasn’t."

That would explain why Dr. Das has been pulled from the co-editor job he had at one of those journals. They must believe him. And that would also shore up all those allegations of prejudice against East Indian researchers, since the editor of that journal is. . .well, he's Indian too, but you know what I mean. (Personally, if I were from India myself, I'd be furious at Das for helping to drag the reputation of my country's scientists through the mudhole, but maybe that's just me.)

No, I hope these press releases keep on coming. So far, we have lots of elaborate reasons why Dr. Das had nothing to do with all these fabricated Western blots, but who cares, right, since they're only a tiny part of his papers, which are great and important work even though he really doesn't write them anyway, and no, he has almost no connection with Longevinex and Resveratrol Partners, which is why the head of the company is spending all this time defending him in this case of minor stuff he never did, all 600 pages of summary and 60,000 pages of investigation material, and that explains why the journals that believe him are ditching him from their mastheads and publishing retractions of those great papers. Because it's all a conspiracy. Yeah. That's it.

Comments (28) + TrackBacks (0) | Category: Aging and Lifespan | The Dark Side | The Scientific Literature

January 6, 2012

SciFinder Access For the Unemployed

Email This Entry

Posted by Derek

If you had SciFinder access, but are now unemployed and would like to use it during your job hunt, CAS now has a program to make that possible for free. I'm glad to see them taking this step; a lot of people have asked for something like this for some time now.

Comments (24) + TrackBacks (0) | Category: Business and Markets | The Scientific Literature

January 5, 2012

Lead-Oriented Synthesis - What Might That Be?

Email This Entry

Posted by Derek

A new paper in Angewandte Chemie tries to open another front in relations between academic and drug industry chemists. It's from several authors at GSK-Stevenage, and it proposes something they're calling "Lead-Oriented Synthesis". So what's that?

Well, the paper itself starts out as a quick tutorial on the state and practice of medicinal chemistry. That's a good plan, since Angewandte Chemie is not primarily a med-chem journal (he said with a straight face). Actually, it has the opposite reputation, a forum where high-end academic chemistry gets showplaced. So the authors start off by reminded the readership what drug discovery entails. And although we've had plenty of discussions around here about these topics, I think that most people can agree on the main points laid out:

1. Physical properties influence a drug's behavior.
2. Among those properties, logP may well be the most important single descriptor,
3. Most successful drugs have logP values between 1 and perhaps 4 or 5. Pushing the lipophilicity end of things is, generally speaking, asking for trouble.
4. Since optimization of lead compounds almost always adds molecular weight, and very frequently adds lipophilicity, lead compounds are better found in (and past) the low ends of these property ranges, to reduce the risk of making an unwieldy final compound.

As the authors take pains to say, though, there are many successful drugs that fall outside these ranges. But many of those turn out to have some special features - antibacterial compounds (for example) tend to be more polar outliers, for reasons that are still being debated. There is, though, no similar class of successful less polar than usual drugs, to my knowledge. If you're starting a program against a target that you have no reason to think is an outlier, and assuming you want an oral drug for it, then your chances for success do seem to be higher within the known property ranges.

So, overall, the GSK folks maintain that lead compounds for drug discovery are most desirable with logP values between -1 and 3, molecular weights from around 200 to 350, and no problematic functional groups (redox-active and so on). And I have to agree; given the choice, that's where I'd like to start, too. So why are they telling all this to the readers of Angewandte Chemie? Because these aren't the sorts of compounds that academic chemists are interested in making.

For example, a survey of the 2009 issues of the Journal of Organic Chemistry found about 32,700 compounds indexed with the word "preparation" in Chemical Abstracts, after organometallics, isotopically labeled compounds, and commercially available ones were stripped out. 60% of those are outside the molecular weight criteria for lead-like compounds. Over half the remainder fail cLogP, and most of the remaining ones fail the internal GSK structural filters for problematic functional groups. Overall, only about 2% of the JOC compounds from that year would be called "lead-like". A similar analysis across seven other synthetic organic journals led to almost the same results.

Looking at array/library synthesis, as reported in the Journal of Combinatorial Chemistry and from inside GSK's own labs, the authors quantify something else that most chemists suspected: the more polar structures tend to drop out as the work goes on. This "cLogP drift" seems to be due to incompatible chemistries or difficulties in isolation and purification, and this could also illustrate why many new synthetic methods aren't applied in lead-like chemical space: they don't work as well there.

So that's what underlies the call for "lead-oriented synthesis". This paper is asking for the development of robust reactions which will work across a variety of structural types, will be tolerant of polar functionalities, and will generate compounds without such potentially problematic groups as Michael acceptors, nitros, and the like. That's not so easy, when you actually try to do it, and the hope is that it's enough of a challenge to attract people who are trying to develop new chemistry.

Just getting a high-profile paper of this sort out into the literature could help, because it's something to reference in (say) grant applications, to show that the proposed research is really filling a need. Academic chemists tend, broadly, to work on what will advance or maintain their positions and careers, and if coming up with new reactions of this kind can be seen as doing that, then people will step up and try it. And the converse applies, too, and how: if there's no perceived need for it, no one will bother. That's especially true when you're talking about making molecules that are smaller than the usual big-and-complex synthetic targets, and made via harder-than-it-looks chemistry.

Thoughts from the industrial end of things? I'd be happy to see more work like this being done, although I think it' going to take more than one paper like this to get it going. That said, the intersection with popular fragment-based drug design ideas, which are already having an effect in the purely academic world of diversity-oriented synthesis, might give an extra impetus to all this.

Comments (34) + TrackBacks (0) | Category: Chemical News | Drug Assays | Drug Development | The Scientific Literature

January 4, 2012

The Changing Literature

Email This Entry

Posted by Derek

The folks behind Retraction Watch (Adam Marcus and Ivan Oransky) have a piece in Nature on what it's been like since they started the blog.

They note a new trend - for new data to appear when a retraction is called for (or made), but without any clarity about whether these new corroborative results have been peer-reviewed themselves. And they're absolutely right that a retraction should state exactly why the paper is being retracted; those "This paper has been withdrawn by the authors" notices are less than useless.Their experiences have them calling for a different way of looking at scientific papers in general:

". . . It is important to point out that an increase in retractions isn't necessarily a bad thing, because they correct the scientific record. But the greater visibility of papers and retractions today adds to the evidence revealing why editors need to handle retractions more transparently. In turn, researchers need to stop emphasizing the paper so much.

What is needed, instead, is a system of publication that is more meritocratic in its evaluation of performance and productivity in the sciences. It should expand the record of a scientific study past an individual paper, including additional material such as worthy blog posts about the results, media coverage and the number of times that the paper has been downloaded."

It's true that more and more of this is being done out here on the internet, in public, and in real time. (I'm glad to say that some of it is done on this site). The new Crossmark system (now being tested) might be a way to keep up with all these extensions, and link them to the original paper. Such a system would have come in very handy indeed during the "arsenic bacteria" business, during which just finding all the useful comments was a real job in itself. There are authors who will not care for this sort of thing, but when you publish a paper, you're opening the door to public comment (and criticism). It's just that now we have the tools to do that more quickly and thoroughly.

Comments (4) + TrackBacks (0) | Category: The Scientific Literature

December 28, 2011

Nowhere to Go But Up?

Email This Entry

Posted by Derek

I wanted to let people know that I've got a "Perspective" piece in ACS Medicinal Chemistry Letters, entitled "Nowhere to Go But Up?". The journal is starting to run these opinion/overview articles, and contacted me for one - I hope it's the sort of thing that they were looking for!

Comments (38) + TrackBacks (0) | Category: Drug Industry History | The Scientific Literature

December 20, 2011

Best Paper You Read This Year?

Email This Entry

Posted by Derek

Since the year is winding down, I had a question for the readership: what's the best scientific paper you read this year? Let's restrict the field to chemistry and biology; I'm not ready for particle physics. And I don't necessarily mean "best written", although you're free to nominate some if you can find any that stood out. What I had in mind was more "Most interesting" or "Most unexpectedly useful", or "Most surprising". I've got a couple in mind myself - I'll turn the suggestions into another post or two.

Comments (58) + TrackBacks (0) | Category: The Scientific Literature

December 6, 2011

Riding to the Rescue of Rhodanines

Email This Entry

Posted by Derek

There's a new paper coming to the defense of rhodanines, a class of compound that has been described as "polluting the scientific literature". Industrial drug discovery people tend to look down on them, but they show up a lot, for sure.

This new paper starts off sounding like a call to arms for rhodanine fans, but when you actually read it, I don't think that there's much grounds for disagreement. (That's a phenomenon that's worth writing about sometime by itself - the disconnects between title/abstract and actual body text that occur in the scientific literature). As I see it, the people with a low opinion of rhodanines are saying "Look out! These things hit in a lot of assays, and they're very hard to develop into drugs!". And this paper, when you read the whole thing, is saying something like "Don't throw away all the rhodanines yet! They hit a lot of things, but once in a while one of them can be developed into a drug!" The argument is between people who say that elephants are big and people who say that they have trunks.

The authors prepared a good-sized assortment of rhodanines and similar heterocycles (thiohydantoins, hydantoins, thiazolidinediones) and assayed them across several enzymes. Only the ones with double-bonded sulfur (rhodanines and thiohydantoins) showed a lot of cross-enzyme potency - that group has rather unusual electronic properties, which could be a lot of the story. Here's the conclusion, which is what makes me think that we're all talking about the same thing:

We therefore think that rhodanines and related scaffolds should not be regarded as problematic or promiscuous binders per se. However, it is important to note that the intermolecular interaction profile of these scaffolds makes them prone to bind to a large number of targets with weak or moderate affinity. It may be that the observed moderate affinities of rhodanines and related compounds, e.g. in screening campaigns, has been overinterpreted in the past, and that these compounds have too easily been put forward as lead compounds for further development. We suggest that particularly strong requirements, i.e. affinity in the lower nanomolar range and proven selectivity for the target, are applied in the further assessment of rhodanines and related compounds. A generalized "condemnation" of these chemotypes, however, appears inadequate and would deprive medicinal chemists from attractive building blocks that possess a remarkably high density of intermolecular interaction points.

That's it, right there: the tendency to bind off-target, as noted by these authors, is one of the main reasons that these compounds are regarded with suspicion in the drug industry. We know that we can't test for everything, so when you have one of these structures, you're always fearful of what else it can do once it gets into an animal (or a human). Those downstream factors - stability, pharmacokinetics, toxicity - aren't even addressed in this paper, which is all about screening hits. And that's another source of the bad reputation, for industry people: too many times, people who aren't so worried about those qualities have screening commercial compound collections, come up with rhodanines, and published them as potential drug leads, when (as this paper illustrates), you have to be careful even using them as tool compounds. Given a choice, we'd just rather work on something else. . .

Comments (7) + TrackBacks (0) | Category: Drug Assays | Drug Development | The Scientific Literature

December 1, 2011

Nevirapine: Not Chiral. Paper: Not Right. Editorial Staff: Not Doing Their Job

Email This Entry

Posted by Derek

Readers may remember a now-retracted paper that I first blogged about here, the one that claimed to have isolated the reverse transcriptase inhibitor nevirapine as a natural product. Moreover, it claimed that the isolated material was chiral, which would have been very interesting indeed if it were true. (And, as that last post says, would have been worth making a big point of, if the authors really had understood what they were claiming).

Now a group from Manchester has weighed in on that topic. And what they find is what anyone who'd examined the field should have expected: that the nevirapine molecule, although capable of existing in two chiral forms, equilibrates between them on a time scale of seconds at room temperature. Isolating the atropisomers by standard means is not possible.

So everything about that original Tetrahedron paper was wrong; it never should have made it through the review process. And that's why I highlight such things - not to heap scorn on the original authors, which doesn't do that much good, but to heap it on the people who let such papers into print. Reviewers and editors are supposed to notice when a paper has made very unusual claims, and they're supposed to ask the authors to back them up. But the folks at Tetrahedron were asleep at the switch when this one came through. It's important for them (and other editorial staffs) not to let that happen, and it's important for a journal's readers to realize that it can.

Addendum - as an aside, I note that one of this blog's entries (the second link above) is cited in the references of this latest paper. I'm glad to be a cite-able source!

Comments (8) + TrackBacks (0) | Category: Natural Products | The Scientific Literature

November 4, 2011

Table Of Content Graphics, Mocked

Email This Entry

Posted by Derek

Someone has to make fun of lousy graphical abstracts in science journals. And apparently this is that person. I've no idea who this is, but they're helping, in their way, to make the world a better place.

Comments (19) + TrackBacks (0) | Category: The Scientific Literature

September 12, 2011

The Scientific Literature Gets Kicked Around

Email This Entry

Posted by Derek

It seems that the credibility of the scientific literature has been taking a beating recently. This has come about for several reasons, and through several different motivations. I'll get one of the most important out of the way first - politics. While this has been a problem for a long time, there's been a really regrettable tendency in US politics the last few years, a split across broadly left/right lines. Cultural and policy disagreements have led to many on the left claiming the Dispassionate Endorsement of Settled Science, while others on the right end up complaining that it's nothing of the sort, just political biases given a quick coat of paint. Readers will be able to sort several ongoing controversies into that framework.

Political wrangling keeps adding fuel to the can-we-trust-the-literature argument, but it would still be a big issue without it. Consider the headlines that the work of John Ioannidis draws. And there's the attention being paid to the number of retractions, suspicions of commercial bias in the medical literature, the problems of reproducibility of cutting-edge results, and to round it all off, several well-publicized cases of fraud. No, even after you subtract the political ax-grinding, there's a lot of concern left over (as there should be). There are some big medical and public policy decisions to be made based on what the scientific community has been able to figure out, so the first question to ask is whether we've really figured these things out or not.

A couple of recent articles prompted me to think about all this today. The Economist has a good overview of the Duke cancer biomarker scandal, with attention to the broader issues that it raises. And Ben Goldacre has this piece in The Guardian, highlighting this paper in Nature Neuroscience. It points out that far too many papers in the field are using improper statistics when comparing differences-between-differences. As everyone should realize, you can have a statistically significant effect under Condition A, and at the same time a lack of a statistically significant effect under Condition B on the same system. But that doesn't necessarily mean that the difference between using Condition A versus Condition B is statistically significant. You need to go further (usually ANOVA) to be able to say that. The submission guidelines for Nature Neuroscience itself make this clear, as do the guidelines for plenty of other journals. But it appears that a huge number of authors go right ahead and draw the statistically invalid comparison anyway, which means that the referees and editors aren't catching it, either. This is not the sort of thing that builds confidence.

So the questions about the reliability of the literature are going to continue, with things like this to keep everyone slapping their foreheads. One can hope that we'll end up with better, more reliable publications when all this is over. But will it ever really be over?

Comments (10) + TrackBacks (0) | Category: The Scientific Literature

September 8, 2011

Publishing, Perishing, Buying and Selling

Email This Entry

Posted by Derek

Here's another article in the Guardian that makes some very good points about the way we judge scientific productivity by published papers. My favorite line of all: "To have "written" 800 papers is regarded as something to boast about rather than being rather shameful." I couldn't have put it better, and I couldn't agree more. And this part is just as good:

Not long ago, Imperial College's medicine department were told that their "productivity" target for publications was to "publish three papers per annum including one in a prestigious journal with an impact factor of at least five.″ The effect of instructions like that is to reduce the quality of science and to demoralise the victims of this sort of mismanagement.

The only people who benefit from the intense pressure to publish are those in the publishing industry.

Working in industry feels like more of a luxury than ever when I hear about such things. We have our own idiotic targets, to be sure - but the ones that really count are hard to argue with: drugs that people will pay us money for. Our customers (patients, insurance companies, what have you) don't care a bit about our welfare, and they have no interest in keeping our good will. But they pay us money anyway, if we have something to offer that's worthwhile. There's nothing like a market to really get you down to reality.

Comments (26) + TrackBacks (0) | Category: Academia (vs. Industry) | The Scientific Literature

August 31, 2011

The Finest Retraction Notice Ever?

Email This Entry

Posted by Derek

RetractionWatch has this gem from the Journal of Clinical Microbiology. My favorite part is midway through: "Moreover, we realized after our article had been published that major parts of the text had been plagiarized almost verbatim. . .".

Oh, yeah. There is that. But there's more at the link. The RetractionWatch people are trying to get more details, but I wish them luck. This looks like one of those things that no one is going to be very happy to talk about. . .

Comments (12) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

August 30, 2011

Why Isn't There an ArXiv For Chemistry?

Email This Entry

Posted by Derek

Nature has an article on the 20th anniversary of the ArXiv preprint server by its founder Paul Ginsparg. That's gradually worked its way through many parts of physics and mathematics to become a major route for releasing scientific results. (They're now getting about 7,000 submissions a month, with a million downloads a week). But many areas of science remain untouched:

Physicists were quick to adopt widespread sharing of electronic preprints, but other researchers remain reluctant to do so. Fields vary widely in their attitudes to data and ideas before formal review, and in choosing to share electronic preprints, each community will have to develop policies and protocols best suited to their users. A talk I gave in 1997 to a group of biologists helped catalyse the resource now known as PubMedCentral — run by the US National Institutes of Health. I served on the initial advisory board, which soon decided not to host any unrefereed materials, even carefully quarantined, in part for fear of losing essential publisher participation. There remain many legitimate reasons for individual researchers to prefer to delay dissemination, from uncertainty over correctness, to retaining extra time for follow-ups, to sociological differences in the way publication is regarded — in certain fields, the research somehow doesn't count until peer reviewed.

No community that has adopted arXiv usage has renounced it, however, so the growth has been inexorable. . .

But back in its early days, it looked like it would be even more inexorable than that:

The idea that print journals had outlived their usefulness was already in the air in the early 1990s. David Mermin memorably wrote in Physics Today in 1991: “The time is overdue to abolish journals and reorganize the way we do business.”1 By the mid 1990s, it seemed unthinkable that free and unfettered access to non-refereed papers on arXiv would continue to coexist indefinitely with quality-controlled but subscription-based publications. Fifteen years on, researchers continue to access both, successfully compartmentalizing their different roles in scholarly communication and reward structures.

The transition to article formats and features better suited to modern technology than to print on paper has also been surprisingly slow. Page markup formats, such as PDF, have only grudgingly given way to XML-based ones that support features such as manipulable graphics, dynamic views, linked annotations and semantic markup. . .

Well, in chemistry, that transition has been even slower. Our field is still very much dominated by the societies (ACS, RSC, etc.) and the commercial publishers like Elsevier, Nature, Wiley, etc. When's the last time - when's the first time - you heard of a significant organic chemistry paper appearing anywhere else? There's absolutely no equivalent of the ArXiv system, and although I know that the question has been asked before, I'll ask it again: why not? If someone had started such a thing back in the early 1990s, would it (could it) have taken off? Or are there other factors that would have kept it from doing so (the same ones as now?)

Comments (51) + TrackBacks (0) | Category: The Scientific Literature

August 24, 2011

What Are Impact Factors Doing to Chinese Science?

Email This Entry

Posted by Derek

Here's a useful (and rather brave) editorial from Chinese chemist Nai-Xing Wang in Nature. He's pointing out that the government's funding agencies are taking a crude and harmful approach to who gets research support: publish, and publish according to their schemes, or flippin' well perish:

The biggest problem remains the obsession with journal impact factors. Generally speaking, articles in journals with high impact factors are judged to appeal most to readers, but not every paper published in a high-impact-factor journal is high quality, and papers published in lower-ranked journals are never worthless. Yet some administrators in China take a very crude approach: high-impact-factor publications mean excellent work.

Research proposals are judged according to the impact factor of a scientist's previous publications. (And referees are usually selected on these criteria too.) Worse, the salaries of my chemistry colleagues go up or down depending on a complex mathematical formula based on the impact factor of the journals in which we publish our work — which we must supply in a detailed list.

Now, this sort of thing has been going on around the scientific world for a while, and it's going to be hard to put a complete stop to it. But the Chinese system that's described is about the most blatant that I've come across. The effects are pernicious:

If a high impact factor is the only goal of chemistry research, then chemistry is no longer science. It is changed to a field of fame and game. There are other effects too. Administrators in almost every university and research institute like to evaluate researchers by their papers at the end of each year. As a result, chemists often choose easy research topics that can be written up inside a year. . .

You get what you subsidize; I don't think that law is ever broken. And if the Chinese government wants people to crank out lots of papers in what they feel are high-end journals, well, that's what people will do. But if they want something useful to come out of all that effort, well, things might need to be adjusted a bit. But "useful" is a slippery word. For the people who are gainfully employed in keeping the current system running, it's just about as useful as it can be the way it is.

Comments (26) + TrackBacks (0) | Category: The Scientific Literature

August 11, 2011

Scientific Retractions: A Growth Industry?

Email This Entry

Posted by Derek

The Wall Street Journal has an interesting article based on data from Thomson Reuters on the frequency of retracted papers. It seems to be increasing dramatically:

Since 2001, while the number of papers published in research journals has risen 44%, the number retracted has leapt more than 15-fold, data compiled for The Wall Street Journal by Thomson Reuters reveal.

Just 22 retraction notices appeared in 2001, but 139 in 2006 and 339 last year. Through seven months of this year, there have been 210, according to Thomson Reuters Web of Science, an index of 11,600 peer-reviewed journals world-wide.

They mention Retraction Watch, as well they should. But ten years ago, would there have been enough new material to keep that blog running? Pharmalot has some more from its founder about what might be going on. There are, of course, more journals than ever these days, and many of them are junk. But it's not the bottom-tier journals that are driving this trend, I'd think, since honestly, when does anyone ever retract a paper in one of them? Consistent with that view, this bar chart of PubMed retractions by journal is heavily weighted towards the big dogs. A lousy or nonreproducible paper in one of the top journals is more likely to be of enough interest to attract attention, but one in J. Whatever will just sit there.

No, when you look at this chart, it appears that retractions-per-papers-published have been climbing, so the answer must be some combination of more mistakes, more fraud, or better policing. Retractions due to fraud seem to be where most of the growth is, according to this study, so that takes us down to the latter two explanations.

Software has definitely made the lazier sorts of fraud easier to detect, automatically flagging copy-and-paste hack jobs. But those aren't the kinds of things that show up in the better journals, are they? We may be seeing a mix of greater incentive to commit fraud and a rise in skepticism among readers. There have been enough cases, enough highly-publicized retractions and scandals, that more people may be willing to wonder if some exciting new result is true at all.

That's not a bad thing. The rise in fraud is a bad thing, but a corresponding rise in scrutiny is the only thing that's going to cure it. There are always a few pathological types out there that kind of know that they're going to get caught and kind of don't care. Those we shall always have with us, and not much is going to discourage them. But as for the rest of the fraudsters, the thought that they have a better chance of being found out and punished should give them something to think about.

Comments (15) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

August 1, 2011

Chinese Research: Not Quite the Juggernaut?

Email This Entry

Posted by Derek

A perennial topic around here has been the state of scientific research in China (and other up-and-coming nations). There's no doubt that the number of scientific publications from China has been increasing (be sure to read the comments to that post; there's more to it than I made of it). But many of these papers, on closer inspection, are junk, and are published in junk journals of no impact whatsoever. Mind you, that's not an exclusively Chinese problem - Sturgeon's Law is hard to get away from, and there's a lot of mediocre (and worse than mediocre) stuff coming out of every country's scientific enterprise.

But what about patents? The last couple of years have seen many people predicting that China would soon be leading the world in patent applications as well, which can be the occasion for pride or hand-wringing, depending on your own orientation. But there's a third response: derision. And that's what Anil Gupta and Haiyan Wang provide in the Wall Street Journal. They think that most of these filings are junk:

But more than 95% of the Chinese applications were filed domestically with the State Intellectual Property Office—and the vast majority cover "innovations" that make only tiny changes on existing designs. A better measure is to look at innovations that are recognized outside China—at patent filings or grants to China-origin inventions by the world's leading patent offices, the U.S., the EU and Japan. On this score, China is way behind.

The most compelling evidence is the count of "triadic" patent filings or grants, where an application is filed with or patent granted by all three offices for the same innovation. According to the Organization for Economic Cooperation and Development, in 2008, the most recent year for which data are available, there were only 473 triadic patent filings from China versus 14,399 from the U.S., 14,525 from Europe, and 13,446 from Japan.

Starkly put, in 2010 China accounted for 20% of the world's population, 9% of the world's GDP, 12% of the world's R&D expenditure, but only 1% of the patent filings with or patents granted by any of the leading patent offices outside China. Further, half of the China-origin patents were granted to subsidiaries of foreign multinationals. . .

The authors are perfectly willing to admit that this probably will change with time. But time can make things worse, too: as this editorial in Science last year made clear, the funding of research in China has some real problems. The authors of that piece are professors at two large Chinese universities, and would presumably know what they're talking about. For the biggest grants, they say:

. . .the key is the application guidelines that are issued each year to specify research areas and projects. Their ostensible purpose is to outline “national needs.” But the guidelines are often so narrowly described that they leave little doubt that the “needs” are anything but national; instead, the intended recipients are obvious. Committees appointed by bureaucrats in the funding agencies determine these annual guidelines. For obvious reasons, the chairs of the committees often listen to and usually cooperate with the bureaucrats. “Expert opinions” simply reflect a mutual understanding between a very small group of bureaucrats and their favorite scientists. This top-down approach stifles innovation and makes clear to everyone that the connections with bureaucrats and a few powerful scientists are paramount. . .

Given time, this culture could be changed. Or it could just become more entrenched as the amounts of money become larger and larger and the stakes become higher. China could end up as the biggest scientific and technological powerhouse the world has ever seen - or it could end up never living up to its potential and wasting vast resources on cargo-cult theatrics. It's way too early to say. But if many of those Chinese patents are just being written because someone's figured out that the way to get money and prestige is to file patents - never mind if they're good for anything - then that's not a good sign.

Comments (31) + TrackBacks (0) | Category: Patents and IP | The Scientific Literature | Who Discovers and Why

July 5, 2011

Fakery, As Revealed By Figures

Email This Entry

Posted by Derek

I note via Retraction Watch that the Journal of Biological Chemistry has issued retraction notices for four papers published from the group of the late Maria Diverse-Pierluissi, at the Mt. Sinai School of Medicine. One of their readers looked over the papers (which had been cited a few times, without making any particular huge impact, it seems), and found that some of the figures (Western blots and so on) repeat, even though they're supposed to represent different things (e.g., Figure 3A and 3C here).

Mt. Sinai told the Retraction Watch people that an internal investigation turned up the evidence of misconduct, and that the matter has been referred to the NIH, which funded the work. What those duplicate figures make me wonder, though, is how long it'll be before we have a plagiarized-figure search tool, in the same way that we have plagiarized-text tools running? There's already something similar out there - TinEye - and I'm sure that much nicer systems are available for a fee. Have any scientific journals implemented something like this?

Comments (18) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

June 30, 2011

Transcendental Meditation: Hold That Paper!

Email This Entry

Posted by Derek

I couldn't resist mentioning this one: the Archives of Internal Medicine was set to publish a paper showing a benefit for transcendental meditation in heart attack and stroke. Word was already out in the press - in the UK, the Telegraph had already published a story, with a quote from one of the paper's lead authors (from, ahem, the Maharishi University of Management) that the effect seen was as great or greater than any pharmaceutical intervention.

I don't have a link up to that particular newspaper report; its URL is no longer valid. That's because twelve minutes before the paper was set to be published online, the journal pulled it. (Other sources still have their stories up). We still don't know quite what the problem was. Nature got this statement:

“It became apparent that there was additional data not included in the manuscript that was about to be published, and the editor of Archives thought that the information was significant enough that it needed to be included as part of the paper, and then re-analyzed and verified, so she made the last-minute decision not to publish it. . .It’s an unusual situation, but the bottom line is that our journal wants to make sure that the information we put out is as accurate as can be.”

I'm glad to hear it. Larry Husten at Forbes has the data from the paper, and has a lot of questions. We'll see how things look when (and if) it ever appears. But for now, if you're looking for the latest anyone has ever pulled a paper before publication, we may well have the record.

Update: here's an excellent report on this at Retraction Watch.

Comments (19) + TrackBacks (0) | Category: Snake Oil | The Scientific Literature

June 7, 2011

Even Worse Than Reality

Email This Entry

Posted by Derek

I found this article in The American Scholar via Arts and Letters Daily, entitled "Flacking for Big Pharma". As you might have possibly guessed from the title, it's a broadside against the advertising practices of the drug industry, and particularly against its interactions with physicians and the medical journals.

And I'll say up front that the piece is not, in fact, completely wrong. It's probably not even mostly wrong. There really are big problems in these areas, such as too-aggressive promotion, minimization of side effects, too many payments to "key opinion leaders", too many studies that don't see the light of day, and so on. And these things really do lower the respect that people have for the drug industry - assuming, by this point, that there's much respect left. But overall, this article is sort of a summary version of Marcia Angell's book, for people who would like to hate the drug industry but find themselves pressed for time. And as such, it manages to get some important things wrong in the process of getting some things right.

For example, it makes much of subgroup analysis of clinical trials, but as a way for drug companies to pull the wool over readers' eyes. I wonder how much this really happens, though, since overzealous data mining of a trial that wasn't powered to generate such conclusion is (you'd think) a well-known pitfall by now. Perhaps not, though. But the example given in the article is BiDil:

BiDil proponents published studies that supported their claim of a racially mediated genetic anomaly that was addressed by BiDil, making it an ideal drug for blacks but not for whites.. . .

NitroMed won FDA approval of a new trial that included only 1,050 black subjects, with no white subjects to provide comparison data. Furthermore, BiDil was not tested alone, but only in concert with heart medications that are already known to work, such as diuretics, beta-blockers, and angiotensin-converting enzyme (or ACE) inhibitors. The published results of the trial were heralded as a success when subjects taking the drug combinations that included BiDil enjoyed 43 percent fewer heart-failure deaths.

. . .excluding whites was a medically illogical but financially strategic move because it eliminated the possibility that the drug would test well in whites, thereby robbing NitroMed of its already thin rationale for calling BiDil a black drug. The “black” label was crucial, because BiDil’s patent covering use in all ethnic groups expired in 2007, but the patent for blacks only allows NitroMed to profit from it until 2020. BiDil is a case study in research methodology “flaws” that mask strategies calculated to make a dodgy drug look good on paper, for profit.

But this doesn't appear to be correct. First off, as the article itself mentioned earlier, the BiDil combination was originally tested (twice) in racially mixed (in fact, I believe, mostly white) trial groups. Secondly, the 1,050-patient trial in black patients was done with other therapies because to do otherwise would be unethical (see below). And what you wouldn't realize by reading all this is the BiDil, in fact, was a failure. No one's making piles of profits on BiDil until 2020, especially not NitroMed. You wouldn't even know that NitroMed itself gave up trying to sell BiDil three years ago, and that the company itself was acquired (for a whopping 80 cents a share) in 2009.

Now, about those placebo-controlled trials. This article makes much of a British Medical Journal satire from 2003 on how to make a drug look good. But it's confused:

A placebo, such as a sham or “sugar” pill, has no active ingredient, and, although placebos may evoke some poorly understood medical benefits, called the “placebo effect,” they are weak: medications tend to outperform placebos. Placebo studies are not ethical when a treatment already exists for a disorder, because it means that some in the study go untreated. However, if you care only that your new drug shines in print, testing against placebo is the way to go.

Well, which is it? We can't, in fact, run placebo-controlled trials just to "shine in print" when there's a standard of care, you know. You can only do that when there's no standard of care at all. And in those cases, what exactly should we use as a comparison? Using nothing at all (no pills, nothing) would, in fact, make our drugs look even better than they are, because of that placebo effect. This is a specious objection.

And when there's a standard of care that a new drug will be added to (as was the case with BiDil), then you actually do have to run it with those therapies in place, at least when you get to Phase III. The FDA (and the medical community) want to know how your drug is going to perform in the real world, and if patients out in that real world are taking other medications, well, you can't pretend that they aren't.

In another section, the article makes much of the Merck/Elsevier affair, where Elsevier's "Excerpta Medica" division set up some not-really-journals in Australia (blogged about here). That was, in fact, disgraceful (as I said at the time), but disgraceful apparently isn't enough:

. . .Elsevier, the Dutch publisher of both The Lancet and Gray’s Anatomy, sullied its pristine reputation by publishing an entire sham medical journal devoted solely to promoting Merck products. Elsevier publishes 2,000 scientific journals and 20,000 book-length works, but its Australasian Journal of Bone and Joint Medicine, which looks just like a medical journal, and was described as such, was not a peer-reviewed medical journal but rather a collection of reprinted articles that Merck paid Elsevier to publish. At least some of the articles were ghostwritten, and all lavished unalloyed praise on Merck drugs, such as its troubled painkiller Vioxx. There was no disclosure of Merck’s sponsorship. Librarian and analyst Jonathan Rochkind found five similar mock journals, also paid for by Merck and touted as genuine. The ersatz journals are still being printed and circulated, according to Rochkind, and 50 more Elsevier journals appear to be Big Pharma advertisements passed off as medical publications. Rochkind’s forensic librarianship has exposed the all-but-inaccessible queen of medical publishing as a high-priced call girl.

Fifty journals? Really? As far as I can tell, that figure comes from this analysis at the time, and seems to be mostly nonce publications, one-off conference proceedings, and the like. There is a whole list of "Australasian Journal of So-and-Sos", which would be the same reprint advertorials as the other Excerpta Medica stuff, but do these still exist? (Did all of them on the list, in fact, ever actually publish anything?)

You'd get the impression that Elsevier is (or was, until Big Pharma came along) an absolute shining pinnacle of the medical establishment - but, with apologies to the people I know who work there, that is unfortunately not the case. They're big, and they're very far from the worst scientific publishers out there, but some of their titles are, in fact, not adding much to the total of human knowledge. Nor has the conduct of their marketing department always been above reproach. But no, this has to be built up to look even worse than it is.

The irritating thing is that there's plenty to criticize about this industry without misrepresenting reality. But does that sell?

Comments (10) + TrackBacks (0) | Category: Press Coverage | The Dark Side | The Scientific Literature | Why Everyone Loves Us

Murine Viruses and Chronic Fatigue: Does the Story Continue

Email This Entry

Posted by Derek

Well, one day after writing an obit for the XMRV story comes this abstract from Retrovirology. The authors, from Cornell and SUNY-Buffalo, say that they've detected other murine retrovirus transcripts from CFS patients (but not in most controls), and that these are more similar to those reported in last year's Lo and Alter paper in PNAS than they are to XMRV itself.

So perhaps the story continues, and what a mess it is at this point. I continue to think that the XMRV hypothesis itself is in serious trouble, but murine retroviruses as a class are still worth following up on. This is tough work, though, because of the twin problems of detection and contamination, and it's going to be easy for people to fool themselves.

Meanwhile, Retraction Watch has more on Science's "Expression of Concern" that I wrote about yesterday. It appears that the journal asked the authors to retract the paper (so says the Wall Street Journal, anyway) but that co-author Judy Mikovits turned them down (as might have been expected from her previous stands in this area). Science released their editorial note early because of the WSJ piece.

Comments (15) + TrackBacks (0) | Category: Infectious Diseases | The Scientific Literature

June 6, 2011

XMRV and Chronic Fatigue: Down For More Than the Third Time

Email This Entry

Posted by Derek

I meant to blog on this late last week, but (in case you haven't seen it) the whole putative link between XMRV and chronic fatigue syndrome seems now to be falling apart. If you want to see the whole saga via my blog posts and the links in them, then here you go: October 2009 - January 2010 - February 2010 - July 2010 - January 2011. At that last check-in, the whole thing was looking more like an artifact.

And now Science is out with a paper that strongly suggests that the entire XMRV virus is an artifact. It looks like something that's produced by the combination of two proviruses during passaging of the cells where it was detected, and the paper suggests that other human-positive samples are the result of contamination. Another paper is (again) unable to replicate detection of XMRV in dozens of samples which had previously been reported as positive, and finds some low levels of murine virus sequences in commercial reagents, which also fits with the contamination hypothesis.

With these results in print, Science has attached an "Editorial Expression of Concern" to the original 2009 XMRV/CFS paper, which touched off this whole controversy. My take: while there are still some studies ongoing, at this point it's going to take a really miraculous result to bring this hypothesis back to life. It certainly looks dead from here.

There will be also be some people who ask whether Science did the world a favor by publishing the original paper in the first place. But on balance, I'd rather have things like this get published than not, although in hindsight it's always easy to say that more experiments should have been done. The same applies to the arsenic-bacteria paper, another one of Science's recent bombshells. I'm not believing that one, either, at this point - not until I see a lot more supporting data - but in the end, I'm not sad that it was published, either. I think we're better off erring a bit on the wild-ideas end of the scale than clamping down too hard. That said, you do have to wonder if Science in particular is pushing things a bit too hard, itself. While I think that these ideas deserve a hearing, it doesn't necessarily have to be there.

Comments (18) + TrackBacks (0) | Category: Infectious Diseases | The Scientific Literature

June 2, 2011

Biomarkers, Revisited. Unfortunately.

Email This Entry

Posted by Derek

Your genome - destiny, right? That's what some of us thought - every disease was going to have one or more associated genes, those genes would code for new drug targets, and we'd all have a great time picking them off one by one. It didn't work out that way, of course, but there are still all these papers out there in the literature, linking Gene A with the chances of getting Disease B. So how much are those worth?

While we're at it, everyone also wanted (and still wants) biomarkers of all kinds. Not just genes, but protein and metabolite levels in the blood or other tissue to predict disease risk or progression. I can't begin to estimate how much work has been going into biomarker research in this business - a good biomarker can clarify your clinical trial design, regulatory picture, and eventual marketing enormously - if you can find one. Plenty of them have been reported in the literature. How much are those worth, too?

Not a whole heck of a lot, honestly, according to a new paper in JAMA by John Ioannidis and Orestes Panagiotou. They looked at the disease marker highlights from the last 20 years or so, the 35 papers that had been cited at least 400 times. How good do the biomarkers in those papers have to be to be useful? An increase of 35% in the chance of getting the targeted condition? Sorry - only one-fifth of the them rise to that level, when you go back and see how they've held up in the real world.

Subsequent studies, in fact, very rarely show anything as strong as the original results - 29 of the 35 biomarkers show a less robust association after meta-analysis of all the follow-up reports, as compared to what was claimed at first. And those later studies tend to be larger and more powered - in only 3 cases was the highly cited study the largest one that had been run, and only twice did the largest study show a higher effect measure than the original highly cited one. Only 15 of the 35 biomarkers were nominally statistically significant in the largest studies of them.

Ioannidis has been hitting the literature's unreliability for some time now, and I think that it's hard to dispute his points. The first thought that any scientist should have when an interesting result is reported is "Great! Wonder if it's true?" There are a lot of reasons for things not to be (see that earlier post for a discussion of them), and we need to be aware of how often they operate.

Comments (25) + TrackBacks (0) | Category: Biological News | The Scientific Literature

May 17, 2011

Quis Custodiet Ipso Custodes?

Email This Entry

Posted by Derek

Yesterday's look into the Google Ngram data set brought up a discussion in the comments on how good the numbers are in it (and in other large datasets). "Garbage in, garbage out" is as true a statement as ever, so it's a real worry. (Even if the data were perfect, the numbers could still be misused and misinterpreted, of course).

An e-mail from a reader pointed me to another example of this sort of thing. The NIH Chemical Genomics Center (NCGC) has a collection of known pharmaceutically active compounds for use in screening and target ID. This is a good idea, and the same sort of thing is done internally in the drug industry. But the ChemConnector blog has some questions about how robust the dataset is. The rough estimate is that between 5 and 10% of the 7600+ structures are messed up in some way (stereochemistry, salt form, the dreaded pentavalent carbon, and so on).

Read the comments there for some interesting back-and-forthing with the NIH people. The NCGC folks realize that they have some problems, and are willing to put in the work to help clean things up. The problem is, they'd already published on this list, calling it "definitive, complete, and nonredundant", which now seems to be a bit premature. . .

Comments (11) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

May 16, 2011

A Google Oddity

Email This Entry

Posted by Derek

A comment to the last post mentioned that if you search the word "biotechnology" in Google's Ngram search engine, something odd happens. There's the expected rise in the 1970s and 80s, but there's also a bump in the early 1900s, for no apparent reason. Curious about this, I ran several other high-tech phrases through and found the exact same effect.

Here's a good example, with some modern physics phrases. And you get the same thing if you search "nanotechnology", "ribosome", "atomic force microscope", "RNA interference", "laser", "gene transfer", "mass spectrometer" or "nuclear magnetic resonance". There's always a jump back in exactly the same period on the early 1900s.
So what's going on? I can understand some OCR errors, but why do these things show up in this specific Edwardian-age window? Can anyone at Google shed any light on this?

Comments (28) + TrackBacks (0) | Category: General Scientific News | The Scientific Literature

Ups and Downs

Email This Entry

Posted by Derek

I was thinking the other day that I never remembered hearing the phrase "Big Pharma" when I first got a job in this business (1989). Now I have some empirical proof, thanks to the Google Labs Ngram Viewer, that the phrase has only come into prominence more recently. (Fair warning: you can waste substantial amounts of time messing with this site). Here's the incidence rate of "big pharma" in English-language books from 1988 to 2000.big%20pharma%20graph%2Cjpg.jpg
It comes from nowhere, blips to life in 1992, doesn't even really get off the baseline until 1994 or so, and then takes off. (The drops in 2005 and 2008 remain unexplained - did the log phase of its growth end in 2004?)

Update: that graph holds for the uncapitalized version of the phrase. If you put the words in caps, you get the even more dramatic takeoff shown below:

To be fair, though, there seems to have been a general rise in Big Pharma-related literature during that period. Try out this graph, comparing mentions of Merck, Pfizer, and Novartis since 1970. The last-named, of course, didn't even exist until the early 1990s, but they (like the others) have spent the time since then zipping right up, with no apparent end in sight. (Merck, especially - what's with those guys?) And what accounts for this? Business books? Investing guides? Speculation is welcome.

Note: the above paragraph was written before realizing that the Google Ngram search is case-sensitive - so, as was pointed out in the comments, I was picking up on people not caring about capitalization more than anything else. Below is the correct graph, with initial capitals in the search, and it makes more sense. Merck still is the king of book mentions, though, for all the coverage that Pfizer gets.

I'll finish off with this one, using a longer time scale. Yes, folks, for better or worse, it appears that the phrase "organic chemistry" peaked out between book covers around 1950, and has been declining ever since. Meanwhile, "total synthesis" starting rising during the World War II era (penicillin?), and kept on moving up until a peak around 1980. Interestingly, things turned around in 2000 or so, and especially since 2003. And this can't be ascribed to some sort of general surge in chemistry publications - look at the "organic chemistry" line during the same period. Is there some other field that's adopted the phrase?

Comments (20) + TrackBacks (0) | Category: Drug Industry History | General Scientific News | The Scientific Literature

May 9, 2011

What Medicinal Chemists Really Make

Email This Entry

Posted by Derek

Chemists who don't (or don't yet) work in drug discovery often wonder just what sort of chemistry we do over here. There are a lot of jokes about methyl-ethyl-butyl-futile, which have a bit of an edge to them for people just coming out of a big-deal total synthesis group in academia. They wonder if they're really setting themselves up for a yawn-inducing lab career of Suzuki couplings and amide formation, gradually becoming leery of anything that takes more than three steps to make.

Well, now there's some hard data on that topic. The authors took the combined publication output from their company, Pfizer, and GSK, as published in the Journal of Medicinal Chemistry, Bioorganic Med Chem Letters and Bioorganic and Medicinal Chemistry, starting in 2008. And they analyzed this set for what kinds of reactions were used, how long the synthetic routes were, and what kinds of compounds were produced. Their motivation?

. . .discussions with other chemists have revealed that many of our drug discovery colleagues outside the synthetic community perceive our syntheses to consist of typically six steps, predominantly composed of amine deprotections to facilitate amide formation reactions and Suzuki couplings to produce biaryl derivatives. These “typical” syntheses invariably result in large, flat, achiral derivatives destined for screening cascades. We believed these statements to be misconceptions, or at the very least exaggerations, but noted there was little if any hard evidence in the literature to support our case.

Six steps? You must really want those compounds, eh? At any rate, their data set ended up with about 7300 reactions and about 3600 compounds. And some clear trends showed up. For example, nearly half the reactions involved forming carbon-heteroatom bonds, with half of those (22% of the total) being acylations. mostly amide formation. But only about one tenth of the reactions were C-C bond-forming steps (40% of those were Suzuki-style couplings and 18% were Sonogoshira reactions). One-fifth were protecting group manipulations (almost entirely on COOH and amine groups), and eight per cent were heterocycle formation, and everything else was well down into the single digits.

There are some interesting trends in those other reactions, though. Reduction reactions are much more common than oxidations - the frequency of nitro-to-amine reductions is one factor behind that, followed by other groups down to amines (few of these are typically run in the other direction). Among those oxidations, alcohol-to-aldehyde is the favorite. Outside of changes in reduction state, alcohol-to-halide is the single most favorite functional group transformation, followed by acid to acid chloride, both of which make sense from their reactivity in later steps.

Overall, the single biggest reaction is. . .N-acylation to an amide. So that part of the stereotype is true. At the bottom of the list, with only one reaction apiece, were N-alkylation of an aniline, benzylic/allylic oxidation, and alkene oxidation. Sulfonation, nitration, and the Heck reaction were just barely represented as well.

Analyzing the compounds instead of the reactions, they found that 99% of the compounds contained at least one aromatic ring (with almost 40% showing an aryl-aryl linkage) and over half have an amide, which totals aren't going to do much to dispel the stereotypes, either. The most popular heteroaromatic ring is pyridine, followed by pyrimidine and then the most popular of the five-membered ones, pyrazole. 43% have an aliphatic amine, which I can well believe (in fact, I'm surprised that it's not even higher). Most of those are tertiary amines, and the most-represented of those are pyrrolidines, followed closely by piperazines.

In other functionality, about a third of the compounds have at least one fluorine atom in them, and 30% have an aryl chloride. In contrast to the amides, there are only about 10% of the compounds with sulfonamides. 35% have an aryl ether (mostly methoxy), 10% have an aliphatic alcohol (versus only 5% with a phenol). The least-represented functional groups (of the ones that show up at all!) are carbonate, sulfoxide, alkyl chloride, and aryl nitro, followed by amidines and thiols. There's not a single alkyl bromide or aliphatic nitro in the bunch.

The last part of the paper looks at synthetic complexity. About 3000 of the compounds were part of traceable synthetic schemes, and most of these were 3 and 4 steps long. (The distribution has a pretty long tail, though, going out past 10 steps). Molecular weights tend to peak at between 350 and 550, and clogP peaks at around 3.5 to 5. These all sound pretty plausible to me.

Now that we've got a reasonable med-chem snapshot, though, what does it tell us? I'm going to use a whole different post to go into that, but I think that my take-away was that, for the most part, we have a pretty accurate mental picture of the sorts of compounds we make. But is that a good picture, or not?

Comments (24) + TrackBacks (0) | Category: Chemical News | Drug Development | Life in the Drug Labs | The Scientific Literature

May 6, 2011

The Top 100 Chemists?

Email This Entry

Posted by Derek

I'm never sure of how useful these rankings are, but here's Thomson Reuters' rankings of the top 100 chemists of the last ten years. This is based on publications and their impact/citation rate.

Looking over the list, I think that there are some artifacts in it, and boy, don't metal-organic frameworks and nanotech just rate like crazy? But it's an interesting starting point for discussion, especially when you note how (relatively) few organic and synthetic organic chemists make the upper reaches. Thoughts?

Comments (31) + TrackBacks (0) | Category: The Scientific Literature

May 2, 2011

Great Moments in Journal Editing

Email This Entry

Posted by Derek

I don't know if this DOI link resolves yet - or if the problem will be fixed by the time it does. But for now, in the "Articles in Press" queue over at Drug Discovery Today, they have one whose title reads like this:

Utility of protein structures in overcoming ADMET-related issues of drug-like compounds[1. AU: You use ADMET-relevant throughout manuscript. Would you like to change this to ADMET-relevant too?]

Well, would you? They're waiting for someone to answer them, apparently.

Comments (10) + TrackBacks (0) | Category: The Scientific Literature

April 29, 2011

Keep On Scrollin'

Email This Entry

Posted by Derek

I've been browsing through my journal RSS feeds, and a question occurs to me. When you're scanning through the current literature, what sort of paper makes you most likely to keep scrolling? What kind of work are you least likely to actually read?

We'll stipulate that you're looking at a journal or subject that's relevant to your field - I read a lot of stuff, but I'm most certainly not going to slow down to look over (say) a theoretical paper calculating the stability of isomeric inorganic complexes. But that said, there are things that make you slow down while going through the abstracts, and there are things that absolutely made you speed up.

My particular biases are to walk more quickly past the following sorts of titles, which have been only slightly exaggerated for effect. And you?

"Synthesis of A Natural Product That You Don't Care About, Using Methods That Bore You"

"Slight Enantiomeric Excess Realized Through Use Of A Humungous Catalyst That Takes Nine Steps To Make All By Itself"

"Nanorods Attached to Nanoplates by Nanosprings: Progress Toward a Nanomattress"

"Green Chemistry, Part 87: A Novel Reagent to Prepare Nitriles from Oximes"

"Isolation Of Known Terpenoid Natural Products From Weeds In Our Back Yard"

Comments (46) + TrackBacks (0) | Category: The Scientific Literature

April 28, 2011

Just A Few More Month's Work, That's All I'm Asking Here

Email This Entry

Posted by Derek

Here's the cry of someone who's been jerked around by too many journal referee reports. Hidde Ploegh of the Whitehead Institute has a piece in Nature News called "End the Wasteful Tyranny of Reviewer Experiments". That could have just possibly have been phrased more diplomatically, but I know what he's talking about.

Too often, reviewers try to show that they're fulfilling their responsibilities by requesting additional work from the authors of a paper under consideration. This happens more and more as you move up the hierarchy of journals, as both the novelty of the work and the incentive to publish it increase. No one's going to exert themselves too much to get their paper into Acta Retracta, even if some rogue reviewer were to try it, but Science and Nature (among others) can really make you perform tricks.

What this reminds me of is a story about Steve Wozniak, of Apple fame. When he was in college, his dorm had an old TV down in the lobby with a rabbit-ear antenna, which had to be messed with constantly to get a good picture. Woz apparently built a gizmo to fuzz out the reception, and used to sit inconspicuously in the back of the room, trying to see what sort of crazy positions he could twist people into as they held the antenna in what was seemingly the One Perfect Spot.

The referee equivalent is Just One More Experiment, and it's not always justified:

Submit a biomedical-research paper to Nature or other high-profile journals, and a common recommendation often comes back from referees: perform additional experiments. Although such extra work can provide important support for the results being presented, all too frequently it represents instead an entirely new phase of the project, or does not extend the reach of what is reported. It is often expensive and unnecessary, and slows the pace of research to a crawl. Among scientists in my field, there is growing concern that escalating demands by reviewers for the top journals, combined with the increasingly managerial role assigned to editors, now represents a serious flaw in the process of peer review.

Ploegh's point is that too many referees aren't reviewing the paper that they have; they're suggesting a whole new project or phase of research. And some of these wouldn't even affect the results and conclusions of the paper under review very much - they're just "Gosh, wouldn't it be nice if you would also. . ." experiments. The benefit for science, he says, is nowhere near commensurate with the disadvantage of holding up publication, messing with the career prospects of younger investigators, spending extra time and grant money, and so on. His suggestion?

The scientific community should rethink how manuscripts are reviewed. Referees should be instructed to assess the work in front of them, not what they think should be the next phase of the project. They should provide unimpeachable arguments that, where appropriate, demonstrate the study's lack of novelty or probable impact, or that lay bare flawed logic or unwarranted conclusions.

He also suggests that reviewers provide an estimate of the time and cost involved for their suggested experiments, and compare that to their purported benefits. I wouldn't mind seeing editors crack down on this some, either. I've had useful feedback with my own manuscripts, which had identified things that really did need to be shored up. But submitting a paper should not routinely be an exercise in having other people tell you what experiments you should run before you can publish your. When there really is a gap or flaw, naturally, it's appropriate to ask for more, but I agree with Ploegh that a reviewer needs to make a case for such things, rather than just asking for them as a matter of routine.

Ploegh has a larger historical point to make as well. Looking back at the earlier days of , say, molecular biology, you get the impression that if someone sent in an interesting paper that seemed reasonable, it would just get published, without all these trips back to the bench. Somehow, the mechanics of science (and especially scientific publication) have changed. Has it been for the better? Or would we all be better off letting more things through as they stand, if they're clearly presented and logically consistent?

I wonder if journals might consider publishing in this style, while then adding an editorial note about what further experiments had been suggested by reviewers. This would fulfill the function of pointing out potential weak points or areas for further exploration, but without delaying things so much. I don't see this happening - but why not, exactly?

Comments (17) + TrackBacks (0) | Category: The Scientific Literature

April 26, 2011

Graphical Crankiness

Email This Entry

Posted by Derek

And now a brief note from the "trivial but annoying" department, since it's been a couple of years since I last complained about this. Is there any way that we can start a petition, or take up a collection, or do something to make K. C. Nicolaou stop drawing ring systems like this? Coloring the insides of them with gradient fills adds no information and actually obscures elements of the structure.

If all else fails, can we at least send the man a set of Ed Tufte books?

Comments (67) + TrackBacks (0) | Category: The Scientific Literature

So Much For Natural-Product Nevirapine

Email This Entry

Posted by Derek

Remember that weird Tetrahedron paper from last December? The one that claimed that it isolated the reverse transcriptase inhibitor nevirapine as a natural product from an Indian plant? In chiral form, no less?

Well, the journal would now like to say "Never mind". The lead author has retracted the paper, "due to doubt created in the scientific community on the origin of nevirapine from the seeds of Cleome viscosa". That's an odd way to put it. Isn't it? Doubts that other people might have are irrelevant if you're right, aren't they?

No, this is similar to the classic weasel-word apology, the one that goes on about regretting the way that some people took offense rather than regretting the original action itself. The reason this paper was retracted, surely, was that those doubts in the scientific community were well-founded. This paper made no sense on several levels, and those problems should have been caught immediately. Its publication was an embarrassment for Tetrahedron and for Elsevier.

I think that the reason I get so worked up about these things is the laziness and sloppy thinking involved. Scientific research deserves more than that, and the rest of us deserve more from the people who publish it.

Comments (5) + TrackBacks (0) | Category: The Scientific Literature

April 5, 2011

In Which I Reminisce About the Prins Reaction, Chemical Abstracts, and John Keats

Email This Entry

Posted by Derek

Well, this post needs updating. In it I mentioned never running a Prins reaction again since the 1980s, nor any photochemistry, and today what do I find myself doing? Both of them, although not at the same time.

I am, fortunately, not running the Prins this way. But even bringing it up at all recalls to me a key part of my education. When I first joined my graduate school research group, I was put to making some tetrahydropyran systems. I was handed a synthesis, drawn up before my arrival, of how to make the first one, and like most first-year grad students, I gamely dug and and started to work on it.

I should have devoted a bit more thought to it. I won't go into the details, but it was a steppy route that relied, in the final ring-closure step, on getting the cyclic ether to form where one of the partners was a neopentyl center. The organic chemists in the audience will immediately be able to guess just how well that went.

So I beat on it and whacked at it, getting nowhere as I used up my starting material, until I was finally driven to the library. In the spring of 1984, that was a different exercise than it is now, involving the 5-year Chemical Abstracts indices and an awful lot of page flipping. (I haven't so much as touched a bound volume of CA in I don't know how many years now). If you were a nomenclature whiz, you could try looking up your compound, or something like it, in the name index, but a higher-percentage move was often to look up the empirical formula. That gave you a better shot, because (if it was there at all) you could see how CA named your system and work from there.

To my great surprise, the second set of collective indices I checked (the good ol' 9th), yielded a direct hit on an empirical formula, and the name looked like exactly what I had been trying to make. The reference was in Tetrahedron, which we most certainly had on the shelf, and I zipped over to see if there was any detail on how to make the stuff.

There was indeed. A one-stepper Prins cyclization gave just the ring system I'd been trying to make, and that was one step from the intermediate I needed. I just stared at the page, though. I honestly couldn't believe that this was real (as I mentioned, I was in about my second month of grad school lab work). Surely the synthesis I'd been given was the way to make this stuff? Surely the people responsible for it had checked the literature before drawing it up? (After all, it had only taken my a few minutes to find the stuff myself). Surely I couldn't just make the ring in one afternoon using two starting materials I could buy cheaply from Aldrich?

Well, surely I could. And that's just what I did, and got my project moving along until the next interesting difficulty came up a couple of months later. But I still recall standing there in the Duke chemistry library, looking at that journal article "with a wild surmise" that perhaps I should check things out for myself next time instead of just taking everyone else's word. It took a couple more lessons for me to really grasp that principle (Nullius in verba!, but it's helped me out a great deal over the years. I have the 27-year-old photocopy I made that afternoon in front of me now. It's a good reminder.

Comments (15) + TrackBacks (0) | Category: Graduate School | Life in the Drug Labs | The Scientific Literature

March 2, 2011

Now, That's An Abstract

Email This Entry

Posted by Derek

A reader who's attending the International Congress on Heterocyclic Chemistry in Glasgow later this year sent me a note about it. Like many such meetings, they have guidelines for presentation and poster abstracts. But this one was done by someone who's been around the block a few times.

The sample abstract is from a team from the University of Utopia (and in case you're wondering, ".ut" is apparently the internet domain for Utopia). And the authors, in a nice touch, are Black, Schwartz, Nero, Fekete, and Čzerný. (Too bad the other students in the group - Siyah, Dubh, and Musta - couldn't make it onto the list). But here's the text of the thing itself:

Two fundamentally different but complementary transition metal catalyzed chemo-, regio-,diastereo-, enantio-, and grantproposalo-selective approaches to the synthesis of a library of biologically significant nano- and pico-molecules will be presented with the focus on reaction mechanism and egocentric effects. The role of the nature of the metal, ligand, solvent, temperature, time, microwave, nanowave, picowave, ultrasound, hypersound, moon phase, and weather in this catalytic, sustainable, cost-effective, and eco-friendly technology will be discussed in detail.

If nothing else, that's about as grantproposalo-selective as it gets, right there. . .

Comments (30) + TrackBacks (0) | Category: The Scientific Literature

January 28, 2011

What The Referees Really Think

Email This Entry

Posted by Derek

I wish that more journals did this! Environmental Microbiology, which I have never looked at before, has published its favorite reviewer comments from the year just passed. They're not tied to the papers that generated them, naturally, but then, many of these manuscripts didn't quite make the cut:

"The biggest problem with this manuscript, which has nearly sucked the will to live out of me, is its terrible writing style."

"I usually try to be nice, but this paper has got to be one of the worst I have read in a long time."

"I suppose that I should be happy that I don't have to spend a lot of time reviewing this dreadful paper, however, I am depressed that people are performing such bad science."

"It is sad to see so much enthusiasm and effort go into analyzing a dataset that is just not big enough."

There are plenty more, including many from people who are actually happy about what they had to read (and yes, there are some). Check 'em out!

Comments (13) + TrackBacks (0) | Category: The Scientific Literature

January 18, 2011

Retractions: Why The Secrecy?

Email This Entry

Posted by Derek

Ben Goldacre has an excellent point here at Bad Science: when a paper gets retracted from a journal, shouldn't everyone know why it's been retracted?

He highlights the experience of the blog Retraction Watch (which I hadn't heard of until now), when they tried to find out why a paper had been pulled from the Annals of Thoracic Surgery. The journal's editor responded to their query by informing them that "it's none of your damn business".

Gotta disagree there, chief. I think that this is actually important information, and that it should be disclosed as much as possible. There are all sorts of reasons for papers to be retracted, ranging from benign to evil, and it's in the interest of readers to know what category things have fallen into. I understand that in some cases papers are the subject of ongoing investigations, so these details aren't always available, but in that case, why not say something like: "The data in Table II have not been reliably reproduced by other workers. While some of the co-authors of the original work have stated that they stand by the results as published, an investigation has begun into the methods and data of this paper, and the lead authors have asked that it be retracted until this matter is concluded".

But that's not the sort of thing we get. Goldacre cites another example from Retraction Watch, concerning this paper from JACS. When the bloggers contacted the lead author, he gave them more details than you could get from the journal about what was wrong with the paper. So why doesn't JACS tell us these things?

Thanks to the Retraction Watch people for taking the time and effort to do this sort of thing. I just wish that it weren't necessary for anyone to do it at all.

Comments (20) + TrackBacks (0) | Category: The Scientific Literature

January 11, 2011

The Life of a Paper

Email This Entry

Posted by Derek

Angewandte Chemie recently ran a behind-the-scenes article about their journal, with several interesting bits of information. For one thing, they've gotten a lot more selective over the years, as the number of submissions has gone up. They publish many more papers, total, than they used to, but reject a much higher fraction at the same time. (I've added to that total myself a couple of times!).

Mind you, there are times when that rejection rate should have been even a bit higher, but as you might guess, the article doesn't bring up those awkward moments. There's no insight into the vile puns and other pop-culture references that continue to infest their abstracts, either. Can't have everything.

But I found this chart interesting. These are the download statistics for a particular (unspecified) communication in the journal over time. (Note that they've scrubbed the units on the Y-axis, the wimps).
This confirms what most scientists have figured, that your paper has a brief window to be noticed, and then back in the pile it goes. Back to the background rate, with people coming across it in literature searches once in a while.

Comments (9) + TrackBacks (0) | Category: The Scientific Literature

December 20, 2010

Putting Some Numbers on Peer Review

Email This Entry

Posted by Derek

Since we've been talking about peer review on and off around here, this paper in PLoS One is timely. The authors are putting some numbers on a problem that journal editors have long had to deal with: widely varying reviews from different referees for the same exact paper.

It's a meta-analysis of 52 studies of the problem reported over the last few decades. It confirms that yes, inter-reviewer reliability is low. The studies that report otherwise turn out to have smaller sample sizes and other signs of lower reliability. The question now is: to what extent is this a problem?

One of the studies they quote maintains that too high a level of agreement would also be the sign of a problem (that some of the reviewers are redundant, and that the pool of referees might have been poorly chosen). I'm willing to think that total agreement is probably not a good thing, and that total disagreement is also trouble. So what level of gentlemanly disagreement is optimal? And are most journals above it or below?

FIguring that out won't be easy. Some journals would really have to open their books for a detailed look at all the comments that come in. I assume that there are editors who look over their reviewers, looking for those that tend to be outliers in the process. (Um, there are some editors that do this, right?) But that takes us back to the same question - do you value those people for the perspective they provide, or do you wonder if they're just flakes? Without a close reading of what everyone had to say about the crop of submissions, it's hard to say. Actually, it might not be easy, even then. . .

Comments (21) + TrackBacks (0) | Category: The Scientific Literature

December 15, 2010

What A Paper Doesn't Have In It

Email This Entry

Posted by Derek

So, should I apologize for the tone of yesterday's post? I'm not ready to yet, and I'll explain why.

Here's a general rule: if you find yourself having to make excuses for a scientific paper, for the key experiments that weren't done and the rationales that don't appear, then something is wrong. A paper shouldn't make you have to assume that the authors will get around to running Experiment X and Comparison Y and Test Z, and put that in their next manuscript. (If this makes you think of the recent arsenic bacteria controversy, you're right on target). Anything like that which immediately comes to mind should be in the first manuscript. If the authors haven't put it in there, then it's the job of the referees to tell them to go back and do it. And whoever refereed this paper did an incompetent job.

Now, a quick technical note. It's true that medium-sized rings can form isolable atropisomers in some cases. (An atropisomer, for those who aren't chemists - or chirality geeks - is a compound that can exist as two mirror-image forms just through some sort of constrained movement of its parts. A putatively flat compound that comes around and wraps over itself like a screw thread is one example, as is one with a bond that should be freely rotating but it blocked by large side chains from doing so).

Problem is, this compound sure doesn't look like one of those cases. It has no bulky groups that have trouble getting out of each other's way, and it has no helical chirality. The only thing it has, to my eyes, is a nitrogen that could only lead to new isomers if its barrier to inversion were really, really high - and it shouldn't be. For nevirapine, there really should be no way to isolate such an isomer at room temperature. If anyone can provide evidence for isolable atropisomers in a system as small and lightly substituted as this one, I'll certainly consider eating my words - but not until then.

But that brings up a larger point. This is actually one of the things that makes me think this paper is bogus: the presence of such a compound really would be the big selling point of the manuscript, if the authors had thought about it. Instead, their main focus is on how the structure turns out to be a natural product (which I have trouble believing, too),and not on the bizarre nature of it being chiral. The chirality, frankly, seems to be an afterthought, the way things are written. The word "atropisomer" does not make an appearance. References to the other dibenzo ring systems that have shown this interesting phenomenon (which were worth papers all their own) are not cited. There is no mention of a nitrogen inversion. (If that were the explanation, you'd also expect that heating up the sample would eventually start flipping the molecules past that barrier and removing the optical activity - but there's no mention of any such thing). You'd never know that there was something interesting going on, because the authors give us no reason to believe that they knew that, either.

It would also be quite interesting, if you could have such a thing as an optical isomer of nevirapine, to see what its activity would be on its enzyme target, reverse transcriptase. And if nevirapine could exist as enantiomers, how about running the synthetic material down some chiral columns to see if you could resolve it? Then show that your new optically active stuff is only one of those peaks; that would be pretty convincing. Not done, not done, not even mentioned, etc.

Here's another consideration: as mentioned yesterday, I don't think that this structure has been given very good characterization (the X-ray data seem insufficient to talk about chirality). One of the comments to yesterday's post wondered why the authors didn't show HPLC traces with and without a spike of the authentic drug material. That's an excellent idea, and it's something that would be worth showing in an NMR spectrum, too. Claiming that you found nevirapine in a plant is quite weird - you'd want to really hammer down the fact that everything is identical. But these experiments haven't been done, either.

Allow me to mention one more oddity. The authors actually make reference to "optically active nevirapine" (their footnote #10), but the reference they cite (the original paper from Boehringer Ingleheim) does not, as far as I can see, mention any such thing. And that's because no one has ever mentioned any such thing, and that's because I don't think it exists, outside of (just maybe) a low-temperature NMR experiment.

No, I'm still not buying this. I'm upset with the authors for having proposed such a thing with such thin evidence, but I'm really more upset with the editors of what is supposed to be a reputable journal for publishing it.

Comments (27) + TrackBacks (0) | Category: The Scientific Literature

Chiral What? Chiral How?

Email This Entry

Posted by Derek

Thanks to an email from a reader, I can bring you this very weird paper from Tetrahedron. The authors claim to have extracted a local plant and isolated nevirapine, (sold as Viramune by Boehringer Ingleheim as a reverse transcriptase inhibitor for HIV).
That's kind of odd. I'm no natural products expert, but I've sure seen a lot of them over the years, and that framework (and the N-cyclopropyl) don't look so likely to me. But hey, plants do odd things. That's not what's really puzzling about this paper. No, what's had me staring at it this morning is the claim that, in contrast to the marketed drug, this stuff is optically active nevirapine.

Say what? Try as I might, I can't see any plausible way that that's a chiral compound. The authors seem to think it is, though. They claim optical rotation, somehow, and then say that "The detailed structure and stereochemistry of compound 1 was established unambiguously by single crystal X-ray crystallography." But hold on - that's not as easy as it sounds. Getting absolute configurations from the X-ray data of light-atom-only molecules takes special efforts, and I don't see any being taken (molybdenum X-rays, direct methods, no talk of anomalous dispersion, etc.)

I'm just not willing to see that nitrogen atom as a source of chirality - if it were, shouldn't that be the focus of this whole paper? Instead, the authors just blithely tell us how neat it is that they've isolated the chiral material. In fact, they find it so neat that they tell us two times in a row:

This is a very interesting discovery that naturally occurring optically active nevirapine has been biosynthesized in the seeds of C.viscosa and the optically inactive nevirapine was designed as a selective non-nucleoside inhibitor of HIV-1 reverse transcriptase. It is also a remarkable finding that the seed of C.viscosa is the source of optically active nevirapine, which was also designed and synthesized before its isolation from natural source.

This sounds like some sort of lunatic patent-busting exercise, to be honest. And it sounds as if someone doesn't know what a chiral compound is. And that whoever reviewed this for Tetrahedron was incompetent. And that the editor who let it through should be a least a little bit ashamed. Well?

Comments (73) + TrackBacks (0) | Category: Infectious Diseases | Natural Products | The Scientific Literature

December 14, 2010

The Solid Phase

Email This Entry

Posted by Derek

Looking over the chemical literature with an RSS reader can really give you a sense of what the hot topics are, and what's cooling off. Remember when it seemed as if every third paper was about ionic liquids? You still see work in the area, but it's nowhere near as crazy as it was. (I had a colleague come by my office the other day and ask "Did anyone ever find out what to do with those things?") Similarly, gold catalysts have been all over the place in recent years, but seem, to my eye, to be calming down.

Some of these things are research areas that look promising, but die off when their limits become apparent. Some of them are almost sheer fads, with papers coming out from all sorts of odd places because the authors want to get in on the hot, publishable topics while they can. Others keep going because the topics themselves are important but ver hard to exhaust (metal-catalyzed couplings come to mind).

And there are areas that keep going in the literature because they look like they should be important and useful, and eventually will, but no one can quite get them to either work generally enough or get people to recognize that they do. The metal-catalyzed coupling literature was in this shape back in the 1970s and into the 1980s - there were a lot of disparate reactions that you could do with palladium, but none of them had exactly taken over the world. My vote for a current field in this protostar state is engineered solid-phase catalysis.

That may sound odd, since work on solid-phase catalysts has been going on for decades, and is of huge industrial importance. But many of the important catalysts have been arrived at either by luck or by an awful lot of hard slogging. The field is complicated enough - fiendishly so - that it's hard to draw general conclusions. If you have a good solution-phase catalyst, how do you make a solid-supported variety that works just as efficiently? Well. . .if you really want one, you make about a zillion variants and hope for the best, as far as I can see.

Part of the problem (as with the metal-catalyzed coupling world) is that there are just so many variables. The solid supports alone are enough to keep a person occupied for life, what with all the various aluminas, silicas, zeolites, polymers, mesoporous engineered thingies, and so on. Then you have the uncountable schemes for linking these surfaces to active catalysts - what functional groups to use, what density things should be on the surface, what distance you need between the surface and the catalyst, etc. And just linking up to the known catalysts is no light work, either, since most of these things were not made with convenient handles hanging off them.

As we get better at making (and characterizing) new kinds of surfaces and new kinds of macromolecular assemblies, we might start to get our hands around this subject. For now, though, it seems to be mostly in the descriptive stage: papers are of the "Hey, we made this thing and here's what it does" variety, with further work in the series being "Hey, remember that stuff we made? Turns out you can do this with it, too - who knew?" What you don't see, or not too darn often, is a paper describing the general principles of these processes. For the most part, we don't know them yet.

But if I had to pick an area that will eventually blossom into a host of applications, this would be high on the list. It's a mixture of surface chemistry, materials science, nanotechnology, and organic synthesis, and it's got a lot of promise. But then again, it's had a lot of promise for a long time now. . .

Comments (9) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

December 9, 2010

So What's Going On With Peer Review, Anyway?

Email This Entry

Posted by Derek

I have a larger comment, sparked by the controversy over the NASA-arsenic-bacteria paper in Science. But it's not just about that one. It's about the "reactome" paper (also in Science and now retracted), the hexacyclinol synthesis published in Ang. Chem., and others. There have been, I think it's fair to say, a number of very arguable papers published in very high-profile journals in recent years. What's going on?

I want to make it clear that I'm not upset about journals published "out-there" work. In fact, I wish that there were a bit more of it. But at the same time, if you're going to go out there on the edge, you'd better have some solid stuff to report when you come back and write up the paper. Extraordinary claims really do require extraordinary evidence, and that's where things seem to be breaking down.

Peer review is supposed to catch these things. That reactome paper had chemists rolling their eyes as soon as they saw the synthetic schemes in it, and asking if anyone at the journal had thought to call someone who knew organic chemistry during the review process. This latest arsenic paper has other specialists upset, for different reasons (and, to be sure, for reasons that don't require much scientific specialization at all, as detailed in my post after I'd given the paper a close reading). But that hexacyclinol paper appeared in a chemistry journal, and had (one assumes!) been reviewed by competent chemists. How, then, could it have been published to immediate howls of derision about the quality of the evidence in it?

I also want to make clear that I'm not talking about some of the other categories of bad papers, such as the things are are probably true, but of little interest to anyone. And in the probably-not-true category, lower-ranking journals let not-so-good stuff through pretty often. I've been hard on Biorganic and Medicinal Chemistry Letters here before, among other journals, for publishing things that appear to have been incompetently reviewed. But these journals aren't Science or Nature, and the whole point of prestigious journals is that the things that appear in them are supposed to be important, and they're also supposed to be thoroughly vetted.

Is it the push to land the big papers that will make a big splash? Does that cause people in the editorial offices to bend the rules a bit? The official answer from every journal editor that's ever lived to such questions has been "Of course not!", but you have to wonder. Is it a problem with how they're assigning papers for review - who they go to, or how seriously the reviews are taken when they come back? I really don't know. I just know that we seem to be seeing a lot of embarrassing stuff in the literature these days. It's not supposed to work that way.

Comments (40) + TrackBacks (0) | Category: The Scientific Literature

November 22, 2010

Spending and Publishing

Email This Entry

Posted by Derek

Since we were talking about worldwide scientific productivity here the other day, this article in The Economist is timely. They're talking about the share of worldwide R&D (and papers published) by country, and pointing out that the longtime order seems to be changing.

For sheer scientific publications,that order is, of course, the US and Western Europe, followed distantly by everyone else. I've reproduced two graphs from the article, atrocious color schemes and all, and you can see how large the gap has been in the published-paper count.. But there are several interesting features. Note how back in the early 1980s, Russia and Japan were quite similar, but the old Soviet Union (and its successor Russian state) was on the decline even then. Meanwhile, China has come up from nowhere to overtake even Japan. India, South Korea, and Brazil are down in the single digits.
But that brings up some other questions. Take a look at the second graph, on R&D spending as a % of GDP. (This is over a shorter time scale than the paper graph, so adjust your perspective accordingly). Note that Japan has been leading the way here, with South Korea catching up. Neither of them (especially South Korea) publish as much, though, as you'd think, given this investment - is the rest of it going into patents? Or staying inside the labs? Looked at another way, though, the EU is publishing even more than you'd think, given their R&D spending.
You'll see that China is coming up in the spending world, although they're not rising as steeply as South Korea (no one is). India's pretty flat, though, and are being outspent, on this basis, by Brazil. (I hope I'm reading the various shades of aquamarine, teal, and blue-green correctly - you know, the Economist used to be good at presenting information graphically, but whoever let this one through should be whacked on the head).

Neither of these measures is an end in itself. I'd say that robust R&D spending is necessary (but not sufficient) for a country to produce good results. And there are probably a lot of different ways to count things as R&D or not, which we aren't seeing here. As for publications, they're an even rougher measure, since different countries have different cultures (and incentives) for this sort of thing. (Don't forget language barriers, either). And as everyone knows, there are papers and there are papers. Long lists of junk that no one ever reads would be one way to inflate things, but to what good?

Comments (22) + TrackBacks (0) | Category: The Scientific Literature

November 17, 2010

More Fraudulent Papers Coming From the US?

Email This Entry

Posted by Derek

Update: Richard Van Noorden at Nature runs the numbers on this paper, and comes to the same conclusions: you can't use it to say that US papers are more prone to fraud. . .

There's a new paper in the Journal of Medical Ethics looking at fraudulent publications. The author went back and studied the papers in the PubMed database that have been retracted during the last ten years (there are 788 of them), looking for trends. And so far, the press coverage that I've seen of his conclusions seems to be missing the point.

One thing that stands out is that retracted papers tend to come from higher-profile journals. That makes sense to me, because that's (a) where they're more likely to be noticed, and (b) where the editors are more likely to care. As you move down the list, people just seem to start shrugging their shoulders. Has a paper ever been retracted from Bioorganic and Medicinal Chemistry Letters, for example? I can't think of any, and readers are invited to try to find one themselves.

Another conclusion is that retracted papers tend to come from serial offenders. Over half the retracted papers had a first author who had retracted something else. I think that probably represents cases where someone had published a body of fraudulent work which all got exposed at once, but if that's not a serial offender, I don't know what is.

But a third conclusion is the one getting the headline writers going. The paper examines the retractions, looking for whether they were withdrawn for error or for fraud. The US is notable for having more retractions in the latter category, as compared to the rest of the world: one third of its 260 retracted papers are attributed to fraud, while the rest of the world comes in between 20% and 25%. So you get things like "US Scientists More Likely To Commit Fraud". But no, for that conclusion to be valid, you'd want to know how many fraudulent papers were published as a percentage of the whole output.

Even that wouldn't tell you the whole story. Remember, we're looking at papers that have actually been retracted. Most published fraudulent research never gets to that point. Lower-end journals have a terrible problem with plagiarized, derivative junk. Piles of it gets sent to them, then too much of it gets into print, and it just sits there, with no one ever paying any attention. Well, years later, some poor person might try to reproduce a prep, find that it doesn't work, sigh, and try something else, but otherwise. . .

No, in the same way that the prominent journals are over-represented, I think that the US might be a bit over-represented because slightly more fraud gets caught. More US papers appear in prominent journals than average, and fewer appear in the absolute bottom-rung journals. The United States accounts for at least one-third of the total scientific paper output, so those 260 papers out of 788 are exactly what you'd expect if all other things were equal. But other things aren't equal. We may have more prominent (and more harmful) frauds here, but I'd be willing to bet that as a proportion of the whole, we have fewer of them.

One last note: the figures in this paper seem to conflict with an earlier analysis, which seems to have found fewer retracted papers in PubMed, and a higher proportion of them tainted by fraud. Who's right?

Comments (17) + TrackBacks (0) | Category: The Scientific Literature

November 12, 2010

99% Yield? That, Friends, Is Deception

Email This Entry

Posted by Derek

Here's an attention-getting paper from Tomas Hudlicky (and his co-author Martina Wernerova), and I'd like to help it get some more. It begins:

One who has been reading the literature concerned with organic synthesis in recent years, be it methodology, catalysis, or total synthesis of natural products, may have noticed considerable inflation in the values reported for isolated product yields, ratios of diastereomers, and enantiomeric excess values. A comparison of papers published during the period 1955 to 1980 with those published between 1980 and 2005 reveals that those from the more recent period frequently report isolated product yields of reactions >95%. Such large values were rarely found in the older literature and are all but absent in Organic Syntheses, a journal that only publishes procedures that have been independently reproduced. . .

There, does that sound like the chemical literature you know? Just a bit? Hudlicky has tackled this issue before, and the reasons he advances for the problem remain the same: pressure to make your methods stand out (to the scientific community, to the journal editors, to the granting agencies), a decrease in scale in reactions (making accuracy and precision more difficult), and, finally, what he refers to as "deliberate adjustment". That's well put; the rest of us know it as fraud.

He identifies the mid-1980s as roughly the period when things really started to go to pieces, saying that most procedures in reputable journals before that era are reproducible by, as they say, one skilled in the art, while the numbers have been decreasing since then. And he puts some numbers on the problem, performing a series of test experiments with extremely careful weighing and analysis.

These confirm what every working organic chemist knows: the more manipulations, the more sample you lose. Filtration through a plug of silica gel, into one flask, can give you pretty much complete recovery. But if you cut fractions, you're going to lose about 1%. And if you have to do a separation, even between two widely separated compounds on silica, you're going to lose about 2%. So people who report a >98% yield after chromatography from a real-world crude mixture are kidding themselves. The same goes for extractions and other common methods. In general, every manipulation of a reaction is going to cost you 1 to 2% of your material, even with careful technique. Hudlicky again:

Given that most academic groups do not subject day-to-day reactions to serious optimization or matrix-optimization [6] as is done in industry, it is reasonable to assume that the vast majority of the reactions reported in the literature do not proceed with quantitative conversions. Such aspect would approximate our experiments with mixtures of pure compounds. Because a minimum of three operations (extraction, filtration, and evaporation) is required in working up most reactions, we conclude that yields higher than ca. 94% obtained by work-up and chromatography of crude reaction mixtures are likely unrealistic and erroneous in nature. Such values may arise as a direct consequence of not following correct protocols, which would be expected in the fast-paced academic environment. (An astute student of the organic literature may discover that this very author has been guilty of reporting yields in this range from time to time!)

He goes on to detail the limits of error in weighing, which depend greatly on the amount of sample and the size of the flask. (The smaller the sample-to-container ratio, the worse things get, as you'd figure). And he turns to analyzing mixures of diastereomers by NMR, LC, and the like. As it turns out, NMR is an excellent way to determine these up to about a ratio of 95:5 , but past that, things get tricky. And "past that" is just where a lot of papers go these days, with a precision that is often completely spurious.

Here's the bottom line:

The conclusion drawn from this set of experiments points to the prevalence of serious discrepancies in the reporting of values for yields and ratios in the current literature. We have demonstrated that the facilities and equipment available in a typical academic laboratory are not adequate to support the accuracy of claims frequently made in the literature. . .The current practice of reporting unrealistically high isolated product yields and stereoisomer ratios creates serious problems in reproducibility and hence leads to diminished credibility of the authors.

He recommends a rigorous disclosure of the spread of product yields over multiple experiments, calibration of LC and GC apparatus, or (failing that) at least admitting that no such analysis has been done. (He also recommends getting rid of the concepts of diastereomeric and enantiomeric excess, in line with my fellow Arkansan Robert Gawley's advice). But I think that these ideas, while perfectly reasonable, don't get at the underlying problems - the inflationary pressure to produce more and more noteworthy results. Hudlicky's rules should be adopted - but I fear that they might just push the self-deception (and outright fraud) into newer territories.

I'm glad he's published this paper, though. Because everyone knows that this is a real problem - we complain about it, we joke about it, we mutter and we grit our teeth. But "officially", in the published literature, it's never mentioned. Let's stop pretending, shall we?

Comments (67) + TrackBacks (0) | Category: The Dark Side | The Scientific Literature

And Now, the Retractome

Email This Entry

Posted by Derek

Back in January, I wrote about the controversial "Reactome" paper that had appeared in Science. This is the one that claimed to have immobilized over 1600 different kinds of biomolecules onto nanoparticles, and then used chemical means to set off a fluorescence assay when any protein recognized them. When actual organic chemists got a look at their scheme - something that apparently never happened during the review process - flags went up. As shown in that January post (and all over the chemical blogging world), the actual reactions looked, well, otherwordly.

Science was already backtracking within the first couple of months, and back in the summer, an institutional committee recommended that it be withdrawn. Since then, people have been waiting for the thunk of another shoe dropping, and now it's landed: the entire paper has been retracted. (More at C&E News). The lead author, though, tells Nature that other people have been using his methods, as described, and that he's still going to clear everything up.

I'm not sure how that's going to happen, but I'll be interested to see the attempt being made. The organic chemistry in the original paper was truly weird (and truly unworkable), and the whole concept of being able to whip up some complicated reactions schemes in the presence of a huge number of varied (and unprotected) molecules didn't make sense. The whole thing sounded like a particularly arrogant molecular biologist's idea of how synthetic chemistry should work: do it like a real biologist does! Sweeping boldly across the protein landscape, you just make them all work at the same time - haven't you chemists every heard of microarrays? Of proteomics? Why won't you people get with the times?

And the sorts of things that do work in modern biology would almost make you believe in that approach, until you look closely. Modern biology depends, though, on a wonderful legacy, a set of incredible tools bequeathed to us by billions of years of the most brutal product-development cycles imaginable (work or quite literally die). Organic chemistry, though, had no Aladdin's cave of enzymes and exquisitely adapted chemistries to stumble into. We've had to work everything out ourselves. And although we've gotten pretty good at it, the actions of something like RNA polymerase still look like the works of angels in comparison.

Comments (13) + TrackBacks (0) | Category: Biological News | The Scientific Literature

November 5, 2010

Peer Review's Problems

Email This Entry

Posted by Derek

Over at Ars Technica, here's an excellent look at the peer review process, which I last spoke about here. The author, Chris Lee, rightly points out that we ask it to do several different things, and it's not equally good at all of them.

His biggest problem is with the evaluation of research proposals for grants, and that has indeed been a problem for many years. Reviewing a paper, where you have to evaluate things that other people have done, can be hard enough. But evaluating what people hope to be able to do is much harder:

. . .Reviewers are asked to evaluate proposed methods, but, given that the authors themselves don't yet know if the methodology will work as described, how objective can they be? Unless the authors are totally incompetent and are proposing to use a method that is known not to work in the area they wish to use it, the reviewer cannot know what will happen.

As usual, there is no guarantee that the reviewer is more of an expert in the area than the authors. In fact, it's more often the case that they're not, so whose judgement should be trusted? There is just no way to tell a good researcher combined with incompetent peer review from an incompetent researcher and good peer review.

Reviewers are also asked to judge the significance of the proposed research. But wait—if peer review fails to consistently identify papers that are of significance when the results are in, what chance does it have of identifying significant contributions that haven't yet been made? Yeah, get out your dice. . .

And as he goes on to point out, the consequences of getting a grant proposal reviewed poorly are much worse than the ones from getting a paper's review messed up. These are both immediate (for the researcher involved) and systemic:

There is also a more insidious problem associated with peer review of grant applications. The evaluation of grant proposals is a reward-and-punishment system, but it doesn't systematically reward good proposals or good researchers, and it doesn't systematically reject bad proposals or punish poor researchers. Despite this, researchers are wont to treat it as if it was systematic and invest more time seeking the rewards than they do in performing active research, which is ostensibly where their talents lie.

Effectively, in trying to be objective and screen for the very best proposals, we waste a lot of time and fail to screen out bad proposals. This leads to a lot cynicism and, although I am often accused of being cynical, I don't believe it is a healthy attitude in research.

I fortunately haven't ever had to deal with this process, having spent my scientific career in industry, but we have our own problems with figuring out which projects to advance and why. Anyone who's interested in peer review, though, should know about the issues that Lee is bringing up. Well worth a read.

Comments (22) + TrackBacks (0) | Category: The Scientific Literature | Who Discovers and Why

October 19, 2010

Trusting the Medical Literature?

Email This Entry

Posted by Derek

How reliable is the medical literature, anyway? This profile of John Ioannidis at The Atlantic is food for thought. Ioannidis is the man behind the famous "Why Most Published Medical Findings Are False" paper a few years ago, and many others in the same vein.

The problems are many: publication bias (negative findings don't get written up and reported as often), confirmation bias, and desire to stand out/justify the time and money/get a grant renewal. And then there's good old lack of statistical power. Ioannidis and his colleagues have noted that far too many studies that appear in the medical journals are underpowered, statistically, relative to the claims made for them. The replication rates of such findings are not good.

Interestingly, drug research probably comes out of his analysis looking as good as anything can. A large confirmatory Phase III study is, as you'd hope, the sort of thing most likely to be correct, even given the financial considerations involved. Even then, though, you can't be completely sure - but contrast that with a lot of the headline-grabbing studies in nutrition or genomics, whose results are actually more likely to be false than true.

Ioannidis's rules from that PLoS Medicine paper are worth keeping in mind:

The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.

The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.

The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.

The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.

And although he's talking about the published literature, these things are well worth keeping in mind when you're looking at your own internal data in a drug discovery project. Some fraction of what you're seeing is wrong.

Comments (17) + TrackBacks (0) | Category: Clinical Trials | Drug Development | The Scientific Literature

October 15, 2010

Those Chemistry Bloggers

Email This Entry

Posted by Derek

After reading this piece on Chembark, I find that I have to help defend the whole chem-blogging enterprise. The latest Analytical Chemistry has an editorial from Royce Murray, on the subject. Unfortunately, it sounds like something written several years ago. He lays out the current model of scientific publishing, and along the way, briefly defends journal impact factors. Then he says that the lay public has traditionally received news of scientific discoveries through reports in magazines and newspapers, but that the finances in those areas has produced a "shrinkage" of the flow of reliable information to the public. And now we come to the part that really worries him:

In the above light, I believe that the current phenomenon of “bloggers” should be of serious concern to scientists. Bloggers are entrepreneurs who sell “news” (more properly, opinion) to mass media: internet, radio, TV, and to some extent print news. In former days, these individuals would be referred to as “freelance writers”, which they still are; the creation of the modern non-word “blogger” does not change the purveyor. An essential change is that these new freelancers, with the megaphone of the internet, can reach a much larger audience of potential clients than was possible in the past (and harness free “information sources”). . .blogging “agencies” are popping up that openly advertise “no formal qualifications are necessary” (as an internet search for “qualifications of bloggers” revealed). Who are the fact-checkers now? There are no reviewers in a formal sense, and writing can be done for any purpose—political, religious, business, etc.—without the constraint of truth.

He goes on to bemoan the lack of a system to qualify and fact-check these "bloggers" - I like the quotation marks, by the way - maybe we should stick those around every word that entered the language less than ten years ago, just to be sure. Now, it would be easy for me to spend a few paragraphs in the same mode as that last sentence, and as Murray accurately notes, there's no editor to stop me. But I won't, because some of his worries are well-founded.

There is indeed a lot of inaccurate nonsense on the internet. And everyone should read what they find online with a thought to who's written it, and why. But everyone should do the same with stuff that's printed on flattened sheets of dead trees, too, even if there are flattened-dead-tree-sheet editors and fact checkers. This is no place to list the stories that have been horribly messed up by even the most respectable of the old media. I'm thinking of a good list right now; any well-informed person should be able to. (If you can't, you're not as well-informed as you think you are). And there is indeed a lot of good science reporting in newspapers and magazines, although we can't ignore the fact that there's an awful lot of lazy and sloppy science reporting, too.

But there's a lot of inaccurate nonsense in the peer-reviewed literature, too. Without editors and reviewers there would surely be more, but too much junk gets through as it is. And if you want to see that stuff flagged, you'll do well to read the chemistry blogs. Murray's editorial doesn't seem to note that the most widely read ones are all written by chemists, not these unqualified people he's worried about. Would it be a cheap shot to point out that some editing and fact-checking might have caught that point before the editorial went out? Or (the other way around) to point out that a quick look through the scientific blogging world would have done just as well?

This, to me, is the real change that blogging has wrought, and I think it's for the better. Now, anyone who has the desire and ability can write about what they really know, about what they do for a living, and find an audience. I am most definitely not making a living as a journalist. My blog is a useful sideline to my real job, which is drug discovery. When I started in this industry, there was no way for me to self-publish my thoughts about it, but now there is, and I couldn't be happier about that.

Murray is suffering, I think, from a mental block, one that comes from his experience of journalism over his lifetime. He (and many other people) seem to feel that reporting is some sort of special profession, and that "real" journalists are the ones who write for the "real" news outlets. And it's a world where everyone knows which ones those are. It was fairly easy to believe that during the last half of the 20th century, but not so easy before it. (Or, as we're seeing, afterwards). All kinds of scruffy, opinionated people used to run newspapers in this country, and now they have internet sites. As do scientists, professors, lawyers, and anyone else with a keyboard and the desire. They're writing "for any purpose", just the way that Murray is worried about. And it's great.

A couple of postscripts: I should point out that I never would have seen Murray's piece at all had it not been for reading the scientific blogs - I hope that gives him some food for thought. He should also check out the above-linked Chembark post, as well as this response from Terra Sigillata (on an ACS site, no less), this one at InSightU, this one at Science 2.0, and this one at Cephalove. There will surely be more. Quite a lot of discussion! And it would be worth wondering how much of an impact this editorial would have had if it had only appeared in the print version of the journal, instead of being picked up by the Blogging Hordes. Could it be that he knows much more about the internet than he seems to, and he's successfully trolled us all?

Comments (41) + TrackBacks (0) | Category: The Scientific Literature

October 14, 2010

Conference Thoughts

Email This Entry

Posted by Derek

Update: here's a trip report on this conference over at Practical Fragments

I'm back from Philadelphia and the FBLD conference. I'm not going to put a trip report up on the blog - although I'm certainly writing one up for my colleagues at work - but a number of people at the meeting asked me what I might say about it here.

Well, I enjoyed it. I tend to like more focused conferences like this one, anyway, where most of the people doing the best work in a field can attend what's still a fairly small meeting. It probably helps that this isn't a very old series of meetings, too. Over time, some sort of scientific entropy sets in, and the topics covered can begin to smear out a bit. Some of the longer-running Gordon Conferences are (to me, anyway) a little blurry about what they're trying to cover.

That same tendency can affect individual talks. We medicinal chemists are particularly guilty of that, since our discipline spreads over a pretty wide area. At a meeting like this one, which was all about fragment-based techniques, people had to resist the temptation to keep going past the fragment-based parts of their talk. Once you get up toward 400 molecular weight, you're not talking about fragments any more - you're doing good old medicinal chemistry. Maybe it's structure-based at that point, maybe not, and maybe you're using some of the biophysical techniques that help out with moving fragment leads forward - but the fragment techniques are what got you to that point, not what's carrying you forward through the concerns about PK, formulations, polymorphs, and all the other later-stage worries of a drug program.

The speakers at this meeting generally did a good job avoiding this pitfall, but I have to admit that the few times I saw PK data come up on the screen, I stopped taking notes. I didn't stop listening, on the chance that there might be something interesting, but it certainly wasn't what I was there to focus on. One could imagine a whole meeting about solving PK problems in drug development - there probably is one, actually. But at that one, you'd have to make sure that the speakers didn't spend time telling you about the neat fragment-based techniques that led to their drug candidate.

As I said, though, there were a lot of interesting speakers at this one, and not a single talk was anything close to a complete waste of time. How many meetings can you say that about? Things ran smoothly, and with notably better food than some of the other conferences I've attended. Some meetings just pitch a bunch of Wissenschaftlerfutter out onto the tables, figuring that people will deal with it - and to be honest, they're usually right. We'll eat most anything in this field, although I've been told that physicists are even less discriminating, so at least we have that.

Comments (5) + TrackBacks (0) | Category: Life in the Drug Labs | The Scientific Literature

September 21, 2010

Want To Make an Amide? Have I Got Good News For You!

Email This Entry

Posted by Derek

Y'know, this is what I call an incremental improvement in the synthetic repetoire. I noticed this new paper in Tetrahedron Letters by its title, and read the whole thing just to make sure that I wasn't missing something.

Yep, that's right: someone has come up with a new way to form amides by reacting acid chlorides and amines. "But hold on," you say, "I thought that acid chlorides and amines form amides like an unstoppable juggernaut, which grinds to a halt only when enough HCl is given off to take the remaining amine out of contention". Well, you'd be right about that: but that's because you didn't think of using samarium metal as an acid scavenger.

Because that's what it seems to be here. The authors report that you have to pretty much use a full equivalent of samarium to get the high yields - control experiments with only 1/3 equivalent didn't work so well. What I wish they'd done is run the freaking control experiments with triethylamine. Or Hünig's base. Or pyridine. Or potassium carbonate, or aqueous 0.1N NaOH, or resin-bound nanocrystalline cesium complexes prepared in ionic liquids through renewable green chemistry whatchamacallits - in fact, with damn near anything else except stoichiometric metallic samarium, of all things. Well, OK: zinc and indium didn't work. I stand corrected. Give these folks another four or five Tet Lett papers, and they'll work their way back to baking soda. Only it'll be samarium bicarbonate, with any luck.

Perhaps I'm being unfair here. But really, amide formation is not a problem that is crying out for a new solution. It's really very, very, well worked out, and the number of options available for the experimentalist are nearly beyond counting. But now there's samarium metal. So if you're looking for the most expensive way you can think of to react an acid chloride with an amine, one that will make your labmates question your sanity and a reaction that will probably be a separate item all on its own come your next annual performance review, then go to it.

Oh, and one more thing: if you bother to read the experimental section, which apparently no one did, the procedure is titled: "General procedure for the homocoupling of terminal alkynes". Wrong samarium reaction, guys.

Comments (66) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

September 20, 2010

Sodium Hydride As Oxidant, Again?

Email This Entry

Posted by Derek

Many readers will remember the "sodium hydride as an oxidizing reagent" story from last year. (For the non-chemists in the audience, the problem here is that sodium hydride is most certainly not what you'd think of as an oxidizing reagent, quite the opposite, in fact. Seeing the paper's title was, for an organic chemist, a bit like reading about a new way to sweeten drinks with vinegar).

This was famously live-blogged over at Totally Synthetic and picked up on around the chemical blog world. The current thinking, though, is that adventitious oxygen is really doing the work here. If you run the reaction under strict inert atmosphere conditions, you get no more oxidation. (And it still doesn't appear that any note has been added to the original JACS paper). Update - make that no note added to the abstract page. The paper itself is still accessible, although it does have notes that it was withdrawn.

Well, now we have another one. This paper in press in Tetrahedron Letters claims oxidation of benzoins to benzils with good ol' sodium hydride. In this case, anyway, the authors (from Korea) did try running the reaction under inert atmosphere, and saw their yield go down. Their proposed mechanism involves molecular oxygen, in fact, and is quite plausible. (I've seen anion-oxygen chemistry myself - if you deprotonate Strecker amines of benzaldehydes, you'll convert them into amides via oxygen in your solvent, that is, if you don't saturate things with inert gas first). Still, I'd rather that they titled this paper differently, since it's not sodium hydride that's doing the oxidation here. You could probably get this to happen with NaHMDS, t-butoxide, or the base of your choice.

And, weirdly, the authors (as far as I can see by going over the PDF) manage not to cite the original JACS NaH oxidation paper at all. You'd never think that anyone had tried this before, especially not in one of the most high-profile chemical journals in the world, just last year, with plenty of added press coverage. What does it take a get a paper cited? Update: given the withdrawn-but-still-available status of the original, this becomes a trickier question. The earlier paper seems to have clearly gone through the same sort of chemistry, but the mechanism - and thus the point of the whole paper - was misassigned. Do you cite it, or not?

Comments (14) + TrackBacks (0) | Category: Chemical News | The Scientific Literature

J. Whoozat Sci.

Email This Entry

Posted by Derek

Not long ago I wrote about a Chinese journal that said that about a third of its submissions turned out to contain plagiarized material. Journal publishing in that country is apparently a real swamp, and the Chinese government has taken the publishers by surprise with an announcement that they're going to drain it:

Li Dongdong, a vice-minister of state and deputy director of the General Administration of Press and Publications (GAPP) — the powerful government body that regulates all publications in China — acknowledged that the country's scientific publishing had a "severe" problem, with "a big gap between quality and quantity", and needed reform.

Opening a meeting of scientific publishers in Shanghai on 7 September, Li announced that by January 2011, new regulations will be used to "terminate" weak journals.

Precisely how this reform will work is the subject of hot debate. . .News of the regulation startled many of the publishers at last week's meeting.

I'll bet it did, particularly those publishers who are turning out junk. And believe me, you know if you're publishing a crappy journal full of papers that no one reads. As that Nature News article goes on to detail, China is full of "campus journals", which exist only for the local grad students and the like to accumulate lines on their publication record. A colleague of mine used to call such titles "The Journal of Our Results", and that's right on target.

But while I can understand China's desire to upgrade its sometimes embarrassing scientific publishing world, I have to worry about the way that they're choosing to do it - not that it doesn't fit the best traditions of the Chinese government, mind you. This sit-still-while-we-fix-you approach may work in the short term, but if there's a demand for the Journal of Our Results (from authors, if not readers), then won't such titles just spring back up again under different names? As longtime readers here will easily be able to guess, I'd prefer a more market-oriented solution.

If the committees evaluating publishing records decided not to value such journals, much of their reason to exist would presumably vanish. While it's true that there's no perfect way to evaluate journals, a situation like China's - overrun with Journal of Whoozat Technical Colleges, 98% stocked with multi-part papers from the faculty of Whoozat - would seem to be waiting for everyone to just stop pretending. This let's-be-honest-here approach would let the people publishing this junk continue to exist, but they'd probably have to find a better business model.

As it is, just shutting these people down doesn't do anything for the let's-pretend side of the market, which I presume will continue to exist. And it'll probably be filled by the all-new Proceedings of the Whoozat Academy, Whoozat Letters, Acta Whoozatica, and who knows what else.

Comments (9) + TrackBacks (0) | Category: