Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Encoded Libraries Versus a Protein-Protein Interaction | Main | More on the Science Chemogenomic Signatures Paper »

April 11, 2014

Biology Maybe Right, Chemistry Ridiculously Wrong

Email This Entry

Posted by Derek

Note: critique of this paper continues here, in another post.

A reader sent along a puzzled note about this paper that's out in Science. It's from a large multicenter team (at least nine departments across the US, Canada, and Europe), and it's an ambitious effort to profile 3250 small molecules in a broad chemogenomics screen in yeast. This set was selected from an earlier 50,000 compounds, since these realiably inhibited the growth of wild-type yeast. They're looking for what they call "chemogenomic fitness signatures", which are derived from screening first against 1100 heterozygous yeast strains, one gene deletion per, representing the yeast essential genome. Then there's a second round of screening against 4800 homozygous deletions strain of non-essential genes, to look for related pathways, compensation, and so on.

All in all, they identified 317 compounds that appear to perturb 121 genes, and many of these annotations are new. Overall, the responses tended to cluster in related groups, and the paper goes into detail about these signatures (and about the outliers, which are naturally interested for their own reasons). Broad pathway effects like mitrochondrial stress show up pretty clearly, for example. And unfortunately, that's all I'm going to say for now about the biology, because we need to talk about the chemistry in this paper. It isn't good.

enamine.png
phenol.pngAs my correspondent (a chemist himself) mentions, a close look at Figure 2 of the paper raises some real questions. Take a look at that cyclohexadiene enamine - can that really be drawn correctly, or isn't it just N-phenylbenzylamine? The problem is, that compound (drawn correctly) shows up elsewhere in Figure 2, hitting a completely different pathway. These two tautomers are not going to have different biological effects, partly because the first one would exist for about two molecular vibrations before it converted to the second. But how could both of them appear on the same figure?

And look at what they're calling "cyclohexa-2,4-dien-1-one". No such compound exists as such in the real world - we call it phenol, and we draw it as an aromatic ring with an OH coming from it. Thiazolidinedione is listed as "thiazolidine-2,4-quinone". Both of these would lead to red "X" marks on an undergraduate exam paper. It is clear that no chemist, not even someone who's been through second-year organic class, was involved in this work (or at the very least, involved in the preparation of Figure 2). Why not? Who reviewed this, anyway?

There are some unusual features from a med-chem standpoint as well. Is THF really targeting tubulin folding? Does adamantane really target ubiquinone biosynthesis? Fine, these are the cellular effects that they noted, I guess. But the weirdest thing on Figure 2's annotations is that imidazole is shown as having one profile, while protonated imidazole is shown as a completely different one. How is this possible? How could anyone who knows any chemistry look at that and not raise an eyebrow? Isn't this assay run in some sort of buffered medium? Don't yeast cells have any buffering capacity of their own? Salts of basic amine drugs are dosed all the time, and they are not considered - ever - as having totally different cellular effects. What a world it would be if that were true! Seeing this sort of thing makes a person wonder about the rest of the paper.

Nitro.pngMore subtle problems emerge when you go to the supplementary material and take a look at the list of compounds. It's a pretty mixed bag. The concentrations used for the assays vary widely - rapamycin gets run at 1 micromolar, while ketoconazole is nearly 1 millimolar. (Can you even run that compound at that concentration? Or that compound at left at 967 micromolar? Is it really soluble in the yeast wells at such levels? There are plenty more that you can wonder about in the same way.

And I went searching for my old friends, the rhodanines, and there they were. Unfortunately, compound SGTC_2454 is 5-benzylidenerhodanine, whose activity is listed as "A dopamine receptor inhibitor" (!). But compound SGTC_1883 is also 5-benzylidenerhodanine, the same compound, run at similar concentration, but this time unannotated. The 5-thienylidenerhodanine is SGTC_30, but that one's listed as a phosphatase inhibitor. Neither of these attributions seem likely to me. There are other duplicates, but many of them are no doubt intentional (run by different parts of the team).

I hate to say this, but just a morning's look at this paper leaves me with little doubt that there are still more strange things buried in the chemistry side of this paper. But since I work for a living (dang it), I'm going to leave it right here, because what I've already noted is more than troubling enough. These mistakes are serious, and call the conclusions of the paper into question: if you can annotate imidazole and its protonated form into two different categories, or annotate two different tautomers (one of which doesn't really exist) into two different categories, what else is wrong, and how much are these annotations worth? And this isn't even the first time that Science has let something like this through. Back in 2010, they published a paper on the "Reactome" that had chemists around the world groaning. How many times does this lesson need to be learned, anyway?

Update: this situation brings up a number of larger issues, such as the divide between chemists and biologists (especially in academia?) and the place of organic chemistry in such high-profile publications (and the place of organic chemists as reviewers of it). I'll defer these to another post, but believe me, they're on my mind.

Update 2 Jake Yeston, deputy editor at Science, tells me that they're looking into this situation. More as I hear it.

Update 3: OK, if Figure 2 is just fragments, structural pieces that were common to compounds that had these signatures, then (1) these are still not acceptable structures, even as fragments, and (2), many of these don't make sense from a medicinal chemistry standpoint. It's bizarre to claim a tetrahydrofuran ring (for example) as the key driver for a class of compounds; the chance that this group is making an actual, persistent interaction with some protein site (or family of sites) is remote indeed. The imidazole/protonated imidazole pair is a good example of this: why on Earth would you pick these two groups to illustrate some chemical tendency? Again, this looks like the work of people who don't really have much chemical knowledge.

0560-0053.pngA closer look at the compounds themselves does not inspire any more confidence. There's one of them from Table S3, which showed a very large difference in IC50 across different yeast strains. It was tested at 400 micromolar. That, folks, was sold to the authors of this paper by ChemDiv, as part of a "drug-like compound" library. Try pulling some SMILES strings from that table yourself and see what you think about their drug likeness.

Comments (130) + TrackBacks (0) | Category: Chemical Biology | Chemical News | The Scientific Literature


COMMENTS

1. RM on April 11, 2014 8:31 AM writes...

Ten to one they just obtained a bunch of compounds from a number of sources and then passed the structures and annotations through unperturbed and unquestioned. The question that remains is what kind of shady chemical supplier is whiffing it on the structures and annotation?

Permalink to Comment

2. Anchor on April 11, 2014 8:47 AM writes...

Given their narrow understanding in organic chemistry, we can safely speculate that perhaps all the authors in that science paper were biologists. Where I work (academia) the role of chemists have been marginalized by the biology driven projects (hate to admit it, but biology is the driver!). These days the tools like Scifinder and other scientific data bases are so good that the collaborative projects that I am currently on is getting critical advices in organic chemistry from the biologists! And, one more thing rhodanine(s) must be disqualified from the biology studies especially during the initial lead discovery phase!

Permalink to Comment

3. See Ar Oh on April 11, 2014 9:09 AM writes...

Look at Fig 3, p. 211 - Compound "4215-0184," a pivotal result of their paper, looks to be tosylamide floating in space next to a pyrimidinyl imine.

D'oh!

Permalink to Comment

4. SP on April 11, 2014 9:15 AM writes...

Yeah, drawing error aside, figure 3 claims that because 4215-0184 and uracil have the same core ring structure it's logical that they're functionally similar. Carbonyls? We don't need no stinkin' carbonyls, they don't do anything important.

Permalink to Comment

5. David Borhani on April 11, 2014 9:17 AM writes...

I'm not sure complaints on this blog (sorry, Derek) will have any effect, but, here goes:

1. Science should promptly and unilaterally retract the paper. The mere fact that identical molecules (even if drawn as fleeting, very high-energy tautomers) give different biological "effects" invalidates the whole of the work.

2. Science should promptly publish, in full, the complete editorial review documentation (reviews, editorial comments, correspondence) on the manuscript. This is an egregious reviewing error---in many ways far exceeding the notorious As-DNA fiasco. The review process at Science appears to be horribly broken. The only way the journal can regain the credibility it deserves is to come clean. And then, having determined where they erred, to fix their review process.

Permalink to Comment

6. Bobby Shaftoe on April 11, 2014 9:26 AM writes...

Wow. Just wow.

I agree with David Borhani. It is stunning that this manuscript made it through the review process.

I have collaborated with a two of the coauthors. They are first rate scientists in their own right, but I really feel bad for them to be associated with this.

Permalink to Comment

7. Old Timer on April 11, 2014 9:34 AM writes...

What a bunch of rookies! Science is a disaster (the journal, not necessarily the field).

Permalink to Comment

8. MarkySparky on April 11, 2014 9:34 AM writes...

From the paper: "...we expect that many of our 317 chemical-genetic probes...may support novel targets as opportunities to pursue for therapeutic
intervention."

I guess we can add this to the list of glorious NIH-funded studies that industry exploits for unfair profit margins. If they play their cards right, GSK will buy their phenol lead compound for a princely sum...

Permalink to Comment

9. Old Timer on April 11, 2014 9:34 AM writes...

What a bunch of rookies! Science is a disaster (the journal, not necessarily the field). I also strongly agree with David Borhani.

Permalink to Comment

10. David Borhani on April 11, 2014 9:37 AM writes...

@6, Bobby: Ron Davis is a National Academy member and a professor of Biochemistry (and Genetics). No slouch, by any stretch of the imagination. Jacqueline Cherfils is also an excellent scientist. Where was their own, pre-submission review? Surely they cannot be so busy or eager to pad their CVs that they didn't have the time to glance at Figure 2 (let alone actually review the data)?

@Derek: Suggested new title for this post: "Biology Maybe Right Likely Wrong, Chemistry Ridiculously Wrong"

Email This Entry"

Permalink to Comment

11. Cato the Elder on April 11, 2014 9:43 AM writes...

Once all the industry jobs are gone maybe we can be manuscript editors for the biologists?

Permalink to Comment

12. Jonathan on April 11, 2014 9:44 AM writes...

It seems that these multidisciplinary studies that everyone *says* they want always end up doing a disservice to at least one of the fields involved. Sigh.

Still, perhaps biologist and chemists can unite against the real threat: arrogant physicists!

Permalink to Comment

13. luysii on April 11, 2014 9:45 AM writes...

Well, an attempt is being made to phase out organic chemistry from the pre-med curriculum, which I think is a huge mistake. See -- http://luysii.wordpress.com/2009/09/01/why-organic-chemistry-should-always-be-taken-and-passed-by-pre-meds/

Unfortunately it appears to have been phased out of some biology curricula as well.

Permalink to Comment

14. p on April 11, 2014 9:59 AM writes...

As luysii says, orgo is taking it on the chin. The new wisdom seems to be that organic chemistry has been overvalued. In the narrow matter of determining who gets into med school, this may well be correct (I don't really know but suspect it is).

But the backlash seems to be that no one needs to know that stuff because biochemistry is where it's really at. No matter how often you point out that biochem IS organic chemistry, you get ignored. I mean, fine, the folks writing and reviewing this paper don't need to know details of organometallic mechanisms. But a little basic chemistry would be nice.

Permalink to Comment

15. Bobby Shaftoe on April 11, 2014 10:01 AM writes...

@10, David, good point. Many (most!) of the authors have fantastic credentials, and it is hard to understand how this could happen. Does such a long and institutionally scattered authorship list mean that everyone assumes that someone else has done due diligence on the content?

Permalink to Comment

16. anonymous coward on April 11, 2014 10:03 AM writes...

The third structure appears to be a really nice Michael acceptor (especially at nearly 1 mM). It seems like it'd be hard to assume much selectivity for it.

This seems really embarrassing. If you're going to play with other people's structures and derive biological understanding from them, shouldn't you know what the basic rules for valid and chemically reasonable structures are, and have a rudimentary understanding of what they mean, or run the work by people who do? It might even help to have an idea of what structures have been known to cause problems in previous assays, so that you can run them through hoops to see that they're giving you reasonable and robust results. (It would also help if you want to make any good SAR from the results.) Aren't these reasonable things to ask of anyone publishing a paper on this?

Permalink to Comment

17. JAB on April 11, 2014 10:09 AM writes...

Sheesh! They're all molecular biologists, that's for sure, but they don't know molecules. Compound suppliers are noted in supplemental info. The usual suspects..."but we filtered the compound sets for non-reactivity"!

Permalink to Comment

18. CMCguy on April 11, 2014 10:09 AM writes...

Sure as chemists these are ridiculous avoidable errors and should be vetted in process by an appropriate reviewer (more a task for the paper's submitter not publisher) however I would bet that often when a biologist reads a typical medchem paper they will cringe at basic mistakes or misconceptions present when chemists attempt to summarize bioassay procedures or data in the paper (and wonder why the biologist involved didn't correct the chemists although they probably tried but chemists did not listen)

Permalink to Comment

19. Anonymous on April 11, 2014 10:11 AM writes...

what a bunch of morons

Permalink to Comment

20. Pete on April 11, 2014 10:15 AM writes...

Take a look at Nature Chemical Biology 8:639-645 (2012) the DOI for which I've linked as the URL for this comment. Tautomeric preferences vary within a structural series. I'm assuming that the authors took the tautomeric form from the vendor databases.

Permalink to Comment

21. David Borhani on April 11, 2014 10:21 AM writes...

@18, CMCguy: I don't think your analogy is at all apt, because you seem to be confusing the two very different types of error made by this paper. The drawing mistakes or absurdities are stupid, for sure. Far, far more serious, however, is that the *same* compound gives different biological effects. A fundamental tenet of pharmacology (and I don't care if you call it "Chemical Biology" or some other new-fangled name) is that the same molecule gives the same effect. Like I said in my original post, ALL of the data in this paper are thus very suspect.

Permalink to Comment

22. Anonymous on April 11, 2014 10:22 AM writes...

You are essentially pointing out that the authors did not draw/name the structures correctly. If the tautomer gets different biological response, then it is obviously another error in drawing the structure. It seems as if you were overdimensioning these aesthetic issues in order to justify the important role that chemists could have played in this study. However, if this is all you can contribute with, I can see why they did not feel the need to involve any chemist.

Permalink to Comment

23. Curious Wavefunction on April 11, 2014 10:23 AM writes...

This is kind of interesting because it reminds of the "arsenic bacteria" paper which had also not been sent out to organic chemists as reviewers. Recall that among all scientists, chemists were the most skeptical about the stability of the purported "arsenic-DNA" backbone. Yet because the paper was not sent to the right kind of reviewer it failed to be properly vetted (Benner, Baines and Seager published an article about this: click on my handle). A similar thing seems to have happened here.

Permalink to Comment

24. The Iron Chemist on April 11, 2014 10:25 AM writes...

I wish that I could say that I was surprised by all of this, but I've seen too many other cases of biologists being woefully ignorant of fundamental chemistry.

One area of concern is whether the compounds that they are screening are even pure. The different activities of the two imidazoles, for instance, could well be attributed to different levels and identities of impurities. Again, this is nothing new: I've seen previous cases where biological effects were attributed to compounds that were maybe 80% pure, at best.

Permalink to Comment

25. anonymous coward on April 11, 2014 10:30 AM writes...

@22: The problem is that attributing different activities to different tautomers doesn't make sense when one of the tautomers doesn't exist under biological conditions. If the authors couldn't understand that, then what other fundamental misunderstandings have they made in their work?

The concentration and polypharmacological issues are within the direct scope of competence of the authors as well, and they seem to have been dropped on the floor. That doesn't help.

Permalink to Comment

26. David Borhani on April 11, 2014 10:34 AM writes...

@24: Exactly why I mentioned As-DNA.

Permalink to Comment

27. MoMo on April 11, 2014 10:36 AM writes...

Whoa! Hold your horses angry chemists everywhere! I see it differently, that they are presenting only the fragments from the identified series responsible for bioactivity, hence the improbable structures. No self-respecting library vendor would supply improbable structures anyway- they can't using typical software.

I think this paper is brilliant! It identifies the skeletal functional groups with specific activities!

Chemical skeleton keys! Brilliant!

Permalink to Comment

28. Dave M on April 11, 2014 10:43 AM writes...

Just a lowly biologist here. All I can say is that, I assume that all on board gave their best efforts and it is possible that many of the complaints are due to some fairly straightforward mistakes that don't invalidate the whole study.

But...clearly there were some mistakes made. And if I am on the paper, right now I am really wishing that *somebody* caught something. Yes, they should have had chemists vet the manuscript, but what's to say they didn't try? Why didn't the journal have a single chemist review it? Or maybe they did and the review wasn't done properly. There are a lot of factors at play here

Permalink to Comment

29. Justin on April 11, 2014 10:50 AM writes...

Not surprising, really. The good news (hopefully) is that the attention this garners will entice other molecular biologists to walk across the hall/campus and have structures checked out by a chemist.

I guess this is post-publication review that could have been avoided by a good peer review process.

Permalink to Comment

30. Mike on April 11, 2014 10:53 AM writes...

"No self-respecting library vendor would supply improbable structures."

Hah! No, that would never happen.

Permalink to Comment

31. fluorogrol on April 11, 2014 10:57 AM writes...

I think their structure images come directly from ChemSpider.

Visually, they look exactly like ChemSpider structures, and the dienone tautomer of phenol has a ChemSpider entry, for reasons that escape me.

Also, if you search ChemSpider for thiazolidine-2,4-quinone, then it returns thiazolidine-2,4-dione, although the weird quinone name isn't listed as a synonym.

Permalink to Comment

32. anon on April 11, 2014 10:59 AM writes...

29, Justin

Well, what you are observing here is a form of peer review: the peers are reviewing the paper and are finding it wanting. One might ask why did a *formal* peer review, a part of an established scientific publication protocol, failed. The answer is known to anyone who sees how dysfunctional academic science promotion and scientific publishing have become.

Permalink to Comment

33. anonandon on April 11, 2014 11:08 AM writes...

@22, you are very confused.

In any event, these mistakes are so simple, you don't need a chemist to spot them. This is more like an April fool.

Permalink to Comment

34. johnnyboy on April 11, 2014 11:10 AM writes...

Am neither a biologist or a chemist so can't comment on this article, but to me this is yet another illustration of how the current manuscript review system is just no longer adequate, especially for these high impact journals. It's fine for journals that cover fairly restricted fields, where together the 2-3 reviewers have a better chance to have the necessary knowledge to critically evaluate the data. But how can you possibly pretend that 3 reviewers will be able to do a good job at reviewing papers that contain data on chemistry, pharmacology, molecular biology, cell biology, histology and whole animal studies - which is common in Nature journals ? Crap is bound to go through, and lord knows it does. If you're looking for an explanation for the "50% of published research can't be replicated" situation, you don't need to search much further.

Permalink to Comment

35. AiursLight on April 11, 2014 11:11 AM writes...

"not even someone who's been through second-year organic class"

I'm sure these researchers took O-chem. Whether or not they remember anything from the class or even paid attention is another matter entirely.

Bobby should probably talk with the two co-authors he knows and see what happened. Perhaps they don't know, and would want to withdraw their authorship, especially if the paper is as bad as you say.

Permalink to Comment

36. Anonymous on April 11, 2014 11:19 AM writes...

@34: I think that they should adopt a similar policy as the NEJM. I don't know if it is always the case, but they do something like two completely independent review processes on the paper - each with a set of 3 reviewers. In that way, they can cover the different fields more accurately and get two sessions of review to ensure the paper is not erroneous.

Permalink to Comment

37. Morgan Price on April 11, 2014 11:20 AM writes...

In terms of solubility -- the experiments are all in 1% DMSO. As a biologist who took O-chem long long ago, I don't know if that's enough DMSO to keep these molecules in solution.

Permalink to Comment

38. A Nonny Mouse on April 11, 2014 11:22 AM writes...

Reminds me of my old med chem days when the biologists discovered a diverse array of compounds that were novel anti-cancer agents. People (chemists) were trying to figure out how such different structures could be functioning in such a similar way.

After a couple of months it turned out that they were all oxalate salts..........

Permalink to Comment

39. anon-review on April 11, 2014 11:22 AM writes...

Having reviewed and recommended for rejection this sort of dreck (only to see it eventually published in Science) I can only say I'm not surprised. It has been clear to me as a reviewer and reader of these sorts of papers that they are not properly vetted at the review stage and that none of the authors knows anything about either small-molecule-based screening nor chemistry.

Negative reviews are frequently ignored and dismissed by the 'esteemed authors' and editors as 'irrelevant' or 'unnecessary'.

I note that ketoconazole is a well known antifungal agent so no surprise that it has activity in the screen....http://de.wikipedia.org/wiki/Ketoconazol

Permalink to Comment

40. A Nonny Mouse on April 11, 2014 11:27 AM writes...

Reminds me of my old med chem days when the biologists discovered a diverse array of compounds that were novel anti-cancer agents. People (chemists) were trying to figure out how such different structures could be functioning in such a similar way.

After a couple of months it turned out that they were all oxalate salts..........

Permalink to Comment

41. fluorogrol on April 11, 2014 11:31 AM writes...

"Take a look at that cyclohexadiene enamine - can that really be drawn correctly, or isn't it just N-phenylbenzylamine? The probl