Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Encoded Libraries Versus a Protein-Protein Interaction | Main | More on the Science Chemogenomic Signatures Paper »

April 11, 2014

Biology Maybe Right, Chemistry Ridiculously Wrong

Email This Entry

Posted by Derek

Note: critique of this paper continues here, in another post.

A reader sent along a puzzled note about this paper that's out in Science. It's from a large multicenter team (at least nine departments across the US, Canada, and Europe), and it's an ambitious effort to profile 3250 small molecules in a broad chemogenomics screen in yeast. This set was selected from an earlier 50,000 compounds, since these realiably inhibited the growth of wild-type yeast. They're looking for what they call "chemogenomic fitness signatures", which are derived from screening first against 1100 heterozygous yeast strains, one gene deletion per, representing the yeast essential genome. Then there's a second round of screening against 4800 homozygous deletions strain of non-essential genes, to look for related pathways, compensation, and so on.

All in all, they identified 317 compounds that appear to perturb 121 genes, and many of these annotations are new. Overall, the responses tended to cluster in related groups, and the paper goes into detail about these signatures (and about the outliers, which are naturally interested for their own reasons). Broad pathway effects like mitrochondrial stress show up pretty clearly, for example. And unfortunately, that's all I'm going to say for now about the biology, because we need to talk about the chemistry in this paper. It isn't good.

enamine.png
phenol.pngAs my correspondent (a chemist himself) mentions, a close look at Figure 2 of the paper raises some real questions. Take a look at that cyclohexadiene enamine - can that really be drawn correctly, or isn't it just N-phenylbenzylamine? The problem is, that compound (drawn correctly) shows up elsewhere in Figure 2, hitting a completely different pathway. These two tautomers are not going to have different biological effects, partly because the first one would exist for about two molecular vibrations before it converted to the second. But how could both of them appear on the same figure?

And look at what they're calling "cyclohexa-2,4-dien-1-one". No such compound exists as such in the real world - we call it phenol, and we draw it as an aromatic ring with an OH coming from it. Thiazolidinedione is listed as "thiazolidine-2,4-quinone". Both of these would lead to red "X" marks on an undergraduate exam paper. It is clear that no chemist, not even someone who's been through second-year organic class, was involved in this work (or at the very least, involved in the preparation of Figure 2). Why not? Who reviewed this, anyway?

There are some unusual features from a med-chem standpoint as well. Is THF really targeting tubulin folding? Does adamantane really target ubiquinone biosynthesis? Fine, these are the cellular effects that they noted, I guess. But the weirdest thing on Figure 2's annotations is that imidazole is shown as having one profile, while protonated imidazole is shown as a completely different one. How is this possible? How could anyone who knows any chemistry look at that and not raise an eyebrow? Isn't this assay run in some sort of buffered medium? Don't yeast cells have any buffering capacity of their own? Salts of basic amine drugs are dosed all the time, and they are not considered - ever - as having totally different cellular effects. What a world it would be if that were true! Seeing this sort of thing makes a person wonder about the rest of the paper.

Nitro.pngMore subtle problems emerge when you go to the supplementary material and take a look at the list of compounds. It's a pretty mixed bag. The concentrations used for the assays vary widely - rapamycin gets run at 1 micromolar, while ketoconazole is nearly 1 millimolar. (Can you even run that compound at that concentration? Or that compound at left at 967 micromolar? Is it really soluble in the yeast wells at such levels? There are plenty more that you can wonder about in the same way.

And I went searching for my old friends, the rhodanines, and there they were. Unfortunately, compound SGTC_2454 is 5-benzylidenerhodanine, whose activity is listed as "A dopamine receptor inhibitor" (!). But compound SGTC_1883 is also 5-benzylidenerhodanine, the same compound, run at similar concentration, but this time unannotated. The 5-thienylidenerhodanine is SGTC_30, but that one's listed as a phosphatase inhibitor. Neither of these attributions seem likely to me. There are other duplicates, but many of them are no doubt intentional (run by different parts of the team).

I hate to say this, but just a morning's look at this paper leaves me with little doubt that there are still more strange things buried in the chemistry side of this paper. But since I work for a living (dang it), I'm going to leave it right here, because what I've already noted is more than troubling enough. These mistakes are serious, and call the conclusions of the paper into question: if you can annotate imidazole and its protonated form into two different categories, or annotate two different tautomers (one of which doesn't really exist) into two different categories, what else is wrong, and how much are these annotations worth? And this isn't even the first time that Science has let something like this through. Back in 2010, they published a paper on the "Reactome" that had chemists around the world groaning. How many times does this lesson need to be learned, anyway?

Update: this situation brings up a number of larger issues, such as the divide between chemists and biologists (especially in academia?) and the place of organic chemistry in such high-profile publications (and the place of organic chemists as reviewers of it). I'll defer these to another post, but believe me, they're on my mind.

Update 2 Jake Yeston, deputy editor at Science, tells me that they're looking into this situation. More as I hear it.

Update 3: OK, if Figure 2 is just fragments, structural pieces that were common to compounds that had these signatures, then (1) these are still not acceptable structures, even as fragments, and (2), many of these don't make sense from a medicinal chemistry standpoint. It's bizarre to claim a tetrahydrofuran ring (for example) as the key driver for a class of compounds; the chance that this group is making an actual, persistent interaction with some protein site (or family of sites) is remote indeed. The imidazole/protonated imidazole pair is a good example of this: why on Earth would you pick these two groups to illustrate some chemical tendency? Again, this looks like the work of people who don't really have much chemical knowledge.

0560-0053.pngA closer look at the compounds themselves does not inspire any more confidence. There's one of them from Table S3, which showed a very large difference in IC50 across different yeast strains. It was tested at 400 micromolar. That, folks, was sold to the authors of this paper by ChemDiv, as part of a "drug-like compound" library. Try pulling some SMILES strings from that table yourself and see what you think about their drug likeness.

Comments (130) + TrackBacks (0) | Category: Chemical Biology | Chemical News | The Scientific Literature


COMMENTS

1. RM on April 11, 2014 8:31 AM writes...

Ten to one they just obtained a bunch of compounds from a number of sources and then passed the structures and annotations through unperturbed and unquestioned. The question that remains is what kind of shady chemical supplier is whiffing it on the structures and annotation?

Permalink to Comment

2. Anchor on April 11, 2014 8:47 AM writes...

Given their narrow understanding in organic chemistry, we can safely speculate that perhaps all the authors in that science paper were biologists. Where I work (academia) the role of chemists have been marginalized by the biology driven projects (hate to admit it, but biology is the driver!). These days the tools like Scifinder and other scientific data bases are so good that the collaborative projects that I am currently on is getting critical advices in organic chemistry from the biologists! And, one more thing rhodanine(s) must be disqualified from the biology studies especially during the initial lead discovery phase!

Permalink to Comment

3. See Ar Oh on April 11, 2014 9:09 AM writes...

Look at Fig 3, p. 211 - Compound "4215-0184," a pivotal result of their paper, looks to be tosylamide floating in space next to a pyrimidinyl imine.

D'oh!

Permalink to Comment

4. SP on April 11, 2014 9:15 AM writes...

Yeah, drawing error aside, figure 3 claims that because 4215-0184 and uracil have the same core ring structure it's logical that they're functionally similar. Carbonyls? We don't need no stinkin' carbonyls, they don't do anything important.

Permalink to Comment

5. David Borhani on April 11, 2014 9:17 AM writes...

I'm not sure complaints on this blog (sorry, Derek) will have any effect, but, here goes:

1. Science should promptly and unilaterally retract the paper. The mere fact that identical molecules (even if drawn as fleeting, very high-energy tautomers) give different biological "effects" invalidates the whole of the work.

2. Science should promptly publish, in full, the complete editorial review documentation (reviews, editorial comments, correspondence) on the manuscript. This is an egregious reviewing error---in many ways far exceeding the notorious As-DNA fiasco. The review process at Science appears to be horribly broken. The only way the journal can regain the credibility it deserves is to come clean. And then, having determined where they erred, to fix their review process.

Permalink to Comment

6. Bobby Shaftoe on April 11, 2014 9:26 AM writes...

Wow. Just wow.

I agree with David Borhani. It is stunning that this manuscript made it through the review process.

I have collaborated with a two of the coauthors. They are first rate scientists in their own right, but I really feel bad for them to be associated with this.

Permalink to Comment

7. Old Timer on April 11, 2014 9:34 AM writes...

What a bunch of rookies! Science is a disaster (the journal, not necessarily the field).

Permalink to Comment

8. MarkySparky on April 11, 2014 9:34 AM writes...

From the paper: "...we expect that many of our 317 chemical-genetic probes...may support novel targets as opportunities to pursue for therapeutic
intervention."

I guess we can add this to the list of glorious NIH-funded studies that industry exploits for unfair profit margins. If they play their cards right, GSK will buy their phenol lead compound for a princely sum...

Permalink to Comment

9. Old Timer on April 11, 2014 9:34 AM writes...

What a bunch of rookies! Science is a disaster (the journal, not necessarily the field). I also strongly agree with David Borhani.

Permalink to Comment

10. David Borhani on April 11, 2014 9:37 AM writes...

@6, Bobby: Ron Davis is a National Academy member and a professor of Biochemistry (and Genetics). No slouch, by any stretch of the imagination. Jacqueline Cherfils is also an excellent scientist. Where was their own, pre-submission review? Surely they cannot be so busy or eager to pad their CVs that they didn't have the time to glance at Figure 2 (let alone actually review the data)?

@Derek: Suggested new title for this post: "Biology Maybe Right Likely Wrong, Chemistry Ridiculously Wrong"

Email This Entry"

Permalink to Comment

11. Cato the Elder on April 11, 2014 9:43 AM writes...

Once all the industry jobs are gone maybe we can be manuscript editors for the biologists?

Permalink to Comment

12. Jonathan on April 11, 2014 9:44 AM writes...

It seems that these multidisciplinary studies that everyone *says* they want always end up doing a disservice to at least one of the fields involved. Sigh.

Still, perhaps biologist and chemists can unite against the real threat: arrogant physicists!

Permalink to Comment

13. luysii on April 11, 2014 9:45 AM writes...

Well, an attempt is being made to phase out organic chemistry from the pre-med curriculum, which I think is a huge mistake. See -- http://luysii.wordpress.com/2009/09/01/why-organic-chemistry-should-always-be-taken-and-passed-by-pre-meds/

Unfortunately it appears to have been phased out of some biology curricula as well.

Permalink to Comment

14. p on April 11, 2014 9:59 AM writes...

As luysii says, orgo is taking it on the chin. The new wisdom seems to be that organic chemistry has been overvalued. In the narrow matter of determining who gets into med school, this may well be correct (I don't really know but suspect it is).

But the backlash seems to be that no one needs to know that stuff because biochemistry is where it's really at. No matter how often you point out that biochem IS organic chemistry, you get ignored. I mean, fine, the folks writing and reviewing this paper don't need to know details of organometallic mechanisms. But a little basic chemistry would be nice.

Permalink to Comment

15. Bobby Shaftoe on April 11, 2014 10:01 AM writes...

@10, David, good point. Many (most!) of the authors have fantastic credentials, and it is hard to understand how this could happen. Does such a long and institutionally scattered authorship list mean that everyone assumes that someone else has done due diligence on the content?

Permalink to Comment

16. anonymous coward on April 11, 2014 10:03 AM writes...

The third structure appears to be a really nice Michael acceptor (especially at nearly 1 mM). It seems like it'd be hard to assume much selectivity for it.

This seems really embarrassing. If you're going to play with other people's structures and derive biological understanding from them, shouldn't you know what the basic rules for valid and chemically reasonable structures are, and have a rudimentary understanding of what they mean, or run the work by people who do? It might even help to have an idea of what structures have been known to cause problems in previous assays, so that you can run them through hoops to see that they're giving you reasonable and robust results. (It would also help if you want to make any good SAR from the results.) Aren't these reasonable things to ask of anyone publishing a paper on this?

Permalink to Comment

17. JAB on April 11, 2014 10:09 AM writes...

Sheesh! They're all molecular biologists, that's for sure, but they don't know molecules. Compound suppliers are noted in supplemental info. The usual suspects..."but we filtered the compound sets for non-reactivity"!

Permalink to Comment

18. CMCguy on April 11, 2014 10:09 AM writes...

Sure as chemists these are ridiculous avoidable errors and should be vetted in process by an appropriate reviewer (more a task for the paper's submitter not publisher) however I would bet that often when a biologist reads a typical medchem paper they will cringe at basic mistakes or misconceptions present when chemists attempt to summarize bioassay procedures or data in the paper (and wonder why the biologist involved didn't correct the chemists although they probably tried but chemists did not listen)

Permalink to Comment

19. Anonymous on April 11, 2014 10:11 AM writes...

what a bunch of morons

Permalink to Comment

20. Pete on April 11, 2014 10:15 AM writes...

Take a look at Nature Chemical Biology 8:639-645 (2012) the DOI for which I've linked as the URL for this comment. Tautomeric preferences vary within a structural series. I'm assuming that the authors took the tautomeric form from the vendor databases.

Permalink to Comment

21. David Borhani on April 11, 2014 10:21 AM writes...

@18, CMCguy: I don't think your analogy is at all apt, because you seem to be confusing the two very different types of error made by this paper. The drawing mistakes or absurdities are stupid, for sure. Far, far more serious, however, is that the *same* compound gives different biological effects. A fundamental tenet of pharmacology (and I don't care if you call it "Chemical Biology" or some other new-fangled name) is that the same molecule gives the same effect. Like I said in my original post, ALL of the data in this paper are thus very suspect.

Permalink to Comment

22. Anonymous on April 11, 2014 10:22 AM writes...

You are essentially pointing out that the authors did not draw/name the structures correctly. If the tautomer gets different biological response, then it is obviously another error in drawing the structure. It seems as if you were overdimensioning these aesthetic issues in order to justify the important role that chemists could have played in this study. However, if this is all you can contribute with, I can see why they did not feel the need to involve any chemist.

Permalink to Comment

23. Curious Wavefunction on April 11, 2014 10:23 AM writes...

This is kind of interesting because it reminds of the "arsenic bacteria" paper which had also not been sent out to organic chemists as reviewers. Recall that among all scientists, chemists were the most skeptical about the stability of the purported "arsenic-DNA" backbone. Yet because the paper was not sent to the right kind of reviewer it failed to be properly vetted (Benner, Baines and Seager published an article about this: click on my handle). A similar thing seems to have happened here.

Permalink to Comment

24. The Iron Chemist on April 11, 2014 10:25 AM writes...

I wish that I could say that I was surprised by all of this, but I've seen too many other cases of biologists being woefully ignorant of fundamental chemistry.

One area of concern is whether the compounds that they are screening are even pure. The different activities of the two imidazoles, for instance, could well be attributed to different levels and identities of impurities. Again, this is nothing new: I've seen previous cases where biological effects were attributed to compounds that were maybe 80% pure, at best.

Permalink to Comment

25. anonymous coward on April 11, 2014 10:30 AM writes...

@22: The problem is that attributing different activities to different tautomers doesn't make sense when one of the tautomers doesn't exist under biological conditions. If the authors couldn't understand that, then what other fundamental misunderstandings have they made in their work?

The concentration and polypharmacological issues are within the direct scope of competence of the authors as well, and they seem to have been dropped on the floor. That doesn't help.

Permalink to Comment

26. David Borhani on April 11, 2014 10:34 AM writes...

@24: Exactly why I mentioned As-DNA.

Permalink to Comment

27. MoMo on April 11, 2014 10:36 AM writes...

Whoa! Hold your horses angry chemists everywhere! I see it differently, that they are presenting only the fragments from the identified series responsible for bioactivity, hence the improbable structures. No self-respecting library vendor would supply improbable structures anyway- they can't using typical software.

I think this paper is brilliant! It identifies the skeletal functional groups with specific activities!

Chemical skeleton keys! Brilliant!

Permalink to Comment

28. Dave M on April 11, 2014 10:43 AM writes...

Just a lowly biologist here. All I can say is that, I assume that all on board gave their best efforts and it is possible that many of the complaints are due to some fairly straightforward mistakes that don't invalidate the whole study.

But...clearly there were some mistakes made. And if I am on the paper, right now I am really wishing that *somebody* caught something. Yes, they should have had chemists vet the manuscript, but what's to say they didn't try? Why didn't the journal have a single chemist review it? Or maybe they did and the review wasn't done properly. There are a lot of factors at play here

Permalink to Comment

29. Justin on April 11, 2014 10:50 AM writes...

Not surprising, really. The good news (hopefully) is that the attention this garners will entice other molecular biologists to walk across the hall/campus and have structures checked out by a chemist.

I guess this is post-publication review that could have been avoided by a good peer review process.

Permalink to Comment

30. Mike on April 11, 2014 10:53 AM writes...

"No self-respecting library vendor would supply improbable structures."

Hah! No, that would never happen.

Permalink to Comment

31. fluorogrol on April 11, 2014 10:57 AM writes...

I think their structure images come directly from ChemSpider.

Visually, they look exactly like ChemSpider structures, and the dienone tautomer of phenol has a ChemSpider entry, for reasons that escape me.

Also, if you search ChemSpider for thiazolidine-2,4-quinone, then it returns thiazolidine-2,4-dione, although the weird quinone name isn't listed as a synonym.

Permalink to Comment

32. anon on April 11, 2014 10:59 AM writes...

29, Justin

Well, what you are observing here is a form of peer review: the peers are reviewing the paper and are finding it wanting. One might ask why did a *formal* peer review, a part of an established scientific publication protocol, failed. The answer is known to anyone who sees how dysfunctional academic science promotion and scientific publishing have become.

Permalink to Comment

33. anonandon on April 11, 2014 11:08 AM writes...

@22, you are very confused.

In any event, these mistakes are so simple, you don't need a chemist to spot them. This is more like an April fool.

Permalink to Comment

34. johnnyboy on April 11, 2014 11:10 AM writes...

Am neither a biologist or a chemist so can't comment on this article, but to me this is yet another illustration of how the current manuscript review system is just no longer adequate, especially for these high impact journals. It's fine for journals that cover fairly restricted fields, where together the 2-3 reviewers have a better chance to have the necessary knowledge to critically evaluate the data. But how can you possibly pretend that 3 reviewers will be able to do a good job at reviewing papers that contain data on chemistry, pharmacology, molecular biology, cell biology, histology and whole animal studies - which is common in Nature journals ? Crap is bound to go through, and lord knows it does. If you're looking for an explanation for the "50% of published research can't be replicated" situation, you don't need to search much further.

Permalink to Comment

35. AiursLight on April 11, 2014 11:11 AM writes...

"not even someone who's been through second-year organic class"

I'm sure these researchers took O-chem. Whether or not they remember anything from the class or even paid attention is another matter entirely.

Bobby should probably talk with the two co-authors he knows and see what happened. Perhaps they don't know, and would want to withdraw their authorship, especially if the paper is as bad as you say.

Permalink to Comment

36. Anonymous on April 11, 2014 11:19 AM writes...

@34: I think that they should adopt a similar policy as the NEJM. I don't know if it is always the case, but they do something like two completely independent review processes on the paper - each with a set of 3 reviewers. In that way, they can cover the different fields more accurately and get two sessions of review to ensure the paper is not erroneous.

Permalink to Comment

37. Morgan Price on April 11, 2014 11:20 AM writes...

In terms of solubility -- the experiments are all in 1% DMSO. As a biologist who took O-chem long long ago, I don't know if that's enough DMSO to keep these molecules in solution.

Permalink to Comment

38. A Nonny Mouse on April 11, 2014 11:22 AM writes...

Reminds me of my old med chem days when the biologists discovered a diverse array of compounds that were novel anti-cancer agents. People (chemists) were trying to figure out how such different structures could be functioning in such a similar way.

After a couple of months it turned out that they were all oxalate salts..........

Permalink to Comment

39. anon-review on April 11, 2014 11:22 AM writes...

Having reviewed and recommended for rejection this sort of dreck (only to see it eventually published in Science) I can only say I'm not surprised. It has been clear to me as a reviewer and reader of these sorts of papers that they are not properly vetted at the review stage and that none of the authors knows anything about either small-molecule-based screening nor chemistry.

Negative reviews are frequently ignored and dismissed by the 'esteemed authors' and editors as 'irrelevant' or 'unnecessary'.

I note that ketoconazole is a well known antifungal agent so no surprise that it has activity in the screen....http://de.wikipedia.org/wiki/Ketoconazol

Permalink to Comment

40. A Nonny Mouse on April 11, 2014 11:27 AM writes...

Reminds me of my old med chem days when the biologists discovered a diverse array of compounds that were novel anti-cancer agents. People (chemists) were trying to figure out how such different structures could be functioning in such a similar way.

After a couple of months it turned out that they were all oxalate salts..........

Permalink to Comment

41. fluorogrol on April 11, 2014 11:31 AM writes...

"Take a look at that cyclohexadiene enamine - can that really be drawn correctly, or isn't it just N-phenylbenzylamine? The problem is, that compound (drawn correctly) shows up elsewhere in Figure 2, hitting a completely different pathway. These two tautomers are not going to have different biological effects, partly because the first one would exist for about two molecular vibrations before it converted to the second. But how could both of them appear on the same figure?"

If I understand the paper, they are not suggesting that the structures drawn in fig. 2 have these bio effects, rather structures containing the fragment shown.

So presumably what they've done (due to lack of chemical understanding) is draw the bit that's common to several structures, but without indicating any R groups etc. Stick R and R' on the sp3 carbon in cyclohexadienone and suddenly it doesn't look so bad.

Permalink to Comment

42. Dave Fernig on April 11, 2014 11:44 AM writes...

If the chemistry is wrong, the biology is also wrong. Looking forward to seeing these and many other comments up on PupPeer, so that when people do a PubMed search with the plugin, they get to see the comments. One purpose of post publication peer review. Otherwise, it is not advisable to hold one's breath waiting for action on the paper. Many molecular cell biologists know no chemistry and will take the paper as gospel (hence the importance of getting critique up on PubPeer, where is has a broad visibility).
As a biologist who not only knows where his Chemistry Department, but can navigate to any office there with ease, in my mind the purity/solubility questions alone warrant immediate retraction. Compounds can exert profound activity at nM-uM concentrations, so one has to be careful even at 99% purity.
At least this is one less paper for me to read, much obliged for reducing my workload!

Permalink to Comment

43. MDACC Grad on April 11, 2014 11:49 AM writes...

If you think this is bad, you should peak behind the curtain of what is going on with cell lines!

Permalink to Comment

44. Anonymous on April 11, 2014 11:49 AM writes...

@41: It's a bit naive to propose that such simple structures (like a THF ring or pyrrole) are responsible for the common phenotypes these compounds possess, especially at the concentrations used.

And imidazole versus protonated imidazole, can't really see how those are different fragments...

Permalink to Comment

45. fluorogrol on April 11, 2014 11:56 AM writes...

@44: Presumably it's actually alkylated imidazole, turned into protonated imidazole by their process of chopping out the common fragment from the structure.

But yes, I agree that a lot of the correlations seem likely to be meaningless.

Permalink to Comment

46. bad wolf on April 11, 2014 12:18 PM writes...

Wait... did someone just call SciFinder "good"?

Permalink to Comment

47. And Yet on April 11, 2014 12:40 PM writes...

The people involved with this paper will all have jobs. Frankly, given the incredibly poor job outlook for the younger generation, let's just throw all these people, especially the professors, out on the street. A few job openings that will go to people that won't do such a terrible job would be great.

Permalink to Comment

48. lol on April 11, 2014 12:51 PM writes...

I was trained in molecular biology. The problem is you become a one trick pony that relies on kits to handle all the detail.

To them the library was a kit, and in this case they got burned. Molecular tech is very forgiving and relatively easy to do so it creates the illusion that you are doing a lot. Hence you end up loosing that critical edge and make a lot of assumptions about you "kits" that may not be true because you don't really understand the detail.

Most of the time the shotgun approach will sort out your mistakes and give you what you need even if its a low probability event. But fundamentals like this is where you get burned.


my feeling is this is just the tip of "the house of cards built on molecular biology" iceberg

Permalink to Comment

49. Algirdas on April 11, 2014 1:16 PM writes...

Derek (and others),

MoMo @27 and Fluorogrol @41 are correct. Read the caption to figure 2. It says "Signatures are connected to chemical moiety nodes where signature compounds are enriched". Chemical moiety nodes, not compounds! (Also, check Table S1, THF or imidazole are not there).

Seems to me the authors are not guilty of poor chemistry, but rather of making a crowded, complex figure. One could argue that these "chemical moiety nodes" do not convey much meaningful information, but that is not the same as the authors not being able to distinguish uncharged and protonated imidazole.

Permalink to Comment

50. David Borhani on April 11, 2014 1:21 PM writes...

Several additional comments:

@3, See Ar Oh: The line between the N atoms in Fig. 3 is just misdrawn: they should be connected, making the tosyl hydrazone of a substituted 4-hydroxybenzaldehyde. Much more important than this sloppy & avoidable mistake, however, is that a 5-fluoropyrimidine is not, I believe, equivalent to a 5-fluorouracil (-uridine). (Synthetic chemists, please correct me if I'm wrong here.) I don't think the 5-fluorine atom in a 5-fluoropyrimine can be displaced in the same way as in a 5-fluorouracil (photochemistry: yes; nucleophilic displacement or SnAR; no). So equating these compounds by placing them in the same biological pathway (without oxidation of the pyrimidine to a uracil) strikes me as an untenable reach. Moreover, the compound also happens to be a 2-chloropyrimidine (with additional activation by the 5-fluoro substituent), i.e. reactive. Reactive enough to be indiscriminate in a cellular assay? Perhaps. Maybe it hits the active site cysteine in thymidylate synthase too, which is just one of the many biochemical end points responsible for the useful biological activity of 5-fluorouracil (-uridine).

@41, Fluorogrol: Your beneficial reading of Fig. 2 involves the whole concept of "Signatures". What is a
"Signature"? As I read the article (from its first mention in the article title, and then onward), a "chemogenomic response signature" is a phenotypic response to a compound or compounds that is tied to a one gene or to a set of genes (hence the haploinsufficiency yeast strain panel). A "chemogenomic response signature" is NOT a chemical fragment (even a stupidly drawn one). IF it were, somehow, a fragment, then what would be the relevance and significance of this paper? That any compound containing, e.g., a tetrahydrofuran fragment will affect tubulin folding? Or that all compounds that affect tubulin folding contain tetrahydrofuran fragments? Surely not!

This work seems to be the Emperor parading about in His new clothes. The beguiling mix of (poorly done) organic/medicinal chemistry, genetics, cell biology, and statistics fools the eye into believing that something substantial has been revealed.

Permalink to Comment

51. fluorogrol on April 11, 2014 2:32 PM writes...

@50

David, I'm pretty sure that I agree with you regarding the conclusions of the paper.

My comment to which you responded was attempting to explain how/why the authors ended up drawing these bizarre structures, not attempting to defend their conclusions (see also my later comment).

It's quite clear from the SI files that the structures in fig. 2 are meant to represent common features of the compounds that hit a particular target. In fact, their spreadsheets contains SMILES strings for the common fragments, which they may well have just pasted back into a drawing program.

Not realising that simply deleting all the other bonds and atoms around a structural fragment will sometimes render the result chemically ludicrous, they ended up with the structures we see.

I believe they ARE implying a correlation between e.g. a THF fragment and tubulin binding. I think that is the major failing of the paper, and the structural stuff (although rookie to us chemists) is a secondary problem. I wonder how they set the cut-off for how small a common fragment could be? I've seen lots of biologically active compounds with 'ethane' fragments...

(Obviously there's a fair amount of conjecture in there, and the paper involves areas that I can't claim to have expertise in. But hey-ho, I've been wrong before and I'll be wrong again.)

Permalink to Comment

52. Becca on April 11, 2014 2:34 PM writes...

Equally disturbing is the fact I show discrepancies in this paper to fellow postdocs (in chemistry at unnamed ivy league, prior to finding this blog entry) and they didn't see why it was a big deal. They argued that the phenylbenzylamine tautomers are equally stable and could likely exhibit different activities against different targets.... In my opinion, this reflects the divide between organic and 'not' organic chemistry exposure in graduate studies. Though I would be one of those 'not' organic chemists, my background in assay design and screening required many months of investigating active scaffolds - though I believe many people abandon organic chemistry after sophomore undergraduate exposure. Happy to have found this blog (even though it was through searching the net for discussions around the discrepancies in this paper).

Permalink to Comment

53. Bring the Movies on April 11, 2014 2:49 PM writes...

" They argued that the phenylbenzylamine tautomers are equally stable and could likely exhibit different activities against different targets..."

And why not? Is it so unreasonable to think that a macromolecule (like a protein) can stabilize a specific tautomer? The bases in DNA are keto, yet in solution are enol. Not unusual for protein active sites to change the pKa of ionizable functional groups dramatically.

Are you really Ivy league?

Permalink to Comment

54. David Borhani on April 11, 2014 2:53 PM writes...

@52, Becca: Scary indeed. These are chemistry postdocs? Just not *organic* chemistry postdocs? If you were to show me some crazily impossible inorganic structure, I should hope I have at least a 50:50 chance of thinking something seems amiss...

@51, fluorogrol: Agreed, the usefulness of the fragments they identify is near zero in most cases. "1,4-Naphthoquinone" and "cyclohex-2-end-1,4-quinone" (i.e., dihydroquinone) result in copper-dependent oxidative stress, but "p-benzoquinone" and "7,8,9,10-tetrahydrotetracene-5,12-quinone" don't? (Aside: It's "interesting" that both an oxidant and a reductant give this Cu-dependent oxidative stress.)

Out of curiosity, I'm converting their large SMILES list into a format I can more easily look at. I'm a bit interested, for example, in what is that molecule with the phenyl benzylamine tautomer (implies that the fragment must have something like a gem-dimethyl to be stable, no?).

Permalink to Comment

55. Anonymous on April 11, 2014 2:59 PM writes...

Perhaps instead of judging how good/bad papers are, based on the journals they're in, it's time to judge how good/bad the journals are, based on what papers are in them?

Permalink to Comment

56. David Borhani on April 11, 2014 3:10 PM writes...

@53, Bring the Movies: In general, of course, you are correct. But, the actual tautomer energy difference *does* matter. In this case, I think it's on the order of 18 kcal/mol. With appropriate substitution, that energy difference could be lowered.

This is similar to the issue pointed out by @23, Curious Wavefunction: Arsenic esters in DNA may seem more reasonable to those with experience in one field (e.g., geology) than to those in another (e.g., organic chemistry). Tautomerization? Sure...but the energy differences matter.

Permalink to Comment

57. fluorogrol on April 11, 2014 3:10 PM writes...

@54

David, re: the mad tautomers, yes, that's exactly what I was getting at when I mentioned a cyclohexadienone with two groups on the sp3 carbon in my earlier comment.

Permalink to Comment

58. Chemist on April 11, 2014 3:11 PM writes...

@53. Before you start making snarky comments towards other people about where they got their education, please think about what you just said. A benzene ring is not "an ionizable functional group." Aromatic vs. nonaromatic is completely different from keto-enol tautomers. You must be a biologist. Please stop talking, you sound like a dumbass. Where the hell do YOU go to school?

Permalink to Comment

59. a. nonymaus on April 11, 2014 3:15 PM writes...

It's very easy to have imidazole and imidazolium have different biological effects. Just make sure that the imidazolium counter-ion is cyanide :-) At least the biological screening data has been gathered. All the authors need to do now is the analytical chemistry to determine what each library member really is.

Permalink to Comment

60. Bring the Movies on April 11, 2014 3:19 PM writes...

@58: Well, Becca did bring up her education first.

To think that different tautomer stabilization in a different active site is perfectly reasonable, as her co-workers were thinking, possibly in principle. Unusual pKa's of ionizable amino acids in active sites could help this out dramatically.

Get off your high horse.

Permalink to Comment

61. Civility is good, whether you're a biologist or a chemist. on April 11, 2014 3:19 PM writes...

@58. While I agree that @53 was out of line with the "are you really Ivy League," do you not see your hypocrisy in calling him out as a biologist? As if that makes him a second-class citizen and a "dumbass?"

Let's try to keep this civil. If you want to call people names and demean their professions, do it on Youtube.

Permalink to Comment

62. anonymous coward on April 11, 2014 3:24 PM writes...

@53: I don't imagine the keto/enol energy difference is all that big. Aromatic to nonaromatic tautomerization is a lot of energy for something to be ponying up - benzene has on the order of 35 kcal/mol resonance energy, most of which goes away in the tautomer.

Of course biological systems can do a lot things that we can't, but if you're suggesting that that tautomer is the key binding point (rather than the benzylamine), you'd better have some serious evidence. I'm guessing they don't.

Permalink to Comment

63. neutral ground on April 11, 2014 3:27 PM writes...

@61. I have to agree with @58. "Bring the movies" posted the first rude comment and of course there will be retaliation. And come on, "second-class citizen" is overly dramatic. You're also being a little hypocritical in your more one-sided criticism of @58 than @53/60.

This whole post seems to be divided between chemists vs. biologists.

Permalink to Comment

64. amused on April 11, 2014 3:32 PM writes...

While @58 maybe should have used nicer language, both @58 and @62 make excellent points about nonaromatic-aromatic tautomerization energy.

It is amusing how chemists and biologists still hate each other to this day.

Permalink to Comment

65. MoBio on April 11, 2014 3:35 PM writes...

Some very bizarre values in Supplementary Table S3 which lists IC20 and IC50 values for selected compounds in uM.

As you can see several compounds have IC50 values such as: 2.97E+14

That would represent 2.97E+8 Molar concentration (clearly an extrapolation).

Even a quick perusal of Fig S4 by a reviewer with a passing knowledge of pharmacology and drug action would result in elimination of most of the curves as 'useless'.

I also note this curious item:

"For each curve, a fitted line is also shown (sigmoid in most cases, linear in cases where sigmoid fits are poor)."

I'm unaware of any drug-target interaction which yields a 'linear fit'.....

Permalink to Comment

66. Becca on April 11, 2014 3:38 PM writes...

@Bring the Movies: Perhaps I should clarify my point (sorry, english is my second language) - the new generation of chemists (myself included) are not being taught organic chemistry sufficiently in graduate studies. We do some cool experiments and memorize a bunch of mechanisms in undergrad, then forget about it until it comes up 8 years later and we have a big problem with the conclusions of a paper. Perhaps I shouldn't have mentioned the ivy league part but I did because this is supposed to be 'top notch' group of chemists so we should know better - (i.e. when another postdoc tells me both tautomers are equally stable under aqueous conditions) - I didn't intend to have an elitist attitude.

Permalink to Comment

67. Bring the Movies on April 11, 2014 3:40 PM writes...

@Becca- I apologize for my snarky comment. I did get a PhD in Chemistry, not ivy league (2nd tier) and am analyzing qPCR data as we speak. Best of luck in your future studies.

Permalink to Comment

68. anonymous coward on April 11, 2014 4:13 PM writes...

This references papers giving the energy differences in solution and gas phase for nucleotide base tautomers -

http://chemistry.stackexchange.com/questions/89/how-large-is-the-energy-difference-between-keto-and-enol-form-of-guanine-and-ur

I would guess that hydrogen bonding pays for some of the energy (an H-bond could be up to 5 kcal/mol, but probably less - 0.5-5? kcal/mol), but I don't know enough to say. The benzylamine tautomer, however, lacks a lot of the handles to hydrogen bond. It's possible to come up with other sources of energy to pull it off, but those sources probably aren't as common, even in enzymes.

Permalink to Comment

69. David Borhani on April 11, 2014 4:24 PM writes...

OK, some data to add to our now re-civilized discussion. The "mad tautomers" (57 of them) are all actually imines formed by combining (substituted) 2-hydroxybenzaldehydes with (substituted) anilines. Keto-enol tautomerization IS known to be quite relevant for such salicyladimines, and the position of the equilibrium depends on the factors one might expect (pH, temperature, solvent, just what the substituents are and their locations).

A+ to @62, anonymous coward!

So, the authors of this paper have presented an unintelligible portion of the actual "pharmacophore" for molecules that exhibit "ERAD & cell cycle" and/or "RSC & ERG11" signatures. Lovely.

@65, MoBio: Agreed, much of the data in S4 look quite poor, and many of the fits are of very low quality. Again speaks to lack of raw data review, by both the authors and the Science reviewers (and please don't tell me that reviewers shouldn't need to look at the supplemental data).

The shame of all this is, this could be, at root, good work. But the authors appear to be so indiscriminate with the compounds they have used, and so oblivious to the actual data, that finding the diamonds in all this gravel will be very difficult.

Permalink to Comment

70. Malarkey on April 11, 2014 4:30 PM writes...

Hi
Biologist( OK bioinformatics) here. I'm not surprised by this...well perhaps a little surprised after this: http://www.nature.com/nature/journal/v504/n7480/full/nature12831.html

I know nothing about the chemistry you mention but trust you should know. I don't trust any of these high throughput molecular screening assays. Not shRNA, siRNA, kinase assays or drugs. I've seen the data on a couple of projects now and the "normalizations" etc used. When you see how sausages are made it puts you off.

Permalink to Comment

71. homolog.us on April 11, 2014 4:34 PM writes...

Ron Davis writes papers without taking critical look at the claims. You guys probably do not know that he 'discovered' 85% of a genome to be active long before ENCODE.

http://www.homolog.us/blogs/blog/2014/04/11/who-deserves-the-encode-nobel-prize-ron-davis/

Permalink to Comment

72. ortho quinone methide on April 11, 2014 4:57 PM writes...

Looking at the supplementary data, the phenol tautomer is actually an ortho quinone methide substructure. Chemically reasonable perhaps, but not something I would want to put in a screening assay.....

Permalink to Comment

73. Cellbio on April 11, 2014 4:58 PM writes...

As a molecular biologist turned cellular pharmacologist with a perspective that a functional literacy of chemistry is required to adequately speak of pharmacology and data like this generated from library screens, this is a sad day.

One quick question for those of you that have access, is there any QC of the compounds? First lesson I learned was positives are first associated with a well, after rescreen, associated with the chemical contents of the source that filled the well, but only associated with the presumed compound after resynthesis, repurification or confirmation of exceptional purity. This is especially true when finding uM and above hits. Could this explain the finding that the same 'compound' is associated with distinct biology?

Permalink to Comment

74. pharmacog on April 11, 2014 9:03 PM writes...

According to the book "The Chemistry of Phenols", edited by Zvi Rappoport, on page 127 the authors state the pKa of cyclohexa-2,4-dien-1-one was evaluated to be -2.89. Based on acidity considerations alone, there is no way this molecule can exist in neutral form in aqueous solution at rt. The energy difference between phenol and the keto tautomer is 73 kJ/mol which should warm things up a little in the microtitre plate well. Its safe to say the compounds the authors think they bought weren't the active species responsible for the hits they saw in their assays. It reminds me of the days when I was in the lab and a classmate who was new to organic chemistry asked me why the methyl iodide wasn't dissolving in the ether and instead was just staying in shiny flakes floating on the liquid surface. It turns out he spooned in the vermiculite packing material and didn't realize there was a bottle of methyl iodide buried in the can a little further. He was no dummy, he just never encountered these little things in the world of chemistry before.

Permalink to Comment

75. ned on April 11, 2014 9:25 PM writes...

Interesting discussion. It reminds me of this sketch:
https://www.youtube.com/watch?v=zrzMhU_4m-g

Permalink to Comment

76. David Borhani on April 11, 2014 9:30 PM writes...

@73, Cellbio: There's 73 pages of SI, and ~10 SI tables. But as far as I can tell, there doesn't seem to be any analytical data on any of the hits, nor any of the typical "is it the compound I think it is?" validation I or you or any other reasonable pharmacologist would do. Table S1 lists the compounds; no % purity is provided. The quote below is the only thing I can find that even comes close to "validation"...if you can call it that. Everything else is informatics...once it's in the computer, it must be correct, right?

From the SI, p. 7:

Assessing the promiscuity of chemical-genetic probes To roughly assess the promiscuity of chemical-genetic probes, we extracted from PubChem (on December 15, 2013) the number of bioassays in which each probe was tested, and the number of bioassays in which the probe was found to be active. Of the 317 chemical-genetic probes, 175 have been tested in bioassays, and they tend to be active in relatively few bioassays (median = 1.4% of the bioassays in which the probe was tested). For those probes for which we have validation data beyond confirmation of drug-induced haploinsufficiency by individual strain analysis, only 3013-0144 appeared active for the majority of the bioassays in which it was tested (5 of 6 bioassays). Interestingly, the probe was active in a bioassay involving the mammalian polo-like kinase PLK-1 as a target, which co-localizes to the cytokinetic furrow, a structure that shares similarity with the yeast septin ring. This is consistent with data in this study supporting the inhibition of septin by 3013-0144. Taken together, these results suggest that the chemical-genetic probes are not promiscuous or broadly reactive.
Permalink to Comment

77. The googles do nothing on April 12, 2014 2:54 AM writes...

Ah big data. Looking at the example curves in the SI, how high do you have to stack crap data until it no longer stinks?

Permalink to Comment

78. Anonymous on April 12, 2014 7:20 AM writes...

I am a biologist and was willing to accept they got themselves into a bit a bind with the chemistry but then i saw this comment here quoting from the paper:

"For each curve, a fitted line is also shown (sigmoid in most cases, linear in cases where sigmoid fits are poor)."

Eh? Who would even have the idea to do that? No class in University or any pharmacologist I have ever met would suggest such a thing.

And then, the curves. Most of these I would be embarrassed to show at an internal meeting never mind publish in science. Some start at 0, 20 and 50. I don't even know what they are trying to tell me as the fits and error are so poor.

Permalink to Comment

79. Anonymous on April 12, 2014 8:43 AM writes...

To both biologists and chemists: Unless you start working together and actually make something useful with all the money that gets thrown at you, none of you will have a job soon. FYI, I make my living getting rid of unproductive scientists on behalf of investors.

Permalink to Comment

80. scientist on April 12, 2014 9:00 AM writes...

@79 first of all, we don't get $ thrown at us. the average grad student makes $6/h, which is below minimum wage. postdocs are not much better off. only 5-7% of grants get funded. so where is "all the money that gets thrown at us?" also why don't you try doing some science and see how difficult it is yourself before you start calling us unproductive. sure lots of things don't work out, the human system is complicated. but...where would society be without all of the work we have done?! so do something useful with your time other than make it your job to fire a bunch of "unproductive scientists" and posting rude comments on a science blog when you can't contribute any scientific knowledge to our discussions.

Permalink to Comment

81. pharma on April 12, 2014 9:07 AM writes...

@80. well said. @79 once you fire all of us "unproductive scientists," YOU won't have a job anymore either.

Permalink to Comment

82. cliffintokyo on April 12, 2014 9:29 AM writes...

Shoot the authors, and shoot the peer reviewers twice! To make sure none of them can publish any more manurescripts.

Permalink to Comment

83. RayPerkins on April 12, 2014 9:29 AM writes...

I'm reminded of the admonition from grad school days back in the Jurassic: Physicists believe labels on bottles, and Chemists believe settings on dials. Outside our immediate sphere of expertise, all is taken at face value, as though the skepticism we apply to our narrow skill set should not apply to other disciplines. In the practical world of in-specialty peer review, any reference or canned, library citation is sufficient.

Sad but true.

And what to Biologists believe? Both labels on bottles and settings on dials.

Permalink to Comment

84. Anonymous on April 12, 2014 9:44 AM writes...

"@79 first of all, we don't get $ thrown at us. the average grad student makes $6/h, which is below minimum wage. postdocs are not much better off. only 5-7% of grants get funded. so where is "all the money that gets thrown at us?"

Well that's still way more than the money you actually make, so you're a net loss.

"also why don't you try doing some science and see how difficult it is yourself before you start calling us unproductive."

I did, and I know it's difficult. But that doesn't justify throwing more money at things that are no longer paying back. So it's up to you to fix that problem, not me.

"Sure lots of things don't work out, the human system is complicated. but...where would society be without all of the work we have done?!" Well, it seems all that productivity and value created was in the past. Now it's very different and you just can't deliver anymore.

"Do do something useful with your time other than make it your job to fire a bunch of "unproductive scientists" and posting rude comments on a science blog when you can't contribute any scientific knowledge to our discussions."

The value I add is to stop people like you destroying value. So either fix the problem fast, or say goodbye to your jobs. Hah!

Permalink to Comment

85. Ron Richardson on April 12, 2014 10:01 AM writes...

1) This paper appears to be more compounds tested in the same way as Hillenmeyer et al., Science 2008, which tested 1144 compounds, which was simply more compounds than earlier papers from the group. I don't see anything fundamentally new here.

2) The technique of yeast chemical genomic profiling is actually one of the best ways to get an initial idea of the bioactivity of a compound (in yeast)--Novartis uses it in house on all of their compounds showing bioactivity, and has a pretty strong database of 5,000 compounds run through the assay.

3) Regardless of the chemistry being sloppy and the compounds nonspecific, I just don't see what the major new contribution was here. There were no new drug::target pairs identified for which there was no previous compound known to hit the target. There was no great biological insight from the screen. I could see this being published if the method was novel, but it is at least 10 years old now. It seems the the "45 signatures" is the new idea, but it is so poorly and loosely defined what this means and how it is useful that it certainly doesn't merit publication in Science.

Permalink to Comment

86. Wes on April 12, 2014 10:12 AM writes...

in response to #76, their "hits" were validated with PubChem data mining, a database notoriously awash in errors...

Permalink to Comment

87. pharma on April 12, 2014 10:51 AM writes...

@84: "Well, it seems all that productivity and value created was in the past. Now it's very different and you just can't deliver anymore."

Well, for someone who advises investors, you seem woefully ignorant of the current successes in pharma and the promising new drugs. Pharma is in fact improving significantly: big blockbusters to come: (1) Gilead's sovaldi for hep C treatment, (2) roche's GA101, (3) sanofi's lemtrada, (4) johnson and johnson's ibrutinib and hep C drugs. Gilead's PI3K inhibitors and many others by other drug companies are all very exciting and promising.

Permalink to Comment

88. Anonymous on April 12, 2014 10:55 AM writes...

@87: Talk about cherry picking the data points you want. I do the same, but with people, so what's the difference?

Permalink to Comment

89. pharma on April 12, 2014 11:06 AM writes...

@88. how is it cherry-picking when many big pharma companies have blockbuster drugs coming out? I merely listed a few, and there are many more.

also, i was not discussing your profession of firing people. i was addressing your statement that pharma "can't deliver anymore."

Permalink to Comment

90. Anothercommentator on April 12, 2014 11:18 AM writes...

Derek said "Try pulling some SMILES strings from that table yourself and see what you think about their drug likeness."

I did that, and I don't think they're drug-like either, but the compounds whose structures are shown were selected from the 50k library because they were hits in the yeast screen. I'm not surprised, because I've come to expect that hits in cell-based screening papers will look ugly and reactive, especially ones where growth inhibition is the end-point.


Permalink to Comment

91. Anonymous on April 12, 2014 11:30 AM writes...

@89: Well, pharma might be delivering, but is delivering less value than we put in.

Permalink to Comment

92. Anothercommentator on April 12, 2014 11:36 AM writes...

@84. You're trolling and off-topic on this post. We were discussing a paper from a gang of academics, and none of them is likely to be much interested in profits.

Permalink to Comment

93. Anonymous on April 12, 2014 11:42 AM writes...

@92: No, I'm watching a bunch of biologists and chemists argue with each other about how the other is incompetent, when actually you all are.

Permalink to Comment

94. Jeem on April 12, 2014 12:04 PM writes...

Perfect example for the saying "Just because it's in Science doesn't necessarily mean it's wrong"

Permalink to Comment

95. chem on April 12, 2014 12:06 PM writes...

@92. let's all ignore @84/93. clearly he has nothing better to do on a saturday morning than go onto a science blog and tell all of us we're all going to lose our jobs and we're all incompetent. the more attention we give him, the more trolling he'll do. clearly is angry and bitter about something. hopefully derek will see this and block him.

now we can resume our scientific discussion again!

Permalink to Comment

96. Cellbio on April 12, 2014 12:06 PM writes...

93 is certainly a troll, but I'lll give it a go. The bulk of biologists and chemists I know compliment each other well and work wonders, regrettably not as often as we'd like, to bring new medicines into practice. There are however, real bad examples where people don't reach across the aisle to expertise they need but rather presume they can plug in info that serves their story. As a biologist, I see this is most commonly associated with biologists' who view everything from the lens of their paradigms and models whereas more physically oriented scientist strive for a more complete understanding of the real world. This is more true in industry than academia, where in one a story sells and raises money and in the other great sums of money are required to find the truth. The clinic does not lie.

Permalink to Comment

97. Anothercommentator on April 12, 2014 12:19 PM writes...

@93. Well, I wasn't one of the ones arguing, and most chemists and biologists aren't incompetent.

Since you like to generalize, though, and it's unity between chemists and biologists that you want, then most of us probably agree that the organizations we work for are hamstrung by listening too much to know-nothing trend-following investment advisors.

Permalink to Comment

98. Anonymous on April 12, 2014 12:23 PM writes...

This reminds me two papers by Wells:
Science 2009, 326, 853-858.
J. Am. Chem. Soc. 2011, 133, 19630–19633.

Permalink to Comment

99. Anonymous on April 12, 2014 12:24 PM writes...

Good, that's more like it, now start working together and fix science so that it delivers once again.

Permalink to Comment

100. Anonymous on April 12, 2014 12:35 PM writes...

If chemistry is wrong, biology is in doubt.

Permalink to Comment

101. Anonymous on April 12, 2014 2:14 PM writes...

@98:

How does this remind you of the Wells papers? What are the similarities?

Permalink to Comment

102. Pinko Punko on April 12, 2014 2:29 PM writes...

I wonder if the 3rd update should be more prominent (maybe at the top of the post). I also will attempt to read through the paper before confirming my biases. Some (certainly not all) of the above thread reads as if that step was skipped. Given what else has been published in this field, the newness of this paper is in question or how much exactly it is an advance. Certainly a greater contribution from chemists would likely have improved the paper.

Permalink to Comment

103. noah sark on April 12, 2014 2:53 PM writes...

@102 After you read the paper, it would be great to share what exactly a chemist would have contributed to improve the paper. Would it just be a comment about how to draw molecular fragments so that chemists will understand them, would it be simple filtering of the data, or would it be interesting experiments that improve the novelty of the work. Details such as these would likely help biologists understand what chemists could contribute to their work and may encourage a more productive discussion.

Permalink to Comment

104. Pinko Punko on April 12, 2014 3:15 PM writes...

Well, I'm not a chemist. On the biology side, at some basic level, whether the compounds look like good drugs or not is beside the point in understanding patterns of bioactivity or creating a compendium of such profiles. At the first level a ton of random compounds are just perturbations that give profiles and the spectrum of responsiveness over the set of genotypes is the framework to understand mechanisms of actions of possibly more interesting compounds. However, it would be valuable to more explicitly know which of the compounds are more druggy than others- but this allows the paper to serve a greater audience. The paper is a little bit of an avalanche, so will take some time to parse.

Permalink to Comment

105. sgcox on April 12, 2014 3:46 PM writes...

Pinko,
English is not my mother tongue but somehow I feel my failure to comprehend the following sentence has nothing to do with it:

"At the first level a ton of random compounds are just perturbations that give profiles and the spectrum of responsiveness over the set of genotypes is the framework to understand mechanisms of actions of possibly more interesting compounds."

Permalink to Comment

106. Albert on April 12, 2014 3:50 PM writes...

@93 Apparently you fired the wrong one since science does not pay off anymore...

Permalink to Comment

107. elvenc on April 12, 2014 4:08 PM writes...

First of all I would like to thank you for a very interesting post! Much obliged. Secondly I think that in studies that screen molecules without any real regard for the molecule in question should first and foremost do some basic work. Such as titration etc. But if most of these molecules are bought there really is no excuse. It is common in biology related papers to do similar things. My organic chemistry knowledge has dwindled over the years but from many arguments I had with my peers people usually just take a compound they bought use it at any given solution and tadaa we got our significant result. Sometimes science is a pain.

Permalink to Comment

108. elvenc on April 12, 2014 4:09 PM writes...

First of all I would like to thank you for a very interesting post! Much obliged. Secondly I think that in studies that screen molecules without any real regard for the molecule in question should first and foremost do some basic work. Such as titration etc. But if most of these molecules are bought there really is no excuse. It is common in biology related papers to do similar things. My organic chemistry knowledge has dwindled over the years but from many arguments I had with my peers people usually just take a compound they bought use it at any given solution and tadaa we got our significant result. Sometimes science is a pain.

Permalink to Comment

109. Pinko Punko on April 12, 2014 4:26 PM writes...

@105. I mean that compounds are like probes or conditions. Responses to different conditions can be valuable even if the nature of the probe or experimental condition is unclear. Once you a large number of possible responses characterized you can compare other compounds to that set of responses to group compounds based on their elicited responses.

Permalink to Comment

110. Anonymous on April 12, 2014 4:36 PM writes...

@103/Pinko Punko:

As a chemical biologist (no one shoot me), I think the issues chemists have arise from two aspects of the paper.

1. The compounds are used at very high concentrations and structurally do not give the impression of being very selective. Many of the compounds are highly reactive (like the one Derek highlighted and the ortho-quinone methides someone else mentioned earlier). While the authors note that yeast have better xenobiotic defenses than mammalian cells, the high concentration of compounds used will always be viewed suspiciously (who knows if they start crashing out, or if all drugs are pumped out equally).

2. Figure 2. If a chemist cannot understand the structures drawn, who can? More importantly, the "chemical moieties" may indeed be enriched in the particular compounds that score (many of which are a series of closely related analogs), but this is constricted by the library and many of these substructures are ridiculously simple; it's very implausible to believe that tetrahydrofuran or succinimide is somehow responsible for targeting tubulin. Maybe the authors did not intend to imply this, but the figure certainly does it for them. Also as David Borhani mentioned above, the insinuation that the very basic 5-fluoropyrimidine substructure in 5-fluorouridine and 4215-0184 are the reason for their similar profiles is highly unlikely since they behave very differently (Figure 3).

I think the biological idea behind the paper is interesting. However, I think a chemist could have contributed in many ways. Your compendium of bioactivity/profiles depends on the quality of your reagents. As a biologist, I/you wouldn't engage in some large shRNA screen using poorly targeting constructs and try to draw a lot of big conclusions from it. I think a chemist could have helped interpret the results/filter data and review the manuscript as well to avoid poor/incorrect structures and the structural/bioactivity implications that come across as very naive.

Permalink to Comment

111. Pinko Punko on April 12, 2014 6:09 PM writes...

I agree with much of @110 but note that many comments above seemed to denounce the paper without even reading or understanding what was in figure 2. In terms of phenotypic profiling, as a geneticist I recognize value in data from compounds that might be suboptimal. The value there would be from additional profiles in the compendium. The purpose of an shRNA library screen would be somewhat different I think in most cases- most likely to identify knock downs that give a particular phenotype instead of using diversity of phenotypic responses to understand something about biology and then using that biology to understand compounds. There will be some fraction of garbage compounds but the profiles are useful beyond those compounds. Certainly having an idea if what fraction of the compound library you really understand would be good. Using some compounds at high concentration is an issue in some senses but not others. The treatment either elicits a response or not. If there is any specificity to the response it can still have value for the compendium. If it hits lots of targets that is still a useful profile because you don't want compounds that elicit similar responses. I will try to sit down with the paper to see exactly how they have sold what they are trying to achieve. I'll hold my response on whether the authors are complete buffoons or maybe their presentation was just suboptimal or oversold (which would almost be expected because that is what Science asks for) when I really get into it.

Permalink to Comment

112. another geneticist on April 12, 2014 6:56 PM writes...

As a biologist, I have no forgiveness here. This looks like nothing more than scientific malpractice committed for the sake of getting another Science publication (and another million dollars of NIH funding). These people either got in over their heads or stretched their attention too thin. They are more concerned with hype than with rigorous science.

This isn't the first time that a high-status scientist put his name on a BS paper when he was not involved enough with the research to actually know if it was legit or not.

Permalink to Comment

113. SomeoneWhoActuallyReadThePaper on April 12, 2014 9:42 PM writes...

I'm pretty sure most of you haven't read the paper which makes me sad indeed for the state of science, that so many folks are just getting on a blind bashing train.

After actually reading the paper, you would probably note that this is actual pretty cool - the whole point is to study a wide variety of compounds against a whole genome, not to quality control every single compound.

Sorry to chemists who are insulted by this, or the fact that figure two was probably driven entirely from computational analysis.

Permalink to Comment

114. David Borhani on April 12, 2014 11:22 PM writes...

@86, Wes: In case it wasn't clear, t wasn't defending their "validation"; rather, I was pointing out how ludicrous and inadequate it is.

@114, SomeoneWhoActuallyReadThePaper: So did I. As other have pointed out, the work is not fundamentally new, it does not reveal any new or unexpected biology, and it is based on biological responses to (usually) massive concentrations of largely non-selective and/or reactive compounds. The data are of mostly poor quality, and even simple fits to those data have been done in questionable ways, leaving one in doubt to the validity and reliability of the more complicated statistical measures used to extract the "signatures".

In short, this seems like Garbage (GIGO=Garbage In, Garbage Out; and recycled garbage, at that) that should not have been published by Science.


Permalink to Comment

115. James Kress on April 13, 2014 1:33 PM writes...

The Chemistry in this article is wrong. Obviously the "peer review" processes failed. The paper should be withdrawn, the Chemistry fixed, the Biology redone and then it should be resubmitted for review by Biologists AND Organic Chemists.

Permalink to Comment

116. Scott on April 13, 2014 6:56 PM writes...

@113: "After actually reading the paper, you would probably note that this is actual pretty cool - the whole point is to study a wide variety of compounds against a whole genome, not to quality control every single compound."

What's the point of having a wide variety of compounds studied if you don't know what the compounds being studied actually are?!?

Look, I'm a HR major. No Masters, no PhD. (I normally check the blog about once a month to see if there's some new chemical Derek doesn't want to work with, or if he has a new tale of someone blowing up lab equipment.)

But even *I* know that the first rule of statistics (for getting usable data) is to compare like with like.

All of these chemicals should have been tested at identical concentrations, just for a baseline of comparison. In order to get identical concentrations, I'd have been tempted to abuse a platoon of graduate assistants, or maybe even undergrads, to go through and determine what exactly was in each of those containers. Hey, it'd be a good practical lab for the entire chemistry department, so bonus for teaching things! (Again, I'm an HR major, but guess who 'owns' most company training departments?)

Oh, yeah, we'd also need to verify whatever the various chemicals are dissolved in (if any), to make sure that we weren't getting any false positives due to the solvent instead of the chemical we thought we were testing.

If I was attempting to broadly classify a whole bunch of chemicals for their bioactivity, I'd also want to run a wide range of concentrations. I have constant pain courtesy of an old back injury, so I'm well aware of the fine line between the *optimum* dose of acetaminophen and the *lethal* dose. I would suspect that many of the chemicals in the library list may have a certain minimum concentration before they showed any bioactivity, and would want to document that, too.

So, now that we're dealing with truly known chemicals in concentrations that we know are accurate, diluted by a known chemical, we have a ginorumous data set. Each chemical has a bioactivity at concentration table, for each of the different test yeasts. Then we can cross-reference the various chemicals to see if there are structures in common with the various yeasts and their bioactivity.

Otherwise, your entire study is delivering about as much useful data as something out of the Marketing department. And I think everyone knows that the largest source of "organic bovine fertilizer" is the Marketing department...

Permalink to Comment

117. David Borhani on April 13, 2014 11:14 PM writes...

@116, Scott. HR major? You sound like you could apply for a job at my workplace! Go dude!

Permalink to Comment

118. Pinko Punko on April 13, 2014 11:16 PM writes...

@116, this is not necessarily the case. As I state above, and is likely the vantage point of the authors in some respect, profiles are valuable regardless of knowledge about all the compounds. For compounds you wish to make similarity inferences for (and the get an idea if the chemical moiety approach has any value), obviously you need to know more about that subset and presumably would desire higher validation. I think the authors would probably view this approach as a mechanism for generating hypotheses that would then get tested traditionally.

Permalink to Comment

119. nICK k on April 13, 2014 11:35 PM writes...

#116 Scott: Excellent post. You're a better scientist than the clowns who published this crap.

#118: What is a "profile"? Your post makes no sense. Re-read #116.

Permalink to Comment

120. T on April 14, 2014 4:42 AM writes...

I'm a bit late to the party here but I have a question regarding the "the place of organic chemists as reviewers" of this kind of thing. All the busy organic chemists, if you were sent this paper to review, would you accept and take the time to look at their structures or would you read the title and say "There is no new chemistry in here, just applying known compounds to bio stuff: I'm not interested/qualified"? If the latter, what should editors do to fix the problem? Not a rhetorixcal question, I'm genuinely interested.

Permalink to Comment

121. SAR Screener on April 14, 2014 5:53 AM writes...

@118
If you don't know what's in the wells you’re testing then you're not generating any useful hypothesis and could well end up sending a lot of people who are trying to replicate your work up a blind alley (what do you mean the sample was only active due to the presence of a metal ion contaminant…?)

I have some reservations about the curves as well as those mentioned in post 78 about fitting to a straight line. Quite a few of the active cps in the supplementary materials seem to have slopes much greater than 1, which usually triggers alarm bells.

Permalink to Comment

122. David Borhani on April 14, 2014 8:29 AM writes...

@120, T: I would have reviewed it and recommended rejection on the basis of my earlier posts. I would have left the door open to the Authors coming back with suitable analytical and other supporting information. That said, the overall low quality of the raw data, the vague and perhaps overly broad "signature" concept, and the inappropriate extrapolations from one chemical reactivity class to another (cf. 5-FU does not equal a 5-fluoro-2-halopyrimidine), even if compound identity and purity were verified, would likely have resulted in a rejection in my view.

Maybe it's worth pointing out that a lot of organic and medicinal chemists who have the (trivial) expertise to serve as reviewers for a paper such as this are working at the interface of "pure" chemistry and biology or medicine. Reviewing a paper like this one is par for the course.

Permalink to Comment

123. Curt F. on April 14, 2014 8:31 AM writes...

@121. SAR Screener and @116. Scott.

Of course identifying the compounds tested is better than not identifying them, but Pinko Punko is right that there is value in knowing how response of the library of single-gene knockouts respond to diverse stressors even if the identity of the stressors is not known precisely.

Think of unknown compounds ("stressors") as columns of a matrix. And think of the genes in the single-gene knockout collection as rows. Then the matrix elements are 1s if the gene knockout i grows in the presence of compound j and zero otherwise. When people talk about two "profiles" being similar they mean that for two genes i and i', their responses across all columns are similar -- i.e. where i,j is 1, i', j likely to be 1 and where i, j is 0 i', j is likely to be zero as well.

The matrix contains information regardless of whether the identity of the stressors is known.

Amazon uses somewhat similar techniques all the time to decide what items to show you in the "Customers who bought X also bought Y" areas of a product page. Their algorithms (at least, the foundational ones) are agnostic to whether X is a book and Y is a toaster.

Now, all that isn't an attempt to apologize for the paper in any way. It does seem like a mess. The point is that your criticisms are overwrought.

Permalink to Comment

124. Cellbio on April 14, 2014 9:03 AM writes...

Curt,

I don't agree that the criticisms are overwrought. While for sure one can make signatures, the missing part for me is earlier in your post. What is the magnitude of value you assume when compiling cell response to unknowns at high concentrations?

As someone who has done this kind of work, measuring 26,000 compounds against 22 biological measures in primary human immune cells, signatures are easy. And plentiful! The subsequent and lasting value is when these perturbations are biologically interesting (not reactive or 'active' at solubility limits), by a known agent that can be resynthesized anywhere and be active, and not a one-off so one has flexibility in SAR to make a tool (add handles or labels or pharmacology properties) or make a drug.

It strikes me that the question of whether some single gene KOs respond differently to "stressors" is uninteresting as the answer is clear a priori. Therefore, populating a data table and annotating with signatures is window dressing. For me, the whole thing is pharmacology, which demands great control of source material and great caution in interpretation.

Permalink to Comment

125. anonymous coward on April 14, 2014 9:12 AM writes...

But isn't the point of the paper to correlate types of structural features and resultant types of stress in yeast and the corresponding genes that respond to or are affected by the stress? If you don't know the structural and thus chemical features that cause the stress, then there isn't a whole lot of helpful information present.

It seems like trying to draw a topographical map of your backyard with just the heights/depths wrt standard but without positions correlated to them - it may be useful to know that your backyard varies in height by five feet but it doesn't really tell you anything you can use to draw a map or do much else - without knowing where those variations are, you can't do anything. It might be useful for something (though I'm not sure what), but it doesn't seem like a Science paper.

Permalink to Comment

126. synthesis on April 14, 2014 9:35 AM writes...

Probably, Organic Chemistry and Organic Synthesis is overlooked by a lot of these journals, reviewers, and even by the funding agencies. It is a difficult area of all the sciences because it requires characterization data and proof of what ever the claims that one makes- comparing with the nano-science and any other fields.

Even the supplier of these libraries of compounds do not know what is being supplied in the vials. We had a biologist approach us to make one of the hit compounds that had identified after screening 30,0000 chemicals. He showed us the structure, NMR data, and we made the compound for him. However, it was not active though the NMR data matched. It was supposed to conjugated chalcone. Thy did not have any sample left, except for DMSO stock solution made of some micrograms. So, we just checked the UV absorption of the solution and found that it was not a conjugated compound. So, what fun is to fund such studies, and end up with nothing to show for. This probably speaks to the disastrous situation that pharma industry is in.

Traditionally, you would hire bunch of chemists, and have them systematically modify the scaffolds and identify something tangible for the time and efforts spent and learn something meaningful than as some body pointed out: "garbage in and garbage out"

I think it is time to realize where the focus should be.


Permalink to Comment

127. Sylvia on April 14, 2014 2:07 PM writes...

As 27. MoMo says...."only the fragments from the identified series responsible for bioactivity" - I want to go with that. No serious reviewer would let that slip through (I did not read the publication to cofirm).

But...in general - everyone who works with screening libraries should actually use proper software to organize their data. Chemical registration/compound registration software that is. Chemical registration today can also deal with a great number of tautomers and save scientists some embarassing moments.....

Permalink to Comment

128. marta on April 15, 2014 8:41 AM writes...

I don't understand why THF or imidazole would be included in library for screening.

Permalink to Comment

129. Scott on April 16, 2014 1:03 AM writes...

Again, I'm not a chemist or even a biologist, but I don't see how identifying a specific reaction to unknown stressors counts as usable research.

Here, I'll use a real example from marketing: apparently, when people make a small purchase of baby products like diapers or formula, there's often a purchase of beer involved. Assumption: That the husband is making an emergency run for baby supplies, and while he's out grabs beer.

Correlating question prompted by research: can we increase impulse-buys of beer by putting baby products next to the beer cooler (or by putting a beer end-cap display onto the baby products aisle)?

Re-arrange the store at next regular rotation, and you discover that yes, your sales of 'guy products' like beer have increased by some percentage that I don't remember off the top of my head.

But the research in this study is more like saying, "when someone buys diapers, they buy other stuff too."

That's information, but it's not particularly useful data.

I cannot say that yeast i' is or is not reacting to exposure to chemicals sharing a specific "key structure" without knowing what the heck all those chemicals (and the entire structure they have in common) actually are.

As synthesis (#126) tells us, just because it's what the maker said was in the vial doesn't make it what the yeast was reacting to!

Permalink to Comment

130. Jamal on April 25, 2014 6:12 PM writes...

What a horrible paper....OMG this is an absolute embarassment to the state of organic chemistry! We must advocate to have this paper retracted or else we will disrespect the chemgods. Lets all petition for a retraction!!! Lets get people on retraction watch to ridicule this paper!

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
Employment Among New Chemistry PhDs
My Imaginary Friends Would Be Glad to Serve as Referees
Biopharma Stock Events for the Rest of the Year
A Drug Candidate from NCATS
No Scripps/USC
Studies Show? Not So Fast.
Outsourced Assays, Now a Cause For Wonder?
An Alzheimer's Blood Test? Not So Fast.