About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Biology Maybe Right, Chemistry Ridiculously Wrong | Main | Sweet Reason Lands On Its Face »

April 14, 2014

More on the Science Chemogenomic Signatures Paper

Email This Entry

Posted by Derek

This will be a long one. I'm going to take another look at the Science paper that stirred up so much comment here on Friday. In that post, my first objection (but certainly not my only one) was the chemical structures shown in the paper's Figure 2. A number of them are basically impossible, and I just could not imagine how this got through any sort of refereeing process. There is, for example, a cyclohexadien-one structure, shown at left, and that one just doesn't exist as such - it's phenol, and those equilibrium arrows, though very imbalanced, are still not drawn to scale.
Well, that problem is solved by those structures being intended as fragments, substructures of other molecules. But I'm still positive that no organic chemist was involved in putting that figure together, or in reviewing it, because the reason that I was confused (and many other chemists were as well) is that no one who knows organic chemistry draws substructures like this. What you want to do is put dashed bonds in there, or R groups, as shown. That does two things: it shows that you're talking about a whole class of compounds, not just the structure shown, and it also shows where things are substituted. Now, on that cyclohexadienone, there's not much doubt where it's substituted, once you realize that someone actually intended it to be a fragment. It can't exist unless that carbon is tied up, either with two R groups (as shown), or with an exo-alkene, in which case you have a class of compounds called quinone methides. We'll return to those in a bit, but first, another word about substructures and R groups.
Figure 2 also has many structures in it where the fragment structure, as drawn, is a perfectly reasonable molecule (unlike the example above). Tetrahydrofuran and imidazole appear, and there's certainly nothing wrong with either of those. But if you're going to refer to those as common fragments, leading to common effects, you have to specify where they're substituted, because that can make a world of difference. If you still want to say that they can be substituted at different points, then you can draw a THF, for example, with a "floating" R group as shown at left. That's OK, and anyone who knows organic chemistry will understand what you mean by it. If you just draw THF, though, then an organic chemist will understand that to mean just plain old THF, and thus the misunderstanding.

If the problems with this paper ended at the level of structure drawing, which many people will no doubt see as just a minor aesthetic point, then I'd be apologizing right now. Update: although it is irritating. On Twitter, I just saw that someone spotted "dihydrophyranone" on this figure, which someone figured was close enough to "dihydropyranone", I guess, and anyway, it's just chemistry. But they don't. It struck me when I first saw this work that sloppiness in organic chemistry might be symptomatic of deeper trouble, and I think that's the case. The problems just keep on coming. Let's start with those THF and imidazole rings. They're in Figure 2 because they're supposed to be substructures that lead to some consistent pathway activity in the paper's huge (and impressive) yeast screening effort. But what we're talking about is a pharmacophore, to use a term from medicinal chemistry, and just "imidazole" by itself is too small a structure, from a library of 3200 compounds, to be a likely pharmacophore. Particularly when you're not even specifying where it's substituted and how. There are all kinds of imidazole out there, and they do all kinds of things.
So just how many imidazoles are in the library, and how many caused this particular signature? I think I've found them all. Shown at left are the four imidazoles (and there are only four) that exhibit the activity shown in Figure 2 (ergosterol depletion / effects on membrane). Note that all four of them are known antifungals - which makes sense, given that the compounds were chosen for the their ability to inhibit the growth of yeast, and topical antifungals will indeed do that for you. And that phenotype is exactly what you'd expect from miconazole, et al., because that's their known mechanism of action: they mess up the synthesis of ergosterol, which is an essential part of the fungal cell membrane. It would be quite worrisome if these compounds didn't show up under that heading. (Note that miconazole is on the list twice).
But note that there are nine other imidazoles that don't have that same response signature at all - and I didn't even count the benzimidazoles, and there are many, although from that structure in Figure 2, who's to say that they shouldn't be included? What I'm saying here is that imidazole by itself is not enough. A majority of the imidazoles in this screen actually don't get binned this way. You shouldn't look at a compound's structure, see that it has an imidazole, and then decide by looking at Figure 2 that it's therefore probably going to deplete ergosterol and lead to membrane effects. (Keep in mind that those membrane effects probably aren't going to show up in mammalian cells, anyway, since we don't use ergosterol that way).

There are other imidazole-containing antifungals on the list that are not marked down for "ergosterol depletion / effects on membrane". Ketonconazole is SGTC_217 and 1066, and one of those runs gets this designation, while the other one gets signature 118. Both bifonazole and sertaconazole also inhibit the production of ergosterol - although, to be fair, bifonazole does it by a different mechanism. It gets annotated as Response Signature 19, one of the minor ones, while sertaconazole gets marked down for "plasma membrane distress". That's OK, though, because it's known to have a direct effect on fungal membranes separate from its ergosterol-depleting one, so it's believable that it ends up in a different category. But there are plenty of other antifungals on this list, some containing imidazoles and some containing triazoles, whose mechanism of action is also known to be ergosterol depletion. Fluconazole, for example, is SGTC_227, 1787 and 1788, and that's how it works. But its signature is listed as "Iron homeostasis" once and "azole and statin" twice. Itraconzole is SGTC_1076, and it's also annotated as Response Signature 19. Voriconazole is SGTC_1084, and it's down as "azole and statin". Climbazole is SGTC_2777, and it's marked as "iron homeostasis" as well. This scattering of known drugs between different categories is possibly and indicator of this screen's ability to differentiate them, or possibly an indicator of its inherent limitations.

Now we get to another big problem, the imidazolium at the bottom of Figure 2. It is, as I said on Friday, completely nuts to assign a protonated imidazole to a different category than a nonprotonated one. Note that several of the imidazole-containing compounds mentioned above are already protonated salts - they, in fact, fit the imidazolium structure drawn, rather than the imidazole one that they're assigned to. This mistake alone makes Figure 2 very problematic indeed. If the paper was, in fact, talking about protonated imidazoles (which, again, is what the authors have drawn) it would be enough to immediately call into question the whole thing, because a protonated imidazole is the same as a regular imidazole when you put it into a buffered system. In fact, if you go through the list, you find that what they're actually talking about are N-alkylimidazoliums, so the structure at the bottom of FIgure 2 is wrong, and misleading. There are two compounds on the list with this signature, in case you were wondering, but the annotation may well be accurate, because some long-chain alkylimidazolium compounds (such as ionic liquid components) are already known to cause mitochondrial depolarization.

But there are several other alkylimidazolium compounds in the set (which is a bit odd, since they're not exactly drug-like). And they're not assigned to the mitochondrial distress phenotype, as Figure 2 would have you think. SGTC_1247, 179, 193, 1991, 327, and 547 all have this moeity, and they scatter between several other categories. Once again, a majority of compounds with the Figure 2 substructure don't actually map to the phenotype shown (while plenty of other structural types do). What use, exactly, is Figure 2 supposed to be?

Let's turn to some other structures in it. The impossible/implausible ones, as mentioned above, turn out to be that way because they're supposed to have substituents on them. But look around - adamantane is on there. To put it as kindly as possible, adamantane itself is not much of a pharmacophore, having nothing going for it but an odd size and shape for grease. Tetrahydrofuran (THF) is on there, too, and similar objections apply. When attempts have been made to rank the sorts of functional groups that are likely to interact with protein binding sites, ethers always come out poorly. THF by itself is not some sort of key structural unit; highlighting it as one here is, for a medicinal chemist, distinctly weird.

What's also weird is when I search for THF-containing compounds that show this activity signature, I can't find much. The only things with a THF ring in them seem to be SGTC_2563 (the complex natural product tomatine) and SGTC_3239, and neither one of them is marked with the signature shown. There are some imbedded THF rings as in the other structural fragments shown (the succinimide-derived Diels-Alder ones), but no other THFs - and as mentioned, it's truly unlikely that the ether is the key thing about these compounds, anyway. If anyone finds another THF compound annotated for tubulin folding, I'll correct this post immediately, but for now, I can't seem to track one down, even though Table S4 says that there are 65 of them. Again, what exactly is Figure 2 supposed to be telling anyone?

Now we come to some even larger concerns. The supplementary material for the paper says that 95% of the compounds on the list are "drug-like" and were filtered by the commercial suppliers to eliminate reactive compounds. They do caution that different people have different cutoffs for this sort of thing, and boy, do they ever. There are many, many compounds in this collection that I would not have bothered putting into a cell assay, for fear of hitting too many things and generating uninterpretable data. Quinone methides are a good example - as mentioned before, they're in this set. Rhodanines and similar scaffolds are well represented, and are well known to hit all over the place. Some of these things are tested at hundreds of micromolar.

I recognize that one aim of a study like this is to stress the cells by any means necessary and see what happens, but even with that in mind, I think fewer nasty compounds could have been used, and might have given cleaner data. The curves seen in the supplementary data are often, well, ugly. See the comments section from the Friday post on that, but I would be wary of interpreting many of them myself.
There's another problem with these compounds, which might very well have also led to the nastiness of the assay curves. As mentioned on Friday, how can anyone expect many of these compounds to actually be soluble at the levels shown? I've shown a selection of them here; I could go on. I just don't see any way that these compounds can be realistically assayed at these levels. Visual inspection of the wells would surely show cloudy gunk all over the place. Again, how are such assays to be interpreted?

And one final point, although it's a big one. Compound purity. Anyone who's ever ordered three thousand compounds from commercial and public collections will know, will be absolutely certain that they will not all be what they say on the label. There will be many colors and consistencies, and LC/MS checks will show many peaks for some of these. There's no way around it; that's how it is when you buy compounds. I can find no evidence in the paper or its supplementary files that any compound purity assays were undertaken at any point. This is not just bad procedure; this is something that would have caused me to reject the paper all by itself had I refereed it. This is yet another sign that no one who's used to dealing with medicinal chemistry worked on this project. No one with any experience would just bung in three thousand compounds like this and report the results as if they're all real. The hits in an assay like this, by the way, are likely to be enriched in crap, making this more of an issue than ever.

Damn it, I hate to be so hard on so many people who did so much work. But wasn't there a chemist anywhere in the room at any point?

Comments (39) + TrackBacks (0) | Category: Biological News | Chemical Biology | Chemical News | The Scientific Literature


1. weirdo on April 14, 2014 10:01 AM writes...

Everyone associated with this paper -- the authors, the editors, the reviewers -- should feel wholly embarrassed this morning.

I suspect many of them will not, but they should.

Permalink to Comment

2. Anonymous on April 14, 2014 10:06 AM writes...

"Damn it, I hate to be so hard on so many people who did so much work. But wasn't there a chemist anywhere in the room at any point?"

Rhetorical question?

Permalink to Comment

3. PharmaHeretic on April 14, 2014 10:14 AM writes...

Is it really surprising that a highly visible effort to use big data-type techniques in biomedical research was based in fancy technical showmanship rather than accuracy or substance? Isn't that rather typical of a lot of research from large groups in supposedly elite institutions?

Academic-types have now internalized the snake-oil based techniques of businessmen and MBAs. This particular group just had the misfortune of getting caught, otherwise this publication would have been one more high impact paper in support of their next round of grant applications.

Permalink to Comment

4. Justin Peukon on April 14, 2014 10:20 AM writes...

Most of the concerns regarding this paper and quoted in this blog were almost certainly raised during the review process. The key point is what authors did with these reports. I can imagine their reaction: "ooooh noooo!!! They have not understood our Fig. 2 and are asking FOR A NEW ONE!!! Waste of time, pure waste of f...time, and we have NO TIME ALLOWED". The "revised" paper is then submitted with original Fig. 2. Obviously, Science staff doesn’t check (N-O T-I-M-E), and the paper is published.
Science, if you read my post: please release the full set of unedited reviews for this paper.

Permalink to Comment

5. Puff the Mutant Dragon on April 14, 2014 10:27 AM writes...

This is the kind of thing you should expect to see more of if academic labs start doing more "drug discovery" work and "translational research" as has been advocated elsewhere.

Permalink to Comment

6. Am I Lloyd peptide on April 14, 2014 10:27 AM writes...

And people still wonder if open peer review would be a good idea.

Permalink to Comment

7. fluorogrol on April 14, 2014 10:30 AM writes...

Good analysis.

Although the eye-catching impossible structures aren't the major problem with the paper, they're an indicator of the lack of chemical understanding that seems to have led to the real failings.

Permalink to Comment

8. ScienceMag on April 14, 2014 11:39 AM writes...

Justin: This is science. I read your comment. I will snapchat u the deets asap

Permalink to Comment

9. RTW on April 14, 2014 11:39 AM writes...

Likely the lone chemist in the group was someone either newly minted/anointed and little actual experience as a medicinal chemist yet and was too timid about is doubts, or a old 20+ years experienced chemist with a BS/MS degree that no one would listen to because he didn't have a PHD! I have seen many supposed PHD chemists dong Computer Aided Drug Design draw such nonsense and proposed them to us bench chemists to synthesize so this is no surprise to me.

Permalink to Comment

10. ESIMS on April 14, 2014 11:43 AM writes...

Has nothing to do with academic labs, they can also run proper controls to "exclude" off-target effects.

In some kind of way related remark:
1% DMSO is not good idea for yeasts (vacuole & osmotic stress effects) and 0.5% for cell lines is again far from optimal.

Permalink to Comment

11. entropyGain on April 14, 2014 12:12 PM writes...

Maybe this is our fault.

By our fault, I mean professional drug hunters who have years of experience screening, dealing with libraries, SAR, off-target activities, ideopathic tox and generally fighting Murphy's law in ways that academic labs have never conceived of previously. So why our fault? Because as a group we don't publish enough of our work. Too often we allow the "competitive" issues or corporate priorities innappropriately outweigh proper scholarship. By not publishing, we leave an opening for naive academic labs to stumble into the same problems then excitedly publish their findings. Different value systems.

I'm not making an excuse for this particular case, which is egregious in its careless disregard for the fundamentals of basic organic chemistry, but for the larger set of examples that find their way to this blog repeating mistakes many of us already know.

It is also partially our fault because we as a group don't value "service" to the scientific community as highly as if our tenure depended upon it. By that I mean as reviewers of grants and manuscripts. It's incredibly hard to find good reviewers willing to donate the time and energy required. That means that study sections and editorial boards are packed with academics that have little background in real drug discovery. No wonder crap like this gets funded and published.

What can we do? Publish and serve as reviewers.
Sounds simple and will probably make a difference, but believe me though, I understand how hard that is for most of us in most of the situations we find ourselves in this industry.

Permalink to Comment

12. PorkPieHat on April 14, 2014 12:29 PM writes...

This appears to me symptomatic of a downward trend or bias against the importance of medicinal chemistry. From my perch over the last decade+, I've seen a devaluation of the role of (medicinal) chemistry in contributions from both academia and industry. Have others experienced the same?

Permalink to Comment

13. jrftzgb on April 14, 2014 12:36 PM writes...

Are scientists becoming too specialized? I mean I hear all about interdisciplinary research, but it often is a chemist showing a synthesis and then the results of an assay that someone else completed and they the chemist can't explain (but look at the difference in potency), or vice versa for the biologist.

I've always felt that the problem with this size of study is that with enough data you will eventually be able to conclude something, or give an insightful analysis etc. Unless we come back with serious questions and force people doing this research to drill down to what these data points mean, then we will continue to not get anything truly useful from the research.

Permalink to Comment

14. Billy on April 14, 2014 12:43 PM writes...

The corresponding author on this paper has as is affiliation "Faculty of Pharmaceutical Sciences, UBC."

Two things:

1) There are plenty of good chemists at UBC, including medicinal chemists...did the PI not thing just to have a quick word with one of them.

2) This is from Pharmaceutical Sciences! Surely they should know basic organic chemistry/med chem?! If it was from a yeast biologist, perhaps you could excuse the ignorance, but a pharmaceutical-ist?

Permalink to Comment

15. John Wayne on April 14, 2014 12:51 PM writes...

"Enriched in crap" is an unfortunately accurate description for the majority of academic drug discovery. There any many people doing great stuff out there, but they are the exception.

Permalink to Comment

16. anonymous coward on April 14, 2014 12:54 PM writes...

A quinone methide fragment is a selective pharmacophore for what? Nucleophiles? Alkenes? Since it's also not real happy (hence lack of selectivity), you'd figure that it would be embodied in various precursors and not as an explicit fragment itself, since the different QM precursors might bind differently to targets and bind differently before they go to town.

Permalink to Comment

17. Normal Scientist on April 14, 2014 1:55 PM writes...

entropyGain makes a good point that experienced drug hunters have an obligation to publish and serve as reviewers.

I would also add that it is important to call out garbage when it appears; one of the comments on the March 20 "Years Worth of the Stuff" post asked:

Can you explain why you continue to post on this? It's been done before, so what's the point, except as a place for you to rant.

The fact that this type of thing keeps getting published means that those who know better need to keep patiently educating people.

Permalink to Comment

18. luysii on April 14, 2014 2:12 PM writes...


"Damn it, I hate to be so hard on so many people who did so much work."

Just think of what you are doing to their self-esteem. Your post could be considered 'bullying', the latest assault on free speech done, as always, for the noblest of reasons.

You should apologize before they sue.

Permalink to Comment

19. Teddy Z on April 14, 2014 2:30 PM writes...

I am wading throught this, but I just got to the adamantyl part: it is not crazy as a pharmacophore. It is the major part of a marketed drug for flu, with a known MOA.

Permalink to Comment

20. MoMo on April 14, 2014 2:38 PM writes...

Still Brilliant this Paper!

If you look at the molecules and their frequency of activity their scaffolds follow Bemis and Murcko's 1996 paper "The properties of known drugs, molecular frameworks" pretty closely.


Permalink to Comment

21. watcher on April 14, 2014 3:14 PM writes...

Screening at it's very best.

Permalink to Comment

22. quinquevalent on April 14, 2014 4:39 PM writes...


It's also a part of SQ109, which is in phase 2 trials for TB:

Permalink to Comment

23. Anonymous on April 14, 2014 4:57 PM writes...

#22 don't forget memantine!

Permalink to Comment

24. Nysted Reagent Never Works on April 14, 2014 5:05 PM writes...

@ 4, I would say that your comment is "juste con", not "un peu".

Permalink to Comment

25. Allison (@DrStelling) on April 14, 2014 5:44 PM writes...

This post and your other post have been linked to the paper's PubMed abstract in PubMed Commons by Christopher Southan.

Permalink to Comment

26. PUI prof on April 14, 2014 6:29 PM writes...

@4 Justin
Why should we assume these things were brought up in peer review? I have encountered nothing in my career to suggest that.

@science mag
Snap chat the deets to one person? That is a good way to propagate the idea that peer review fell down. Get in front of it and release them to everyone.

Permalink to Comment

27. Garrett on April 14, 2014 8:28 PM writes...

I'll prefix this by saying that my last chemistry course was the common-year engineering chemistry I had to take as a part of my software engineering program, and my medical knowledge is limited to me volunteering as an EMT. My med-chem knowledge is roughly that bleach makes a great disinfectant.

That having been said, I don't think that this is a *complete* waste of time. If this had been done because a researcher got a shiny-new auto-sequencer, had a pile of yeast lying around, a chemical supplier had left a hot-drug-like sampler box as a Christmas present, and the resulting un-spell-checked manuscript was published in the Annals of Apocryphal Chemistry, I don't think anybody would have a problem. It seems to me that this isn't completely worthless. Sure, it's testing substances of unknown purity at a concentration closer to that of a pickle jar than a living organism, but it's actual data. As long as it isn't a fabrication, this actually could be useful to somebody, somewhere, at some point in time.
I'd argue (closer to my field) that this is exactly what online low-reputation journals are good for. It allow people to publish data which somebody might care about in the future. If somebody decides to work with one of these compounds and wants to know if any testing has been done, they can scratch one test they have to do off of their list.
Now, why such a selective journal would publish what appears to be low-quality data-dump, I can't understand.

Permalink to Comment

28. David Borhani on April 14, 2014 9:38 PM writes...

RE Adamantane: I don't think Derek was suggesting that it doesn't occur in drugs (it obviously does, as do other bi- and tricyclic ring systems, with or without heteroatoms, e.g. quinine, scopolamine, tiotropium, cocaine, etc.). Rather, an adamantane not attached to anything else is unlikely to be a (very useful) drug, in the same way that cyclohexane or hexane are not particularly useful (though, they are not completely devoid of biological effect). That said, some seemingly too simple compounds are useful anesthetics (e.g., cyclopropane [no longer used due to explosion hazard!] and xenon).

Permalink to Comment

29. Jose on April 14, 2014 10:38 PM writes...

The fact that 'pharmacophore' appear nowhere in the entire text but is replaced by the mongrel 'signatures' reinforces all my biases against academic med chem. It's as if they had written "Yeast, which we term 'wee-beasties'..."

Permalink to Comment

30. cliffintokyo on April 15, 2014 2:14 AM writes...

On a completely different topic, just to have a chance to say something good, in contrast to all this bad and ugly science:
In the ACS weekly e-newsletter for Apr 8, they are touting the talk by Paul Richardson (Pfizer) as the most viewed of 'ACS Presentations On Demand' from the Fall 2013 ACS Natl Meeting - and deservedly so IMHO, having watched it today.
Note: I am neither current nor ex Pfizer, but I know quality chemistry when I see it; this talk highlights oodles of bright ideas/innovation and the copious amounts of work needed to solve synthesis/scale-up issues on a very tight schedule.
Recommended for ALL non-chemists who pretend that organic chemists are not innovative, and 'only' make compounds (and that's not even taking med chem creativity into account).
Go Paul! A new spokesperson (PolCorr, for those who require it, ;-)), for all organic chemistry innovative hero scientists in the pharma industry, who are unrecognized/ignored, unlike their academic counterparts....

Permalink to Comment

31. Wow-chem on April 15, 2014 4:35 AM writes...

Stop your whining. I noticed pubmed lists only 5 grants (where an R01 was so little it was listed last) associated with this paper. I think its worth more than a cool million per year to learn that quinone methides elicit a stress response in fungi cells and that all molecules with a THF in them are Tubulin inhibitors.

Permalink to Comment

32. Pinko Punko on April 15, 2014 3:25 PM writes...

This part:

But there are several other alkylimidazolium compounds in the set (which is a bit odd, since they're not exactly drug-like). And they're not assigned to the mitochondrial distress phenotype, as Figure 2 would have you think. SGTC_1247, 179, 193, 1991, 327, and 547 all have this moeity, and they scatter between several other categories. Once again, a majority of compounds with the Figure 2 substructure don't actually map to the phenotype shown (while plenty of other structural types do). What use, exactly, is Figure 2 supposed to be?

This doesn't seem like what the paper is saying. If they look at compounds that give a signature, those compounds are enriched for presence of some chemical moiety, not that compounds with that moiety are predictive of a signature.

Permalink to Comment

33. Scott on April 16, 2014 1:24 AM writes...

If they look at compounds that give a signature, those compounds are enriched for presence of some chemical moiety, not that compounds with that moiety are predictive of a signature.

Could someone please enlighten the non-chemist/non-biologist as to what the hell a "signature" is, if it is not a statement that "structure X has a very strong correlation to reaction Y, so (almost) every chemical with structure X can be expected to 'cause' reaction Y"?

Permalink to Comment

34. Marc on April 16, 2014 7:04 AM writes...

@32 - I think you're right about what they mean. But then I have to ask, does this observation actually have any meaning? Given the astounding diversity of chemical space, from which they are sampling 3250 compounds, is it in any way significant that four imidazoles had similar activities -especially since their sample is necessarily non-representative (selected first for the fact that someone sold the compounds and then for being able to inhibit yeast growth)? I don't actually see a way to make a meaningful, non-arbitrary measure of statistical representation here. The only way this observation is interesting is if it has predictive power, but its predictive power is curtailed by your (correct) reading of their claim.

Permalink to Comment

35. Pinko Punko on April 16, 2014 11:01 AM writes...


That's right, I have no idea what the perceived enrichment means. That said, the complexities of phenotypes can sometimes reveal similarities that would be impossible just by looking at compounds (because of some of the things I mention below). Underlying similarities that would not necessarily be apparent from looking at all compounds (assuming real meaning to the similarities of course) suggests the method could have usefulness even if it doesn't work for de novo prediction. Again, assuming the similarities revealed could be meaningful, the results would suggest some unknown or cryptic variables that indicate a subset of all compounds with some group might have related function (see more below).

Responding to @33, yeah, in the simplest case yes, but begging the question (i.e. accepting for the sake of argument that their method has some validity) I wouldn't expect every compound with some particular group to have that group be functional, or for that functionality to generate the observed phenotypic response, because many of these compounds likely affect a number of pathways, and the responses observed are the sum of these effects, and there might be interactions between the different responses, such as chemical or response epistasis where one effect of a compound can swamp out other effects. I don't really think that the simplest expectation for predicting effects as in @33 is to be expected a priori.

I'm not defending the paper per se, but some of the criticism has been unfair or imprecise.

Permalink to Comment

36. Marc on April 16, 2014 12:00 PM writes...

@35 - I agree with you regarding the value of looking at many stressors. I'm a biologist, too, and my group has certainly published work that falls in the same conceptual space as this paper ("systems biology" of a tractable model organism).

I think we have to look at what a paper of this kind could reasonably be expected to achieve, given its high-profile venue. When I read papers of this sort, I want them to give me at least one of (1) a generalizable protocol for a screening approach that seems truly useful and genuinely novel or (2) a new, general insight into the process being analyzed, ideally something that really relies on the unbiased or semibiased approach and is not simply an analysis of a single hit or (3) an analysis of a single "home-run"-type hit, ideally something that really would have been hard to see by more traditional techniques. I think the now-notorious Figure 2 is an attempt to give us something meeting my criterion (2)- "imidazoles hit ergosterol!" etc etc. The fact that this fundamental claim as I have just expressed it is pretty obviously stupid unless we caveat it full of holes to the point where it loses predictive power is my ultimate problem with this paper - it is not that they have not done the thing they say they did, it is that I think the thing they say they did does not go far enough to justify this publication. Basically, their claim has no meaning because their analysis has too little power. Which is sad, because it is very obviously a lot of work.

All of that, of course, is beside the point on this blog. What is relevant, though, is the chemical sloppiness of this paper (and it is very clearly sloppy in presentation, if nothing else). This is important here because they might have been able to make a better claim if they had gone into this project and analyzed the data with more chemical insight.

Permalink to Comment

37. chem on April 16, 2014 1:26 PM writes...

Adding to previous comments calling out the paper for not checking the quality control and purity of the massive number of commercial compounds tested in this paper. Today I did a reaction with a commercial chemical that was claimed to be "99.5%" purity. My reaction did not work and I checked the NMR of the commercial reagents. Turns out it was the minor compound

Permalink to Comment

38. Joe Q. on April 17, 2014 11:02 AM writes...

I know two of the authors (PIs) from university days -- one has a chemistry degree and the other a biochemistry degree. But with so many authors, perhaps it's not surprising that bad structures slipped through.

Permalink to Comment

39. jamal on April 26, 2014 11:37 AM writes...

can honestly retract this paper?

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

Phenotypic Assays in Cancer Drug Discovery
Neratinib Comes Through For Puma
How Many Biopharma Employees Would Rather Be Working Somewhere Else?
The Broad Gets $650 Million For Psychiatric Research
Put Them in Cells and Find Out
The Hep C Field Gets Nastier By the Minute
Allergan Twists and Turns
Chemistry Class For the "Food Babe"?