About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Biology Maybe Right, Chemistry Ridiculously Wrong | Main | Sweet Reason Lands On Its Face »

April 14, 2014

More on the Science Chemogenomic Signatures Paper

Email This Entry

Posted by Derek

This will be a long one. I'm going to take another look at the Science paper that stirred up so much comment here on Friday. In that post, my first objection (but certainly not my only one) was the chemical structures shown in the paper's Figure 2. A number of them are basically impossible, and I just could not imagine how this got through any sort of refereeing process. There is, for example, a cyclohexadien-one structure, shown at left, and that one just doesn't exist as such - it's phenol, and those equilibrium arrows, though very imbalanced, are still not drawn to scale.
Well, that problem is solved by those structures being intended as fragments, substructures of other molecules. But I'm still positive that no organic chemist was involved in putting that figure together, or in reviewing it, because the reason that I was confused (and many other chemists were as well) is that no one who knows organic chemistry draws substructures like this. What you want to do is put dashed bonds in there, or R groups, as shown. That does two things: it shows that you're talking about a whole class of compounds, not just the structure shown, and it also shows where things are substituted. Now, on that cyclohexadienone, there's not much doubt where it's substituted, once you realize that someone actually intended it to be a fragment. It can't exist unless that carbon is tied up, either with two R groups (as shown), or with an exo-alkene, in which case you have a class of compounds called quinone methides. We'll return to those in a bit, but first, another word about substructures and R groups.
Figure 2 also has many structures in it where the fragment structure, as drawn, is a perfectly reasonable molecule (unlike the example above). Tetrahydrofuran and imidazole appear, and there's certainly nothing wrong with either of those. But if you're going to refer to those as common fragments, leading to common effects, you have to specify where they're substituted, because that can make a world of difference. If you still want to say that they can be substituted at different points, then you can draw a THF, for example, with a "floating" R group as shown at left. That's OK, and anyone who knows organic chemistry will understand what you mean by it. If you just draw THF, though, then an organic chemist will understand that to mean just plain old THF, and thus the misunderstanding.

If the problems with this paper ended at the level of structure drawing, which many people will no doubt see as just a minor aesthetic point, then I'd be apologizing right now. Update: although it is irritating. On Twitter, I just saw that someone spotted "dihydrophyranone" on this figure, which someone figured was close enough to "dihydropyranone", I guess, and anyway, it's just chemistry. But they don't. It struck me when I first saw this work that sloppiness in organic chemistry might be symptomatic of deeper trouble, and I think that's the case. The problems just keep on coming. Let's start with those THF and imidazole rings. They're in Figure 2 because they're supposed to be substructures that lead to some consistent pathway activity in the paper's huge (and impressive) yeast screening effort. But what we're talking about is a pharmacophore, to use a term from medicinal chemistry, and just "imidazole" by itself is too small a structure, from a library of 3200 compounds, to be a likely pharmacophore. Particularly when you're not even specifying where it's substituted and how. There are all kinds of imidazole out there, and they do all kinds of things.
So just how many imidazoles are in the library, and how many caused this particular signature? I think I've found them all. Shown at left are the four imidazoles (and there are only four) that exhibit the activity shown in Figure 2 (ergosterol depletion / effects on membrane). Note that all four of them are known antifungals - which makes sense, given that the compounds were chosen for the their ability to inhibit the growth of yeast, and topical antifungals will indeed do that for you. And that phenotype is exactly what you'd expect from miconazole, et al., because that's their known mechanism of action: they mess up the synthesis of ergosterol, which is an essential part of the fungal cell membrane. It would be quite worrisome if these compounds didn't show up under that heading. (Note that miconazole is on the list twice).
But note that there are nine other imidazoles that don't have that same response signature at all - and I didn't even count the benzimidazoles, and there are many, although from that structure in Figure 2, who's to say that they shouldn't be included? What I'm saying here is that imidazole by itself is not enough. A majority of the imidazoles in this screen actually don't get binned this way. You shouldn't look at a compound's structure, see that it has an imidazole, and then decide by looking at Figure 2 that it's therefore probably going to deplete ergosterol and lead to membrane effects. (Keep in mind that those membrane effects probably aren't going to show up in mammalian cells, anyway, since we don't use ergosterol that way).

There are other imidazole-containing antifungals on the list that are not marked down for "ergosterol depletion / effects on membrane". Ketonconazole is SGTC_217 and 1066, and one of those runs gets this designation, while the other one gets signature 118. Both bifonazole and sertaconazole also inhibit the production of ergosterol - although, to be fair, bifonazole does it by a different mechanism. It gets annotated as Response Signature 19, one of the minor ones, while sertaconazole gets marked down for "plasma membrane distress". That's OK, though, because it's known to have a direct effect on fungal membranes separate from its ergosterol-depleting one, so it's believable that it ends up in a different category. But there are plenty of other antifungals on this list, some containing imidazoles and some containing triazoles, whose mechanism of action is also known to be ergosterol depletion. Fluconazole, for example, is SGTC_227, 1787 and 1788, and that's how it works. But its signature is listed as "Iron homeostasis" once and "azole and statin" twice. Itraconzole is SGTC_1076, and it's also annotated as Response Signature 19. Voriconazole is SGTC_1084, and it's down as "azole and statin". Climbazole is SGTC_2777, and it's marked as "iron homeostasis" as well. This scattering of known drugs between different categories is possibly and indicator of this screen's ability to differentiate them, or possibly an indicator of its inherent limitations.

Now we get to another big problem, the imidazolium at the bottom of Figure 2. It is, as I said on Friday, completely nuts to assign a protonated imidazole to a different category than a nonprotonated one. Note that several of the imidazole-containing compounds mentioned above are already protonated salts - they, in fact, fit the imidazolium structure drawn, rather than the imidazole one that they're assigned to. This mistake alone makes Figure 2 very problematic indeed. If the paper was, in fact, talking about protonated imidazoles (which, again, is what the authors have drawn) it would be enough to immediately call into question the whole thing, because a protonated imidazole is the same as a regular imidazole when you put it into a buffered system. In fact, if you go through the list, you find that what they're actually talking about are N-alkylimidazoliums, so the structure at the bottom of FIgure 2 is wrong, and misleading. There are two compounds on the list with this signature, in case you were wondering, but the annotation may well be accurate, because some long-chain alkylimidazolium compounds (such as ionic liquid components) are already known to cause mitochondrial depolarization.

But there are several other alkylimidazolium compounds in the set (which is a bit odd, since they're not exactly drug-like). And they're not assigned to the mitochondrial distress phenotype, as Figure 2 would have you think. SGTC_1247, 179, 193, 1991, 327, and 547 all have this moeity, and they scatter between several other categories. Once again, a majority of compounds with the Figure 2 substructure don't actually map to the phenotype shown (while plenty of other structural types do). What use, exactly, is Figure 2 supposed to be?

Let's turn to some other structures in it. The impossible/implausible ones, as mentioned above, turn out to be that way because they're supposed to have substituents on them. But look around - adamantane is on there. To put it as kindly as possible, adamantane itself is not much of a pharmacophore, having nothing going for it but an odd size and shape for grease. Tetrahydrofuran (THF) is on there, too, and similar objections apply. When attempts have been made to rank the sorts of functional groups that are likely to interact with protein binding sites, ethers always come out poorly. THF by itself is not some sort of key structural unit; highlighting it as one here is, for a medicinal chemist, distinctly weird.

What's also weird is when I search for THF-containing compounds that show this activity signature, I can't find much. The only things with a THF ring in them seem to be SGTC_2563 (the complex natural product tomatine) and SGTC_3239, and neither one of them is marked with the signature shown. There are some imbedded THF rings as in the other structural fragments shown (the succinimide-derived Diels-Alder ones), but no other THFs - and as mentioned, it's truly unlikely that the ether is the key thing about these compounds, anyway. If anyone finds another THF compound annotated for tubulin folding, I'll correct this post immediately, but for now, I can't seem to track one down, even though Table S4 says that there are 65 of them. Again, what exactly is Figure 2 supposed to be telling anyone?

Now we come to some even larger concerns. The supplementary material for the paper says that 95% of the compounds on the list are "drug-like" and were filtered by the commercial suppliers to eliminate reactive compounds. They do caution that different people have different cutoffs for this sort of thing, and boy, do they ever. There are many, many compounds in this collection that I would not have bothered putting into a cell assay, for fear of hitting too many things and generating uninterpretable data. Quinone methides are a good example - as mentioned before, they're in this set. Rhodanines and similar scaffolds are well represented, and are well known to hit all over the place. Some of these things are tested at hundreds of micromolar.

I recognize that one aim of a study like this is to stress the cells by any means necessary and see what happens, but even with that in mind, I think fewer nasty compounds could have been used, and might have given cleaner data. The curves seen in the supplementary data are often, well, ugly. See the comments section from the Friday post on that, but I would be wary of interpreting many of them myself.
There's another problem with these compounds, which might very well have also led to the nastiness of the assay curves. As mentioned on Friday, how can anyone expect many of these compounds to actually be soluble at the levels shown? I've shown a selection of them here; I could go on. I just don't see any way that these compounds can be realistically assayed at these levels. Visual inspection of the wells would surely show cloudy gunk all over the place. Again, how are such assays to be interpreted?

And one final point, although it's a big one. Compound purity. Anyone who's ever ordered three thousand compounds from commercial and public collections will know, will be absolutely certain that they will not all be what they say on the label. There will be many colors and consistencies, and LC/MS checks will show many peaks for some of these. There's no way around it; that's how it is when you buy compounds. I can find no evidence in the paper or its supplementary files that any compound purity assays were undertaken at any point. This is not just bad procedure; this is something that would have caused me to reject the paper all by itself had I refereed it. This is yet another sign that no one who's used to dealing with medicinal chemistry worked on this project. No one with any experience would just bung in three thousand compounds like this and report the results as if they're all real. The hits in an assay like this, by the way, are likely to be enriched in crap, making this more of an issue than ever.

Damn it, I hate to be so hard on so many people who did so much work. But wasn't there a chemist anywhere in the room at any point?

Comments (39) + TrackBacks (0) | Category: Biological News | Chemical Biology | Chemical News | The Scientific Literature


1. weirdo on April 14, 2014 10:01 AM writes...

Everyone associated with this paper -- the authors, the editors, the reviewers -- should feel wholly embarrassed this morning.

I suspect many of them will not, but they should.

Permalink to Comment

2. Anonymous on April 14, 2014 10:06 AM writes...

"Damn it, I hate to be so hard on so many people who did so much work. But wasn't there a chemist anywhere in the room at any point?"

Rhetorical question?

Permalink to Comment

3. PharmaHeretic on April 14, 2014 10:14 AM writes...

Is it really surprising that a highly visible effort to use big data-type techniques in biomedical research was based in fancy technical showmanship rather than accuracy or substance? Isn't that rather typical of a lot of research from large groups in supposedly elite institutions?

Academic-types have now internalized the snake-oil based techniques of businessmen and MBAs. This particular group just had the misfortune of getting caught, otherwise this publication would have been one more high impact paper in support of their next round of grant applications.

Permalink to Comment

4. Justin Peukon on April 14, 2014 10:20 AM writes...

Most of the concerns regarding this paper and quoted in this blog were almost certainly raised during the review process. The key point is what authors did with these reports. I can imagine their reaction: "ooooh noooo!!! They have not understood our Fig. 2 and are asking FOR A NEW ONE!!! Waste of time, pure waste of f...time, and we have NO TIME ALLOWED". The "revised" paper is then submitted with original Fig. 2. Obviously, Science staff doesn’t check (N-O T-I-M-E), and the paper is published.
Science, if you read my post: please release the full set of unedited reviews for this paper.

Permalink to Comment

5. Puff the Mutant Dragon on April 14, 2014 10:27 AM writes...

This is the kind of thing you should expect to see more of if academic labs start doing more "drug discovery" work and "translational research" as has been advocated elsewhere.

Permalink to Comment

6. Am I Lloyd peptide on April 14, 2014 10:27 AM writes...

And people still wonder if open peer review would be a good idea.

Permalink to Comment

7. fluorogrol on April 14, 2014 10:30 AM writes...

Good analysis.

Although the eye-catching impossible structures aren't the major problem with the paper, they're an indicator of the lack of chemical understanding that seems to have led to the real failings.

Permalink to Comment

8. ScienceMag on April 14, 2014 11:39 AM writes...

Justin: This is science. I read your comment. I will snapchat u the deets asap

Permalink to Comment

9. RTW on April 14, 2014 11:39 AM writes...

Likely the lone chemist in the group was someone either newly minted/anointed and little actual experience as a medicinal chemist yet and was too timid about is doubts, or a old 20+ years experienced chemist with a BS/MS degree that no one would listen to because he didn't have a PHD! I have seen many supposed PHD chemists dong Computer Aided Drug Design draw such nonsense and proposed them to us bench chemists to synthesize so this is no surprise to me.

Permalink to Comment

10. ESIMS on April 14, 2014 11:43 AM writes...

Has nothing to do with academic labs, they can also run proper controls to "exclude" off-target effects.

In some kind of way related remark:
1% DMSO is not good idea for yeasts (vacuole & osmotic stress effects) and 0.5% for cell lines is again far from optimal.

Permalink to Comment

11. entropyGain on April 14, 2014 12:12 PM writes...

Maybe this is our fault.

By our fault, I mean professional drug hunters who have years of experience screening, dealing with libraries, SAR, off-target activities, ideopathic tox and generally fighting Murphy's law in ways that academic labs have never conceived of previously. So why our fault? Because as a group we don't publish enough of our work. Too often we allow the "competitive" issues or corporate priorities innappropriately outweigh proper scholarship. By not publishing, we leave an opening for naive academic labs to stumble into the same problems then excitedly publish their findings. Different value systems.

I'm not making an excuse for this particular case, which is egregious in its careless disregard for the fundamentals of basic organic chemistry, but for the larger set of examples that find their way to this blog repeating mistakes many of us already know.

It is also partially our fault because we as a group don't value "service" to the scientific community as highly as if our tenure depended upon it. By that I mean as reviewers of grants and manuscripts. It's incredibly hard to find good reviewers willing to donate the time and energy required. That means that study sections and editorial boards are packed with academics that have little background in real drug discovery. No wonder crap like this gets funded and published.

What can we do? Publish and serve as reviewers.
Sounds simple and will probably make a difference, but believe me though, I understand how hard that is for most of us in most of the situations we find ourselves in this industry.

Permalink to Comment

12. PorkPieHat on April 14, 2014 12:29 PM writes...

This appears to me symptomatic of a downward trend or bias against the importance of medicinal chemistry. From my perch over the last decade+, I've seen a devaluation of the role of (medicinal) chemistry in contributions from both academia and industry. Have others experienced the same?

Permalink to Comment

13. jrftzgb on April 14, 2014 12:36 PM writes...

Are scientists becoming too specialized? I mean I hear all about interdisciplinary research, but it often is a chemist showing a synthesis and then the results of an assay that someone else completed and they the chemist can't explain (but look at the difference in potency), or vice versa for the biologist.

I've always felt that the problem with this size of study is that with enough data you will eventually be able to conclude something, or give an insightful analysis etc. Unless we come back with serious questions and force people doing this research to drill down to what these data points mean, then we will continue to not get anything truly useful from the research.

Permalink to Comment

14. Billy on April 14, 2014 12:43 PM writes...

The corresponding author on this paper has as is affiliation "Faculty of Pharmaceutical Sciences, UBC."

Two things:

1) There are plenty of good chemists at UBC, including medicinal chemists...did the PI not thing just to have a quick word with one of them.

2) This is from Pharmaceutical Sciences! Surely they should know basic organic chemistry/med chem?! If it was from a yeast biologist, perhaps you could excuse the ignorance, but a pharmaceutical-ist?

Permalink to Comment

15. John Wayne on April 14, 2014 12:51 PM writes...

"Enriched in crap" is an unfortunately accurate description for the majority of academic drug discovery. There any many people doing great stuff out there, but they are the exception.

Permalink to Comment

16. anonymous coward on April 14, 2014 12:54 PM writes...

A quinone methide fragment is a selective pharmacophore for what? Nucleophiles? Alkenes? Since it's also not real happy (hence lack of selectivity), you'd figure that it would be embodied in various precursors and not as an explicit fragment itself, since the different QM precursors might bind differently to targets and bind differently before they go to town.

Permalink to Comment

17. Normal Scientist on April 14, 2014 1:55 PM writes...

entropyGain makes a good point that experienced drug hunters have an obligation to publish and serve as reviewers.

I would also add that it is important to call out garbage when it appears; one of the comments on the March 20 "Years Worth of the Stuff" post asked:

Can you explain why you continue to post on this? It's been done before, so what's the point, except as a place for you to rant.

The fact that this type of thing keeps getting published means that those who know better need to keep patiently educating people.

Permalink to Comment

18. luysii on April 14, 2014 2:12 PM writes...


"Damn it, I hate to be so hard on so many people who did so much work."

Just think of what you are doing to their self-esteem. Your post could be considered 'bullying', the latest assault on free speech done, as always, for the noblest of reasons.

You should apologize before they sue.

Permalink to Comment

19. Teddy Z on April 14, 2014 2:30 PM writes...

I am wading throught this, but I just got to the adamantyl part: it is not crazy as a pharmacophore. It is the major part of a marketed drug for flu, with a known MOA.

Permalink to Comment

20. MoMo on April 14, 2014 2:38 PM writes...

Still Brilliant this Paper!

If you look at the molecules and their frequency of activity their scaffolds follow Bemis and Mur