Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases.
To contact Derek email him directly: derekb.lowe@gmail.com
Twitter: Dereklowe
I'm baffled by this abstract. Why would you go to the trouble of putting an unusual group (a ferrocene) on a molecule, and then show that putting it on seems to do little or nothing to the properties and activity of the parent compound? "We put a ferrocene on and it didn't kill the molecule" doesn't seem to be enough grounds for a full J. Med. Chem. paper. Does it?
Just a note: the Breslow origins-of-chirality paper, also known more widely (thanks to some bizarre PR work at the ACS) as the "alien dinosaur" paper, has been withdrawn, and on the correct grounds. The pun in the headline is intended.
More disruption in the scientific publishing model: the UK government has announced that it will set up an open-access system for papers that are generated through its funding, similar to the system in the US. The details are still being worked out, and the government is still making noises about not "ruining the value provided by academic publishers", but it's that value that's at issue, isn't it?
A statement from Wiley said that "Publishers enable content digitisation, rigorous peer review, strong editorial infrastructure and support and investment in an effective online platform for dissemination." And yes, they do those things. But how well do they do them? And how well do they do them for the prices they charge? I'm glad that these arguments are finally out on the table.
Over at The Curious Wavefunction, there's a great post looking back at the infamous "negative rate constant" affair (Breslow, Menger, Haim). If you're not familiar with that one, give it a look. I remember this one while it was going on, and in retrospect, you have to imagine what it would have been like if there had been a chemical blog world at the time. It's an extraordinary chapter in chemical (and chemical literature) history.
To that end, there's this opinion piece from yesterday's New York Times. Author Jack Hitt is talking about the tail of comments that now follows any notable article, in any field:
Almost any article worth reading these days generates some version of this long tail of commentary. Depending on whether they are moderated, these comments can range from blistering flameouts to smart factual corrections to full-on challenges to the very heart of an article’s argument. . .
. . .the comments section of any engaging article is almost as necessary a read as the piece itself — if you want to know how insider experts received the article and how those outsiders processed the news (and maybe to enjoy some nasty snark from the trolls).
Should this part of every contemporary article be curated and edited, almost like the piece itself? Should it have a name? Should it be formally linked to the original article or summarized at the top? By now, readers understand that the definitive “copy” of any article is no longer the one on paper but the online copy, precisely because it’s the version that’s been read and mauled and annotated by readers. (If a book isn’t read until it’s written in — as I was always told — then maybe an article is not published until it’s been commented upon.) Writers know this already. The print edition of any article is little more than a trophy version, the equivalent of a diploma or certificate of merit — suitable for framing, not much else.
I think this is exactly what science is about, and exactly what it needs. People should be able to read the latest results, add their opinions and criticisms to them, and those comments in turn should also be available for everyone to see. There's going to be noise in there, but I'll take some noise as the price that gets paid for figuring things out more quickly and more completely than we ever could before. As far as I'm concerned, the "peer" in "peer review" means "Everyone who can read and understand the paper".
Just another nitpicking note: if you're going to publish a paper on glucose conjugates of drugs (aspirin, in this case), you might want to be sure to draw the glucose as the correct enantiomer. I had to do a little head-scratching with this one, since the sugar ring is drawn from a perspective that no carbohydrate chemist would ever use, but as far as I can tell, that's L-glucose instead of D. . .
This is not as big a deal in the grand scheme of things, but it particularly gets to me, as someone who worked with sugars as chiral starting materials for 4 1/2 years. And then, every chemical drawing program extant will draw you the correct stereochemistry. . .
Inspired by a discussion with a colleague, I'm going to take one more crack at the recent discussion here about theJ. Med. Chem. DHFR paper. Those of you with an interest in the topic, read on. Those whose interest has waned, or who never had much interest to start with, take heart: other topics are coming.
It's clear that many people were disappointed with my take on this paper, and my handling of the whole issue. Let me state again that I mishandled the biology aspects of this one thoroughly, through carelessness, and I definitely owe this apology to the authors of the paper (and the readers of this site) for that.
Of course, that's not the only arguable thing about the way I handled this one. As I spent paragraphs rambling on about in yesterday's post, there's a chemical aspect to the whole issue as well, and that's what caught my eye to start with. I think one of the things that got me into trouble with this one is two different ways of looking at the world. I'll explain what I mean, and you can judge for yourself if I'm making any sense.
The authors of the paper (and its reviewer who commented here) are interested in D67 dihydrofolate reductase, from a biological/enzymological perspective. From this viewpoint - and it's a perfectly tenable one - the important thing is that D67 DHFR is an unusual and important enzyme, a problem in bacterial resistance, interesting in its own right as a protein with an odd binding site, and for all that, still has no known selective inhibitors. Anything that advances the understanding of the enzyme and points toward a useful inhibitor of it is therefore a good thing, and worth publishing in J. Med. Chem., too.
I come in from a different angle. As someone who's done fragment-based drug discovery and takes a professional interest in it, I'll take a look at any new paper using the technique. In this case, I gave the target much too cursory a look, and filed it as "DHFR, bacterial enzyme, soluble, X-ray structures known". In other words, a perfectly reasonable candidate for FBDD as we know it. Once I'd decided that this was a mainstream application of something I already have experience with, I turned my attention to how the fragment work was done. By doing so, I missed out on the significance of the DHFR enzyme, which means, to people in the first camp, that I whiffed on the most important part of the entire thing. I can understand their frustration as I brushed that off like a small detail and went on to what (to them) were secondary matters.
But here's where my view of the world comes in. As a drug discovery guy, when I read a paper in J. Med. Chem., I'd like to see progress in, well, the medicinal chemistry of the topic. That was the thrust of my blog post yesterday: that I found the med-chem parts of the paper uncompelling, and that the application of fragment-based techniques seemed to me to have gone completely off track. (I havne't mentioned the modeling and X-ray aspects of the paper, as Teddy Z did at Practical Fragments, but I also found those parts adding nothing to the worth of the manuscript as a whoel). The most potent compounds in the paper seem, to me, to be the sort that are very unlikely to lead to anything, and are unlikely to show selectivity in a cellular environment. If the paper's starting fragment hits are real (which is not something that's necessarily been proven, as I mentioned in yesterday's post), then it seems to me that everything interesting and useful about them is being thrown away as the paper goes on. From the other point of view, things are basically the opposite - the paper gets better and better as the compounds get more potent.
But here's where, perhaps, the two viewpoints I spoke of earlier might find something in common. If you believe that the important thing is that selective inhibitors of D67 DHFR have finally been discovered, then you should want these to be as potent and selective as possible, and as useful as possible in a variety of assays. This, I think, is what's in danger of being missed. I think that a fragment-based effort should have been able to deliver much more potent chemical matter than these compounds, with less problematic structures, which are more likely to be useful as tools.
I'll finish up by illustrating the different angles as starkly as I can. The authors of this paper have, in one view of the world, completed the first-ever fragment screen against an important enzyme, discovered the first-ever selective inhibitors of it, and have published these results in a prestigious journal: a success by any standard. From my end, if I were to lead a drug discovery team against the same enzyme, I might well see the same fragment hits the authors did, since I know that some of these are in the collections I use. But if I proceeded in the same fashion they did, prosecuting these hit compounds in the same way, I would, to be completely honest about it, face some very harsh questioning. And if I persevered in the same fashion, came up with the same final compounds, and presented them as the results of my team's work, I would run the serious risk of being fired. Different worlds.
Update: Prof. Pelletier sends the following:
I certainly have been following this with interest, and learning much from it – not just science.
Throughout the week, I have appreciated your civil tone – many thanks. I willingly accept your apology, just as I accept the constructive criticism that will improve our future work. I think your ‘two-worlds’ point of view smacks of truth. The bottom line from my point of view is that I’m open to collaboration with a real fragment library: if anyone is interested in making this better, they should contact me. I’d be delighted to work with more than what can be scavenged from neighbouring labs in an academic setting.
Your bloggers’ response to this come-and-go was fascinating: the process was admired to an extent that surprised me. A number of responders point out that there are currently few occurrences of open exchange on these blogs and – sorry to disappoint hard-core bloggers – it does not endear me to the blogging process. I don’t blog because I can’t stand anonymous, frequently disrespectful and sometimes poorly researched comments. I nonetheless hope that this will open the door to a more transparent blogging process in the long run.
For any who care, I am brave, not at all desperate, and definitely a woman. ; )
If you feel any of this would be of interest for your blog, please feel free to post. Thanks for seeing this through rather than shaking it off.
So perhaps I should rethink all those nasty things I've been saying about Elsevier journals. Here's someone who submitted a paper to Nuclear Instruments and Methods on a Friday evening, and got it accepted - with two referee reports, yet - on Monday morning. How is that possible, you say? That's what this author is wondering, too. . .
The other day, I had some uncomplimentary things to say about a recent J. Med. Chem. paper on fragment-based dihydrofolate reductase inhibitors. Well, I know that I don't say these things into a vacuum, by any means, but in this case the lead author has written me about the work, and a reviewer of the paper has showed up in the comments. So perhaps this is a topic worth revisiting?
First, I'll give Prof. Joelle Pelletier of U. Montreal the floor to make the case for the defense. Links added are mine, for background; I take responsibility for those, and I hope they're helpful.
I was informed of your recent blog entitled ‘How do these things get published’. I am corresponding author of that paper. I would like to bring to your attention a crucial point that was incorrectly presented in your analysis: the target enzyme is not that which you think it is, i.e.: it is not a DHFR that is part of ‘a class of enzymes that's been worked on for decades’.
Indeed, it would make no sense to report weak and heavy inhibitors against ‘regular’ DHFRs (known as ‘type I DHFRs’), considering the number of efficient DHFR inhibitors we already know. But this target has no sequence or structural homology with type I DHFRs. It is a completely different protein that offers an alternate path to production of tetrahydrofolate (see top of second page of the article). It has apparently evolved recently, as a bacterial response to trimethoprim being introduced into the environment since the ‘60’s. Because that protein is evolutionarily unrelated to regular DHFRs, it doesn’t bind trimethoprim and is thus intrinsically trimethoprim resistant; it isn’t inhibited by other inhibitors of regular DHFRs either. There have been no efforts to date to inhibit this drug resistance enzyme, despite its increasing prevalence in clinical and veterinary settings, and in food and wastewater (see first page of article). As a result, we know nothing about how to prevent it from providing drug resistance. Our paper is thus the first foray into inhibiting this new target – one which presents both the beauty and the difficulty of complex symmetry.
Regular (type I) DHFRs are monomeric enzymes with an extended active-site cleft. They are chromosomally-encoded in all living cells where they are essential for cellular proliferation. Our target, type II R67 DHFR, is carried on a plasmid, allowing rapid dissemination between bacterial species. It is an unusual homotetrameric, doughnut-style enzyme with the particularity of having a single active site in the doughnut hole. That’s unusual because multimeric enzymes typically have the same number of active sites as they do monomers. The result is that the active site tunnel, shown in Figure 4 a, has 222 symmetry. Thus, the front and back entrances to the active site tunnel are identical. And that’s why designing long symmetrical molecules makes sense: they have the potential of threading through the tunnel, where the symmetry of the inhibitor would match the symmetry of the target. If they don’t string through but fold up into a ‘U”, it still makes sense: the top and bottom of the tunnel are also alike, again allowing a match-up of symmetry. Please note that this symmetry does create a bit of a crystallographer’s nightmare at the center of the tunnel where the axes of symmetry meet; again, it is an unusual system.
You have referred to our ‘small, poorly documented library of fragment compounds’. As for the poor documentation, the point is that we have very little prior information on the ligands of this new target, other than its substrates. We cast as wide a net as we could within a loosely defined chemical class, using the chemicals we have access to. Unfortunately, I don’t have access to a full fragment library, but am open to collaboration.
As a result of extending the fragments, the ligand efficiency does take a beating… so would it have been better not to mention it? No, that would have been dishonest. In addition, it is not a crucial point at this very early stage in discovery: this is a new target, and it IS important to obtain information on tighter binding, even if it comes at the cost of heavier molecules. In no way do we pretend that these molecules are ripe for application; we have presented the first set of crude inhibitors to ‘provide inspiration for the design of the next generation of inhibitors’ (last sentence of the paper).
Your blog is widely read and highly respected. In this case, it appears that your analysis was inaccurate due to a case of mistaken identity. I did appreciate your calm and rational tone, and hope that you will agree that there is redeeming value to the poor ligand efficiency, because of the inherent novelty of this discovery effort. I am appealing to you to reconsider the blog’s content in light of the above information, and respectfully request that you consider revising it.
Well, as for DHFRs, I'm guilty as charged. The bacterial ones really are way off the mammalian ones - it appears that dihydro/tetrahydrofolate metabolism is a problem that's been solved a number of different ways and (as is often the case) the bacteria show all kinds of diversity compared to the rest of the living world. And there really aren't any good D67 DHFR inhibitors out there, not selective ones, anyway, so a molecule of that type would definitely be a very worthwhile tool (as well as a potential antibiotic lead).
But that brings us to the fragments, the chemical matter in the paper. I'm going to stand my my characterization of the fragment library. 100 members is indeed small, and claiming lack of access to a "full fragment collection" doesn't quite cover it. Because of the amount of chemical space that can be covered at these molecular weights, a 200-member library can be significantly more useful than a 100-member one, and so on. (Almost anything is more useful than a 100-member library). There aren't more compounds of fragment size on the shelves at the University of Montreal?
More of a case could be made for libraries this small if they covered chemical space well. Unfortunately, looking over the list of compounds tested (which is indeed in the Supplementary Material), it's not, at first glance, a very good collection. Not at all. There are some serious problems, and in a collection this small, mistakes are magnified. I have to point out, to start with, that compounds #59 and #81 are duplicates, as are compounds #3 and #40, and compounds #7 and #14. (There may be others; I haven't made a complete check).
The collection is heavily biased towards carboxylic acids (which is a problem for several reasons, see below). Nearly half the compounds have a COOH group by my quick count, and it's not a good idea to have any binding motif so heavily represented. I realize that you intentionally biased your screening set, but then, an almost featureless hydrophobic compound like #46 has no business in there. Another problem is that some of the compounds are so small that they're unlikely to be tractable fragment hits - I note succinimide (#102) and propyleneurea (#28) as examples, but there are others. At the other end of the scale, compounds such as the Fmoc derivative #25 are too large (MW 373), and that's not the only offender in the group (nor the only Fmoc derivative). The body of the manuscript mentions the molecular weights of the collections as being from 150 to 250, but there are too many outliers. This isn't a large enough collection for this kind of noise to be in it.
There are a number of reactive compounds in the list, too, and while covalent inhibitors are a very interesting field, this was not mentioned as a focus of your efforts or as a component of the screening set. And even among these, compounds such as carbonyldiimidazole (#26), the isocyanate #82, and disuccinimidylcarbonate (#36) are really pushing it, as far as reactivity and hydrolytic stability. The imine #110 is also very small and likely to have hydrolytic stability problems. Finally, the fragment #101 is HEPES, which is rather odd, since HEPES is the buffer for the enzyme assays. Again, there isn't room for these kinds of mistakes. It's hard for me to imagine that anyone who's ever done fragment screening reviewed this manuscript.
The approach to following up these compounds also still appears inadequate to me. As Dan Erlanson pointed out in a comment to the Practical Fragments post, small carboxylic acids like the ones highlighted are not always legitimate hits. They can, as he says, form aggregates, depending on the assay conditions, and the most straightforward way of testing that is often the addition of a small amount of detergent, if the assay can stand it. The behavior of such compounds is also very pH-dependent, as I've had a chance to see myself on a fragment effort, so you need to make sure that you're as close to physiological conditions as you can get. I actually have seen some of your compounds show up as hits in fragment screening efforts, and they've been sometimes real, sometimes not.
But even if we stipulate that these compounds are actually hits, they need more work than they've been given. The best practice, in most cases when a fragment hit is discovered and confirmed, is to take as many closely related single-atom changes into the assay as possible. Scan a methyl group around the structure, scan a fluoro, make the N-for-C switches - at these molecular weights, these changes can make a big difference, and you may well find an even more ligand-efficient structure to work from.
Now, as for the SAR development that actually was done: I understand the point about the symmetry of the enzyme, and I can see why this led to the idea of making symmetrical dimer-type compounds. But, as you know, this isn't always a good idea. Doing so via flexible alkyl or alkyl ether chains is not a good idea, though, since such compounds will surely pay an entropic penalty in binding.
And here's one of the main things that struck both me and Teddy Z in his post: if the larger compounds were truly taking advantage of the symmetry, their ligand efficiency shouldn't go down. But in this case it does, and steeply. The size of the symmetical inhibitors (and their hydrophobic regions, such as the featureless linking chains, make it unsurprising that this effort found some micromolar activity. Lots of things will no doubt show micromolar activity in such chemical space. The paper notes that it's surprising that the fragment 4c showed no activity when its structural motif was used to build some of the more potent large compounds, but the most likely hypothesis is that this is because the binding modes have nothing to do with each other.
To be fair, compounds 8 and 9 are referred to as "poorly optimized", which is certainly true. But the paper goes on to say that they are starting points to develop potent and selective inhibitors, which they're not. The fragments are starting points, if they're really binding. The large compounds are dead ends. That's why Teddy Z and I have reacted as strongly as we have, because the path this paper takes is (to our eyes) an example of how not to do fragment-based drug discovery.
But still, I have to say that I'm very glad to hear a direct reply to my criticism of this paper. I hope that this exchange has been useful, and that it might be of use for others who read it.
Update: the paper has, for now, been pulled by JACS. More to come, no doubt.
I didn't written anything about the Breslow origins-of-chirality paper that mentioned, as a joking aside, the possibility of intelligent alien dinosaurs. As most readers will know, the ACS publicity office, most cluelessly, decided to make that throwaway line the focus of their press release, and much confusion ensued.
But things have gotten weirder. Stu Cantrill read the Breslow paper, and realized that he'd already read a lot of it before. See these three pictures (1, 2, 3) and realize the extent to which this latest paper was apparently a cut-and-paste job.
I've met Breslow many times (he used to consult for one of my former employers), and I've enjoyed reading much of his work. But this really shouldn't be acceptable - we wouldn't put up with it from some unknown chemist, and we shouldn't put up with it from someone famous. Chembark has an excellent summary of the situation, with recommendations about what the ACS should do next. These range from fixing that idiotic press release, to retracting the paper, to barring Breslow from publishing in JACS for a period.
In retrospect, the alien dinosaurs are becoming my favorite part of the whole paper.
Since I've mentioned the scientific publication business today, I thought I'd include this story from Retraction Watch as an example of what you're paying for when you pay Elsevier for quality control. When's the last time you saw a paper withdrawn because it "contains no scientific content"? Or noticed that the lead author's email contract address was at the domain "budweiser.com"?
Several readers sent along this memo from Harvard's library: they see the current price structure of scientific journals as unsustainable, and they're asking the faculty to help them do something about it.
The Faculty Advisory Council to the Library, representing university faculty in all schools and in consultation with the Harvard Library leadership, reached this conclusion: major periodical subscriptions, especially to electronic journals published by historically key providers, cannot be sustained: continuing these subscriptions on their current footing is financially untenable. Doing so would seriously erode collection efforts in many other areas, already compromised.
They're asking faculty members to try to "move prestige" to open-access journals by favoring them for new publications, and in general to do whatever they can to get away from the current scientific publishing model. And that ties in with this post over at the Guardian, saying that not only are the current publishers causing a financial burden, but other burdens that may be even more of a problem:
Research, especially scientific research, thrives in an atmosphere that allows the free exchange of ideas and information: open discussion and debate are essential if the scientific method is to operate properly. Before the arrival of the internet, academic publishers provided a valuable service that was a real benefit to the scientific community. Not any more. . .
. . .But open access isn't just about the end products of research. It's the entire process of scientific enquiry, including the collection and processing of data, scrutiny of the methods used in the analysis, questioning of assumptions, and discussion of alternative interpretations. In particular, it's about access to scientific data.
I believe all data resulting from publicly funded research should be in the public domain, for two reasons. First, it's public money that funds us, so we scientists have a moral responsibility to be as open as possible with the public. Second, the scientific method only works when analyses can be fully scrutinised and, if necessary, replicated by other researchers. In other words, to seek to prevent your data becoming freely available is plain unscientific.
We'll see how far this gets. But this is already the biggest upheaval that I can remember in the scientific literature, and it show no signs of slowing down. . .
Update: I've heard from both the lead author of this paper and one of its reviewers, and I've written a follow-up post on this subject, as well as revising this one where shown below.
I've been saved the trouble of demolishing this J. Med. Chem. paper - the Practical Fragments blog has done it for me. I really hate to say such things, but this appears to be one of the worst papers that journal has published in quite a while.
The authors start out with a small, poorly documented(update: the compounds are, in fact, in the paper's supplementary information, but see the follow-up post) library of fragment compounds. They screen these against dihydrofolate reductase, and get a few possible hits - mind you, there's not much correlation between the numbers and any potency against the enzyme, but these aren't potent compounds, and fragment-level hits don't always perform in high-concentration enzyme assays. But what happens next? The authors string these things together into huge dimeric molecules, apparently because they think that this is a good idea, but they get no data to support this hypothesis at all.
Well, their potency goes from low millimolar to low micromolar, but as Teddy Z at Practical Fragments points out, this actually means taking a terrible beating in ligand efficiency. All that extra molecular weight should buy you a lot more potency than this. There's some hand-waving docking of these structures - which the authors themselves refer to as "poorly optimized" - and some inconclusive attempts at X-ray crystallography, leading to uninterpretable data.
And that's it. That's the paper. This on a class of enzymes that's been worked on for decades, yet.(Update: this characterization is completely wrong on my part - see the follow-up post linked to above for more). Again, I hate to be unkind about this, but I cannot imagine what this is doing in J. Med. Chem., or how it made it through the editorial process. When you submit a scientific manuscript for publication, you open yourself to comments from all comers, and those are mine.
Update: just to make things clear, this is only one aspect of the whole problem, but perhaps an easier one to tackle. More on the rest of the proposals to come!
Yesterday's post brought in a lot of welcome comment, and I want to follow up on the ideas in it. The first problem I wanted to tackle was journal access for entrepreneurs, the recently unemployed, and small shops. Here are some of the comments from the first post, consolidated:
A better bet would be to negotiate a group agreement with DeepDyve. "You can rent the article and view it at DeepDyve for 24 hours or more" for $1. The catch? No printing- you have to read it online (hence the renting not purchasing model).
Something like that might be necessary, because others pointed out that:
I like the idea of the journal access, HOWEVER, you'll pay handsomely for such access. It's no different than a large corp or large library. You have 300 people in your organization? You need to buy a group license.
And also:
OK, how you going to limit access? Is it just single, non-employed people who can get access? Perhaps you'd like to expand it to small companies? What size cut off? 10? 100? 500? Perhaps the mid-size biotechs should join up, and drop their own subscriptions. And maybe the smaller colleges?
And from "Mrs. McGreevy" herself, who kicked off the discussion:
Alternatively, if enough people think this is something the ACS should be doing, and start demanding it loudly and frequently, perhaps the ACS will get around to doing it. Maybe they don't know what to do, either. Cheaper access to ACS publications? That would be nice. Perhaps even negotiating group payment rates or even (*gasp*) subsidies for access to other publishers' papers? I wonder if that's even possible, but I'll bet the ACS hasn't even considered it up until now. Perhaps there wouldn't need to be an independent library if the ACS were willing to take on the job. We should ask them. (In fact, why stop at the ACS? Maybe the unemployed biologists would be willing to pony up some time, money and lobbying power as well.)
Another idea would be to subsidize journal access with website ads as well as membership dues. Perhaps start a paid online directory of consultants or CROs, sort of the Ye Olde Yellowe Pages model. Perhaps there could be a small fee for posting an RFP or a project up for bid, sort of a classified ad business model to help clients, CROs and consultants find each other. Various small fees for various small services ==> money for subsidized journal access.
OK, those are the journal thoughts so far (other than a number of people who agree that it's a major problem, as do I!) Any more ideas on this aspect to add to the pile? I'd never heard of DeepDyve myself, and they sound interesting: anyone have any experience with them, and is there anyone else in that market niche? We'll put together some action points after this round of ideas. . .
I've had an interesting e-mail from a reader who wants to be signed as "Mrs. McGreevy", and it's comprehensive enough that I'm going to reproduce it in full below.
As everyone but the editorial board of C&E News has noticed, jobs in chemistry are few and far between right now. I found your post on virtual biotechs inspiring, but it doesn't look like anyone has found a good solution for how to support these small firefly businesses until they find their wings, so to speak. Lots of editorials, lots of meetings, lots of rueful headshaking, no real road map forward for unemployed scientists.
I haven't seen this proposed anywhere else, so I'm asking you and your readership if this idea would fly:
What about a voluntary association of independent research scientists?
I'm thinking about charging a small membership fee (for non-profit administration and hard costs) and using group buying power for the practical real-world support a virtual biotech would need:
1. Group rates on health and life insurance.
How many would-be entrepreneurs are stuck in a job they hate because of the the health care plan, or even worse, are unemployed or underemployed and uninsurable, quietly draining their savings accounts and praying no one gets really sick? I have no idea how this would work across state lines, or if it is even possible,but would it hurt to find out? Is anyone else looking?
2. Group rates on access to journals and library services.
This is something I do know a bit about. My M.S. is in library science, and I worked in the Chemistry Library in a large research institution for years during grad school. What if there were one centralized virtual library to which unaffiliated researchers across the country could log in for ejournal access? What if one place could buy and house the print media that start-ups would need to access every so often, and provide a librarian to look things up-- it's not like everyone needs their own print copy of the Canada & US Drug Development Industry & Outsourcing Guide 2012 at $150 a pop. (But if 350 people paid $1 a year for a $350/yr online subscription . . . )
Yes, some of you could go to university libraries and look these things up and print off articles to read at home, but some of you can't. You're probably violating some sort of terms of service agreement the library and publisher worked out anyway. It's not like anyone is likely to bust you unless you print out stacks and stacks of papers, but still. It's one more hassle for a small company to deal with, and everyone will have to re-invent the wheel and waste time and energy negotiating access on their own.
3. How about an online community for support and networking-- places for blogs, reviews, questions, answers, exchanges of best practices, or even just encouragement for that gut-wrenching feeling of going out on your own as a new entrepreneur?
4. What sort of support for grantwriting is out there? Is there a hole that needs to be filled?
5. How about a place to advertise your consulting services or CRO, or even bid for a contract? Virtual RFP posting?
6. Would group buying power help negotiate rates with CROs? How about rates for HTS libraries, for those of you who haven't given up on it completely?
Is there a need for this sort of thing? Would anyone use it if it were available? How much would an unaffiliated researcher be willing to pay for the services? Does anyone out there have an idea of what sort of costs are involved, and what sort of critical mass it would take to achieve the group buying power needed to make this possible?
I'd be happy to spark a discussion on what a virtual biotech company needs besides a spare bedroom and a broadband connection, even if the consensus opinion is that the OP an ill-informed twit with an idea that will never fly. What do you need to get a virtual biotech started? How do we make it happen? There are thousands of unemployed lab scientists, and I refuse to believe that the only guy making a living these days from a small independently-funded lab is Bryan Cranston.
A very worthy topic indeed, and one whose time looks to have come. Thoughts on how to make such a thing happen?
You know, I think that we really are seeing the breakup of the current model of scientific publishing. Spät kommt er, doch er kommt: it's coming on slowly, and in stages, but it's looking more and more inevitable. Take this news from the Wellcome Trust, one of the largest funding agencies in the world for medical research:
Sir Mark Walport, the director of Wellcome Trust, said that his organisation is in the final stages of launching a high calibre scientific journal called eLife that would compete directly with top-tier publications such as Nature and Science, seen by scientists as the premier locations for publishing. Unlike traditional journals, however, which cost British universities hundreds of millions of pounds a year to access, articles in eLife will be free to view on the web as soon as they are published. . .
Walport, who is a fellow of the Royal Society, Britain's premier scientific academy, said the results of public and charity-funded scientific research should be freely available to anyone who wants to read it, for whatever purpose they need it. His comments echo growing concerns from scientists who baulk at the rising costs of academic journals, particularly in a time of shrinking university budgets.
That journal is being launched with the Max Planck Gesellschaft and the Howard Hughes Medical Institute, so it's definitely something worth taking seriously. And outside of that new journal, they're going as far as considering sanctions in funding for any researchers who don't adhere to their open-access policies (basically, free within six months of publication in a journal). I was looking up some papers from back in the 1980s the other day, and was reminded, by contrast, of the policies of some of the commercial publishers: never free, ever, no matter how old it is, or who funded it. Gold, those journal archives are: the costs are long gone; it's all been digitized for years and sits on the servers. But anyone who wants to look at a thirty-year-old paper in Tet Lett had better get ready to pony up. Patents expire, as they should, but copyright? Hah!
Elsevier says in the article that they're committed to offering their customers "choice", and that gosh, the subscription model is just so darn popular that they don't see how they can go against the wishes of their customers by getting rid of it. I particularly enjoyed this quote:
A spokesperson for Elsevier said the company was open to any "mechanism or business model, as long as they are sustainable and maintain or improve existing levels of quality control".
This is the Elsevier that can't manage to fix their own RSS feeds for months, that set up a whole fake-but-real journal division for the advertising revenue, that solicits good textbook reviews on Amazon in exchange for $25 gift cards, that charged people $4500 a year to read a journal stuffed with the editor's own nonsensical papers, and whose chemistry titles repeatedly let through howlers that even undergraduates could have spotted?
That level of quality control is going to be quite a strain to keep up. And yes, I know that the other publishers hardly have clean records, but they managed not to be quoted about their quality control. This time, anyway.
Naturehas a comment on the quality of recent publications in clinical oncology. And it's not a kind one:
Glenn Begley and Lee Ellis analyse the low number of cancer-research studies that have been converted into clinical success, and conclude that a major factor is the overall poor quality of published preclinical data. A warning sign, they say, should be the “shocking” number of research papers in the field for which the main findings could not be reproduced. To be clear, this is not fraud — and there can be legitimate technical reasons why basic research findings do not stand up in clinical work. But the overall impression the article leaves is of insufficient thoroughness in the way that too many researchers present their data.
The finding resonates with a growing sense of unease among specialist editors on this journal, and not just in the field of oncology. Across the life sciences, handling corrections that have arisen from avoidable errors in manuscripts has become an uncomfortable part of the publishing process.
I think that this problem has been with us for quite a while, and that there are a few factors making it more noticeable: more journals to publish in, for one thing, and increased publication pressure, for another. And the online availability of papers makes it easier to compare publications and to call them up quickly; things don't sit on the shelf in quite the way that they used to. But there's no doubt that a lot of putatively interesting results in the literature are not real. To go along with that link, the Nature article itself referred to in that commentary has some more data:
Over the past decade, before pursuing a particular line of research, scientists. . .in the haematology and oncology department at the biotechnology firm Amgen in Thousand Oaks, California, tried to confirm published findings related to that work. Fifty-three papers were deemed 'landmark' studies. . . It was acknowledged from the outset that some of the data might not hold up, because papers were deliberately selected that described something completely new, such as fresh approaches to targeting cancers or alternative clinical uses for existing therapeutics. Nevertheless, scientific findings were confirmed in only 6 (11%) cases. Even knowing the limitations of preclinical research, this was a shocking result.
Of course, the validation attempts may have failed because of technical differences or difficulties, despite efforts to ensure that this was not the case. Additional models were also used in the validation, because to drive a drug-development programme it is essential that findings are sufficiently robust and applicable beyond the one narrow experimental model that may have been enough for publication. To address these concerns, when findings could not be reproduced, an attempt was made to contact the original authors, discuss the discrepant findings, exchange reagents and repeat experiments under the authors' direction, occasionally even in the laboratory of the original investigator. These investigators were all competent, well-meaning scientists who truly wanted to make advances in cancer research.
So what leads to these things not working out? Often, it's trying to run with a hypothesis, and taking things faster than they can be taken:
In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set. For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups. Investigators frequently presented the results of one experiment, such as a single Western-blot analysis. They sometimes said they presented specific experiments that supported their underlying hypothesis, but that were not reflective of the entire data set. . .
This can rise, on occasion, to the level of fraud, but it's not fraud if you're fooling yourself, too. Science is done by humans, and it's always going to have a fair amount of slop in it. The same issue of Nature, as fate would have it has a good example of irreproducibility this week. Sanofi's PARP inhibitor iniparib already wiped out in Phase III clinical trials not long ago, after having looked good in Phase II. It now looks as if the compound was (earlier reports notwithstanding) never much of a PARP1 inhibitor at all. (Since one of these papers is from Abbott, you can see that doubts had already arisen elsewhere in the industry).
That's not the whole story with PARP - AstraZeneca had a real inhibitor, olaparib, fail on them recently, so there may well be a problem with the whole idea. But iniparib's mechanism-of-action problems certainly didn't help to clear anything up.
Begley and Ellis call for tightening up preclinical oncology research. There are plenty of cell experiments that will not support the claims made for them, for one thing, and we should stop pretending that they do. They also would like to see blinded protocols followed, even preclinically, to try to eliminate wishful thinking. That's a tall order, but it doesn't mean that we shouldn't try.
Part way through his project to reproduce promising studies, Begley met for breakfast at a cancer conference with the lead scientist of one of the problematic studies.
"We went through the paper line by line, figure by figure," said Begley. "I explained that we re-did their experiment 50 times and never got their result. He said they'd done it six times and got this result once, but put it in the paper because it made the best story. It's very disillusioning."
As has been noted here in the comments sections, the RSS feeds of Elsevier's journals have been hosed, in various ways, for some time now. Things don't come through, or they don't come through correctly, or they're duplicated, or you get abstracts from journals that you never heard of. How many people - those of you who read journals via RSS - are experiencing these problems? And has anyone gotten Elsevier to respond to any complaints?
When you file a patent application, there are plenty of things that the PTO wants you to include. One of the big ones is prior art: you're supposed to disclose all the relevant inventions close to yours that you're aware of, in order to show how your discovery is different. Prior art is naturally to be found in other previous patent filings, and it's also to be found in journal articles and other such public disclosures. If you don't submit relevant prior art that is known to you, your patent application gets into a lot of trouble eventually (and the more worthwhile your invention, the greater the chance becomes of that catching up with you).
So in light of this, you might find it interesting that some of the large scientific publishers are suing over all this. Why? Well, these lawsuits (filed by Wiley and by the American Institute of Physics) allege that the accused law firms violated copyright by submitting unauthorized copies of journal articles with their patent applications.
As that post at PatentlyO goes on to show, the plaintiffs seem to also be very interested in the internal copies of articles that the law firms are making. But I don't really see how they're going to make either of these stick. I mean, I tend to think that a lot of things are "fair use", but aren't these? This really looks like an act of desperation - the traditional scientific publishing model must be in even worse shape than I thought.
The title of this one says it all: "Association of industry funding with the outcome and quality of randomized controlled trials of drug therapy for rheumatoid arthritis". Any number of critics of the drug business will tell you what that association is: we publish the good stuff and bury the bad news, right?
Well, not so much in arthritis, apparently. The authors identified 103 recent clinical trials in the area, over half of them industry-funded. But when it came to outcomes, things were pretty much the same. Trials from the three largest classes of funding (industry, nonprofit, and "unspecified") all tended to strongly favor the tested drug, although the small number (six) of mixed-funding trials ended up with two favoring and four against. The industry-run trials tended to have more subjects, while the nonprofit ones tended to run longer. The industrial trials also tended to have a more complete description of their intent-to-treat and workflow. As you'd figure, the industrial trials tended to be on newer agents, while the others tended to investigate different combinations or treatment regimens with older ones. But the take-home is this:
No association between funding source and the study outcome was found after adjustment for the type of study drug used, number of study center, study phase, number of study subject, or journal impact factor. . .
. . .Though preponderance of data in medical literature shows that industry funding leads to higher chances of pro-industry results and conclusions, we did not find any association between the funding source and the study outcome of "published" (randomized clinical trials) of RA drug therapies.
The one worrying thing they did find was a trend towards publication bias - the industry-sponsored studies showed up less often in the literature. The authors speculate as to whether these were trials with less favorable outcomes, but didn't have enough data to say one way or another. . .
The sponsors of the Research Works Act (Representatives Carolyn Maloney, D-N.Y., and Darrell Issa, R-California) have announced that they will not be bringing it forward. Elsevier's backtrack was indeed the sign.
It may well be. This morning comes news that Elsevier has dropped support for the RWA, which makes one think that they're feeling the pressure:
We have heard expressions of support from publishers and scholarly societies for the principle behind the legislation. However, we have also heard from some Elsevier journal authors, editors and reviewers who were concerned that the Act seemed inconsistent with Elsevier’s long-standing support for expanding options for free and low-cost public access to scholarly literature. That was certainly not our intention in supporting it. . .
While we continue to oppose government mandates in this area, Elsevier is withdrawing support for the Research Work Act itself. We hope this will address some of the concerns expressed and help create a less heated and more productive climate for our ongoing discussions with research funders. . .
You can smell the smoke from the brake pads, and hear the reverse gears engaging. Maybe now the American Chemical Society will make a public statement - I haven't heard anything from them yet, although their default position (as a member of the AAP) is to support it. (They supported a previous version of the bill in 2008).
I don't know how many of you out there like to form azides, but if you do, you've probably used (or thought about using) imidazole-1-sulfonyl azide hydrochloride. This reagent appeared in Organic Letters a few years ago as a safe-to-handle shelf-stable azide transfer reagent, and seems to have found popularity. (I've used it myself).
So it was with some alarm that I noted this new paper on the stability and handling characteristics of the reagent. It's a collaboration between the University of Western Australia (where the reagent was developed, partly by the guy whose lab bench I took over in grad school back in 1983, Bob Stick), the University of British Columbia, and the Klapötke group at Munich. That last bunch is known to readers of "Things I Won't Work With", as experts in energetic materials, and when I saw that name I knew I'd better read the paper pronto.
As it turns out, the hydrochloride isn't quite as optimal as thought. It's impact-sensitive, for one thing, and not shelf-stable. The new paper mentions that it decomposes with an odor of hydrazoic acid on storage - you don't want odors of hydrazoic acid, believe me - and I thought while reading that, "Hmm. My bottle of the stuff is white crystalline powder; that's strange." But then I realized that I hadn't looked at my bottle for a few months. And as if by magic, there it was, turning dark and gooey. I had the irrational thought that the act of reading this paper had suddenly turned my reagent into hazardous waste, but no, it's been doing that slowly on its own.
So if you have some of this reagent around, take care. The latest work suggests that the hydrogensulfate salt, and especially the fluoroborate, are less sensitive and more stable alternatives to the hydrochloride, and I guess I'll have to make some at some point. (They also made the perchlorate - just for the sake of science, y'know - and report, to no one's surprise, that it "should not be prepared by those without expertise in handling energetic materials"). But it needs no ghost come from the grave to tell us this.
So, back to my lab and my waste-disposal problem! And here's a note on the literature. We have the original prep of the reagent, a follow-up note on stability problems, and this latest paper on alternatives. But when you go back to the original paper, there is no mention of the later hazard information. Shouldn't there be a note, a link, or something? Why isn't there? Anyone at Organic Letters or the ACS care to comment on that?
Update: I've successfully opened my bottle, with tongs and behind a blast shield, just to be on the safe side, and defanged the stuff off by dilution.
There's been a movement afoot to boycott Elsevier journals. It's started over in the mathematics community, led by Timothy Gowers, a serious mathematician indeed. The objections to Elsevier are the ones you'd think: high prices, unsplittable bundles of journal subscriptions for institutions, and their strong support for legislation like the Research Works Act.
Writing about this is tricky, since I'm on the editorial board of an ACS journal that competes with Elsevier titles. Of course, as that link in the first paragraph shows, Nature Publishing Group has no problem talking about the issue themselves, and they're competing tooth and claw with Elsevier. At any rate, there's now a central website for the boycott movement, and it continues to gain publicity. There are, of course, some field where Elsevier is more prominent than others - biomedically, the Cell Press journals (and The Lancet) are heavy hitters, so a real test of this movement will be to see how many people from these fields it can attract.
Personally, I think that the current system of scientific publishing is increasingly outmoded, although I'm loath to forecast what will replace it. But we could be looking at another step in its demise.
Here's a helpful translation, and there's more truth in it than there should be. My rule of thumb is to be extremely suspicious of a methods paper that doesn't have at least a couple of low-yielding or "NR" entries. If they aren't there, it means that someone didn't do enough experiments (or, worse, that they're not telling you about those little details).
The editor of the journal Life has published an attempt at detailing how the notorious Andrulis paper managed to make its way into print. See how convincing you find it. In the course of explaining that it can be hard to find reviewers for interdisciplinary topics, and how the journal tries to find reputable people in each field (and carefully checks author suggestions for reviewers), we have this:
Life is a new journal that deals with new and sometime difficult interdisciplinary matters. Consequently, the journal will occasionally be presented with submitted articles that are controversial and/or outside conventional scientific views. Some papers recently accepted for publication in Life have attracted significant attention. Moreover, members of the Editorial Board have objected to these papers; some have resigned, and others have questioned the scientific validity of the contributions. . .
. . .In the case of the Dr. Andrulis’s long paper, the two reviewers were both faculty members of reputable universities different than the author’s and both went to considerable trouble presenting lengthy review reports. Dr. Andrulis revised his manuscript as requested, and the paper was subsequently published.
Really? Is that how it really went? I know what I would have said if they'd sent the paper to me: that it was a perfect example of what happens when an active, learned mind begins to slip loose from its moorings, and that while the paper appeared to have no scientific merit at all, it was quite useful as a diagnostic sign of oncoming psychosis.
If you only read the Life editor's remarks without reading any of the original paper, you might find them reasonable. But that's because you haven't been exposed to a theory that purports to explain the abiotic origins of life, the underlying principles of biochemistry, the formation of the solar system, the expansion of the universe, global weather patterns, the structure of cellular membranes, the distributions of comets and asteroids, the origins of riboviruses, the protein folding problem, the nature of biological aging, and the unification of quantum mechanics with general relativity. I have not made any of that up, it's all in the paper, and I would very much like to see a reviewer who could let all that go past. "Publish with revisions", sure.
The fallout from the bizarre Andrulis paper continues. Carl Zimmer reports that editorial board members are resigning from the journal, having had no idea that their names would wind up over something like this.
Naturally, that brings up the question of just who did let this thing through the review process, but my bet is that we'll never know. Whoever signed off on it is no doubt running for cover.
Another useful feature this affair has had is the chance to see who just posts press releases for fun and profit, and who has some tiny residual bit of editorial discretion. In the former category, apparently, are PhysOrg.com and ScienceDaily.com (the latter has taken down their post. But then again, the Times of Indiabit for it as well. . .
Here's one of the strangest things I've ever seen in the scientific literature. A new journal, Life, apparently solicited papers for their inaugural issues, and one of them was from Erik Andrulis at Cast Western's School of Medicine. The manuscript came in at 105 printed pages, which should have rung at least a tiny alarm bell, you'd think. And if that wasn't a bit concerning, perhaps the title ("Theory of the Origin, Evolution, and Nature of Life") might have seemed a bit sweeping? Or the abstract, which promises that "The theoretical framework unifies the macrocosmic and microcosmic realms, validates predicted laws of nature, and solves the puzzle of the origin and evolution of cellular life in the universe." No? Nothing to worry about yet?
But editors aren't supposed to just look at page counts, titles, and abstracts. Just a riffle through the actual manuscript should have been enough to convince anyone that, rather than a Theory of Everything, that this work is, most unfortunately, the product of a disordered mind. P. Z. Myers has excerpts from the paper on his blog - take a look and see what you think. Here's a sample, and it should really be sufficient:
The ultimate state of gyromnemesis is the stably adapted particle or gyronexus in the gyrobase. . .Finally, although a diquantal IEM (X'') undergoes gyrognosis as the gyrobase of a primary majorgyre, it undergoes gyromnemesis as the gyrapex of an alternagyre.
Right. The paper ranges through the origins of life, organic chemistry, cosmology, geology, astronomy, and who knows what else, all of it explained in language exactly like the above. And yes, there is a multi-page glossary of all those gyro-terms, and no, it does not help. As Myers points out, the spectacularly weird thing is that not only did this paper get published, it got press-released by Case Western. Here, check it out. Whoever put this thing together has gamely attempted to summarize the paper, and not only that, to highlight its importance for the greater glory of Case Western:
To test his paradigm, Dr. Andrulis designed bidirectional flow diagrams that both depict and predict the dynamics of energy and matter. While such diagrams may be foreign to some scientists, they are standard reaction notation to chemists, biochemists, and biologists.
Dr. Andrulis has used his theory to successfully predict and identify a hidden signature of RNA biogenesis in his laboratory at Case Western Reserve University School of Medicine. He is now applying the gyromodel to unify and explain the evolution and development of human beings.
Oh, go take a look and tell me if you see any standard notation. (Update: I see from RetractionWatch that the university has pulled the release from their own sites, saying that they're "evaluating our processes regarding media outreach". I'll bet they are(. Now, I realize that picking up a text on, say, quantum electrodynamics could lead to the same what-is-this-stuff feeling. But any text on QED starts with a grounding in the physical world and the connections of the theory to known physics. And this sort of thing is different in both degree and kind (for one thing, QED has nothing to say about lunar craters). There's a difference between a work that makes you think "Boy, I don't understand this" and one that makes you think "Boy, this person has lost it". The near-infallible signs of scientific derangement include the "Why, this explains everything" aspect, the "Everything you thought you knew is wrong" one, and the intricate details-within-details style, almost always taken to unbearable lengths.
What the Andrulis paper reminds me of, actually, is Alfred Lawson and his Lawsonomy. That one also explains everything from bacteria to the composition of the moon, and brings in "zig-zag and swirl" motions to do so, at excruciating length. No, if you've had any exposure to the fill-the-margins-with-green-ink thinkers, you'll recognize Andrulis' problem, and hope that he can get some sort of help for it. Here's a book-length collection of such, very interesting for what it shows you about the ways that human reason can go off the rails.
That's something I've thought about for a long time - in fact, here's an entry on this blog from ten years ago on that very subject. It's interesting to me that there are a limited number of relatively defined mental illnesses; I think that says something about the deeper structures of human consciousness. The Andrulis paper is a flawless example of one of those categories - the wildly intricate, over-systematized Key to the Universe. I've just never seen one in a scientific journal.
I noted that the Nature Publishing Group has come out against the proposed Research Works Act, which would roll back the requirement that research funded by the US government be made freely available after (at most) one year. They are, I believe, the largest and most prominent journal publisher to take such a stand (although I'll be glad to be wrong about that):
NPG and Digital Science do not support the Research Works Act.
NPG and Digital Science exist to support the creation and dissemination of human knowledge on a sustainable commercial basis. We seek to enable the open exchange of ideas, especially in scientific communities, in line with the requirements and objectives of relevant stakeholders.
Update: from the comments, the AAAS (publishers of Science) have also come out against the RWA, saying that they're fine with the current system, and that their membership in the AAP does not mean that the organization speaks for them on this issue. How about the American Chemical Society? As far as I can tell, the ACS has made no statement, and silence speaks loudly.
Meanwhile, Rich Apodaca at Depth-First surprised me with this post coming out in favor of the RWA. But read the whole thing. He is, as the Marxists used to say, interested in "heightening the contradictions", and sees the scientific publishing industry bringing down the roof on its head even faster if the act passes. And the sooner that happens, he says, the sooner we can get rid of an outmoded system:
Any scientist who has been an active participant in scientific publication as an author, reviewer, and consumer recognizes that the only remaining value added by scientific publishers today is imprimatur. Imprimatur is the implied endorsement received by authors who publish in certain scientific journals, particularly in those that earned a high level of prestige during the pre-digital period of publication scarcity.
Ironically, imprimatur remains so valuable in science that it has kept numerous publishers afloat despite wave upon wave digital destruction being visited on sister industries such as book publication and newspapers.
But imprimatur can lose its luster, particularly in an environment in which fewer and fewer scientist can actually read the publications appearing in ‘high-impact’ journals. Prestige counts for nothing in science if your peers can’t read your papers. Nevertheless, that’s where scientific publication is heading.
I'm not sure which way is faster, myself. But we agree that the current scientific publishing model is being eroded, and that this is an opportunity, not a disaster that has to be repaired with legislation.
Back in December, a short bill was introduced in the House called the "Research Works Act". Its backers, Darrell Issa (R-CA) and Carolyn Maloney (D-NY), describe it as something that will maintain the US's standing in scientific publishing. After looking over its language and reading a number of commentaries on it, I have to disagree: this looks to me like shameless rent-seeking by the commercial scientific publishers.
And it pains me to say that, because I know several people in that business. But it's a business whose long-term model has problems. (See the Addendum below if you're not in the field and want a brief summary of how scientific publishing works). The problem is, the work of the editorial staff has changed a good deal over the years. Back when everyone sent in hard copies of papers, in who knows what sort of format, there was a good deal of work to do just turning the good ones into a consistent journal. Electronic submission has ironed a lot of the grunt work out - it's still work, but it's not what it used to be.
That leaves the higher editorial functions themselves, and here's where the arguing starts. Most, and in some cases all editing of content is done by unpaid peer reviewers. There are journals whose editors exist mainly to keep the flow of submissions moving to the reviewers, and from them back into the official journal, while hardly ever laying a finger on the copy itself. They function as Peer Review Mailroom Managers. And while that's a necessary job, it's the center of the argument about scientific publishing today. How much, exactly, is it worth?
Scientific journal are expensive. I mean, really, really expensive to subscribe to. And if you're not a subscriber, access to individual papers is pretty steep, too - typically in the $15 to $50 range. This is the business model for commercial scientific publishing: create a space with value (reputation, name recognition) and charge the maximum that that traffic will bear. And that's fine; there are a lot of businesses that work the same way - if they can.
The problem is, the information-sharing capabilities of the Internet blow a large hole in some of the traditional publishing model. And another problem is that a large number of papers that come into the journals from US academic researchers have had some (or all) of that work paid for by government grants (NIH, NSF, DOE and so on). As it stands, articles funded by the NIH are available in PubMed Central for free access, no later (by law) than 12 months from the initial journal publication. Researchers can also submit their work to "open access" journals (such as those from the Public Library of Science), which charge a fee to authors to defray editorial costs, but then allow immediate unlimited access to all comers once a paper is accepted. (I should note that some commercial journals get away with "page charges" as well, and some have a model where the authors can pay extra to bring their paper out from behind the paywall).
And here's where we have the Research Works Act. It would forbid any publication in an open access journal for anything funded in academia by US government grants, and it would forbid any public-access repository for such work. That's its purpose. Well, to be more accurate, its purpose, as described by the head of the Association of American Publishers, is that it "ensures the sustainability of the industry". Yep, make my business model part of statutory law, and beggar my competition: what else is a government for, anyway?
Update: see the comments section. I'm interpreting the text of the law to mean the above, but another way to read it - probably the correct one - is that it's mainly rolling back the 2008 law that mandates that NIH-funded papers go open-access after a year. But that's bad enough as it stands.
To their credit, the MIT Press looks like the first big academic publisher to defect from this position. But the commercial publishers (Elsevier, Wiley, and so on) will never give up on this goal. Yes, the RWA, according to them, is aimed at "preventing regulatory interference with private-sector research publishers". Here's Congresswoman Maloney using Elsevier's own press release language, sentence by sentence, as detailed by Michael Eisen, co-founder of PLoS. (He also has an op-ed in the New York Times on this issue).
I see no reason why we should make the current scientific publishing system a matter of law. I think it should change - and be allowed to change - as new technology allows it to. And I think that the Research Works Act is nothing more a blatant attempt to hold on to a profitable business plan.
Addendum: For those outside the scientific world, here's a brief summary of how things have traditionally worked. As a scientist (academic or industrial), you take the time and effort to write up your results for a journal. You have to pick your journal at the start of the process, since each of them have their own ways of organizing a paper, their own preferred way of citing other papers as references, and so on. Anyone can send anything to any journal they feel like, although you'd be well advised to target your paper to the ones that (a) have the best chance to actually accept it and (b) will do you good to have a paper published in. The overlap between those two may not be large, or may not exist at all, depending on your paper. These days, most journals have templates for Word or the like, which standardizes the submissions, and have some sort of automatic PDF generation during the submission step step so you can see how the paper will look when formatted in the journal's style and page layout.
Your manuscript is given a quick check to make sure that it's appropriate for review. A few of of the higher-end journals make this a key step, because they can afford to turn down even rather interesting papers as not necessarily worth their time to go on checking. But in most journals, unless there's something obviously off, your paper goes out for peer review. Two or more scientists from Out There Somewhere look it over and send in comments. Those comment forms have a section for the original authors to see, and a section for remarks that go just to the editorial staff, and you can use those as you see fit. (I, for example, once used the latter forum to ask the editors to please stop sending me papers from a certain author, because I'd done three of them and couldn't stand to see any more. They honored my request.)
As an author, you see the comments when they come back and get a recommendation from the journal - usually it's "Publish as is", "Publish after minor revisions", "Publish after major revisions", or "Go away". That last one usually isn't expressed in quite those words. The middle two are the most common, since most stuff eventually gets published somewhere if the authors are persistent enough (and are willing enough to have their work appear in the Zambodian Journal of Chemistry or what have you).
Now at this point, traditionally, the work of assembling the accepted papers into a printed journal kicks in on the editorial side. And it still does, but that process is becoming less and less important. I honestly can't tell you when I last saw a hard copy of any of the journals I read regularly. Even the idea of separate issues is becoming antiquated, since new papers (in the case of many journals) just plop out onto the web site (and into the RSS feeds) as they emerge from the review process.
I'm getting all the press releases from Bill Sardi, of Resveratrol Partners, as he does damage control from the Das scandal at UConn. And I have to say, he's putting in the hours getting these together. Problem is, on some key points, he doesn't know what his biggest problems are.
The latest one is titled "World Without Resveratrol: Researcher Falsely Accused", and claims that this may all be a plan to "send a message" to any academic who collaborates with the makers of resveratrol pills. The release goes on about how these are old accusations, which Das has refuted since then, and asks why these ancient concerns are coming up now, eh? The phrase "orchestrated hit job" is used. But that glosses over the times of the whole investigation, which has been a very detailed and involved one, and glosses over the amount of due process involved as well. There are a lot of problems with the publications from the Das lab, as detailed in the report that I linked to the other day, and tying them together has involved a lot of work.
But here comes my favorite part of the latest Sardi release:
". . .I asked Dr. Das directly, did he altered (sic) western blot images, or directed others in his lab to do so. While his initial answer was no, meaning he had not fabricated or altered any scientific finding, altering western blot images are a common practice in laboratories for reasons other than deception. The university chose to present their findings in a derogatory manner. Dr. Das explains that editors at scientific publications commonly request researchers enhance faded images of western blot tests so they can be duplicated in their publications. Western blot tests are frequently altered to remove backgrounds, enhance contrast and increase dots-per-inch resolution so they are suitable for publication. This had been fully explained to university officials long before. . .
No, no, no. The problems with the Das papers have nothing to do with enhancing the contrast on Western blots. They have to do with cutting and pasting sections of them, rearranging them, reusing them, and creating them out of pieces of other experiments. Look at that report. These people appear to have spent a ridiculous amount of time assembling "Western blots" out of miscellaneous digitized chunks. The resulting figures purport, in many cases, to represent particular experiments, but they do no such thing. They represent a bunch of previous bands from other experiments entirely, sliced and diced in a way that would seem to have no other possible motive than to deceive. Come on.
Oh, but there's more. Here, according to the press release, is how a cutting-edge academic lab works these days:
"As I drilled Dr. Das’ former students with questions, I found that lead researchers like Dr. Das do not do any lab bench experiments. Students do all the work and submit their results to him via e-mail or by directly downloading data into his computer. Dr. Das says when he is not traveling his office is open and students can enter and download data directly onto his computer. I had previously visited Dr. Das at the University of Connecticut and noticed his office door was left open and anyone could have access to his computer.
One former student told me that typically lead researchers like Dr. Das write the introduction and conclusion of experiments and the students enter all the data, before publication in scientific journals. Dr. Das, who is busy lecturing all over the globe because of his groundbreaking studies, does not directly oversee tests that are performed, and neither do most other lead researchers. The University of Connecticut report says the university holds Dr. Das responsible for all of the data. Probably most lead researchers in scientific laboratories around the globe are vulnerable to errors or even fabrication of data by their students."
Where to start? What the heck is this "download data directly into his computer" stuff? And what about all the doctored files found on other machines in the group? And yes, while lead authors are indeed vulnerable to errors and fabrication, this sort of thing typically does not involved years of work spread out across dozens of papers in multiple journals. Even the busiest and most distracted principal investigator might be expected to take the time to notice, eventually, that his group's work is a tower of fraud. And yes, the University should hold Dr. Das responsible for the data in his papers. His name is on the grants, his name is on the office door, he's the one with a high-paying tenured position while the students are cranking away under low salaries and stipends, and it's his name with an asterisk next to it on all those papers, as the contact person for any questions about them. Damn right he's responsible. He's responsible for making sure that anything going out into the literature with his name on it is something that he can stand behind.
Ah, but not to worry. It's all being taken care of:
"Dr. Das says many editors at scientific journals don’t believe the University of Connecticut report. They full-well know that editing of western blot tests is common practice and that the tests in question in no way invalidate his work and were only one part of the evidence provided in his papers from which Dr. Das drew conclusions. This is the case of scientific fraud that wasn’t."
That would explain why Dr. Das has been pulled from the co-editor job he had at one of those journals. They must believe him. And that would also shore up all those allegations of prejudice against East Indian researchers, since the editor of that journal is. . .well, he's Indian too, but you know what I mean. (Personally, if I were from India myself, I'd be furious at Das for helping to drag the reputation of my country's scientists through the mudhole, but maybe that's just me.)
No, I hope these press releases keep on coming. So far, we have lots of elaborate reasons why Dr. Das had nothing to do with all these fabricated Western blots, but who cares, right, since they're only a tiny part of his papers, which are great and important work even though he really doesn't write them anyway, and no, he has almost no connection with Longevinex and Resveratrol Partners, which is why the head of the company is spending all this time defending him in this case of minor stuff he never did, all 600 pages of summary and 60,000 pages of investigation material, and that explains why the journals that believe him are ditching him from their mastheads and publishing retractions of those great papers. Because it's all a conspiracy. Yeah. That's it.
If you had SciFinder access, but are now unemployed and would like to use it during your job hunt, CAS now has a program to make that possible for free. I'm glad to see them taking this step; a lot of people have asked for something like this for some time now.
A new paper in Angewandte Chemie tries to open another front in relations between academic and drug industry chemists. It's from several authors at GSK-Stevenage, and it proposes something they're calling "Lead-Oriented Synthesis". So what's that?
Well, the paper itself starts out as a quick tutorial on the state and practice of medicinal chemistry. That's a good plan, since Angewandte Chemie is not primarily a med-chem journal (he said with a straight face). Actually, it has the opposite reputation, a forum where high-end academic chemistry gets showplaced. So the authors start off by reminded the readership what drug discovery entails. And although we've had plenty of discussions around here about these topics, I think that most people can agree on the main points laid out:
1. Physical properties influence a drug's behavior.
2. Among those properties, logP may well be the most important single descriptor,
3. Most successful drugs have logP values between 1 and perhaps 4 or 5. Pushing the lipophilicity end of things is, generally speaking, asking for trouble.
4. Since optimization of lead compounds almost always adds molecular weight, and very frequently adds lipophilicity, lead compounds are better found in (and past) the low ends of these property ranges, to reduce the risk of making an unwieldy final compound.
As the authors take pains to say, though, there are many successful drugs that fall outside these ranges. But many of those turn out to have some special features - antibacterial compounds (for example) tend to be more polar outliers, for reasons that are still being debated. There is, though, no similar class of successful less polar than usual drugs, to my knowledge. If you're starting a program against a target that you have no reason to think is an outlier, and assuming you want an oral drug for it, then your chances for success do seem to be higher within the known property ranges.
So, overall, the GSK folks maintain that lead compounds for drug discovery are most desirable with logP values between -1 and 3, molecular weights from around 200 to 350, and no problematic functional groups (redox-active and so on). And I have to agree; given the choice, that's where I'd like to start, too. So why are they telling all this to the readers of Angewandte Chemie? Because these aren't the sorts of compounds that academic chemists are interested in making.
For example, a survey of the 2009 issues of the Journal of Organic Chemistry found about 32,700 compounds indexed with the word "preparation" in Chemical Abstracts, after organometallics, isotopically labeled compounds, and commercially available ones were stripped out. 60% of those are outside the molecular weight criteria for lead-like compounds. Over half the remainder fail cLogP, and most of the remaining ones fail the internal GSK structural filters for problematic functional groups. Overall, only about 2% of the JOC compounds from that year would be called "lead-like". A similar analysis across seven other synthetic organic journals led to almost the same results.
Looking at array/library synthesis, as reported in the Journal of Combinatorial Chemistry and from inside GSK's own labs, the authors quantify something else that most chemists suspected: the more polar structures tend to drop out as the work goes on. This "cLogP drift" seems to be due to incompatible chemistries or difficulties in isolation and purification, and this could also illustrate why many new synthetic methods aren't applied in lead-like chemical space: they don't work as well there.
So that's what underlies the call for "lead-oriented synthesis". This paper is asking for the development of robust reactions which will work across a variety of structural types, will be tolerant of polar functionalities, and will generate compounds without such potentially problematic groups as Michael acceptors, nitros, and the like. That's not so easy, when you actually try to do it, and the hope is that it's enough of a challenge to attract people who are trying to develop new chemistry.
Just getting a high-profile paper of this sort out into the literature could help, because it's something to reference in (say) grant applications, to show that the proposed research is really filling a need. Academic chemists tend, broadly, to work on what will advance or maintain their positions and careers, and if coming up with new reactions of this kind can be seen as doing that, then people will step up and try it. And the converse applies, too, and how: if there's no perceived need for it, no one will bother. That's especially true when you're talking about making molecules that are smaller than the usual big-and-complex synthetic targets, and made via harder-than-it-looks chemistry.
Thoughts from the industrial end of things? I'd be happy to see more work like this being done, although I think it' going to take more than one paper like this to get it going. That said, the intersection with popular fragment-based drug design ideas, which are already having an effect in the purely academic world of diversity-oriented synthesis, might give an extra impetus to all this.
The folks behind Retraction Watch (Adam Marcus and Ivan Oransky) have a piece in Nature on what it's been like since they started the blog.
They note a new trend - for new data to appear when a retraction is called for (or made), but without any clarity about whether these new corroborative results have been peer-reviewed themselves. And they're absolutely right that a retraction should state exactly why the paper is being retracted; those "This paper has been withdrawn by the authors" notices are less than useless.Their experiences have them calling for a different way of looking at scientific papers in general:
". . . It is important to point out that an increase in retractions isn't necessarily a bad thing, because they correct the scientific record. But the greater visibility of papers and retractions today adds to the evidence revealing why editors need to handle retractions more transparently. In turn, researchers need to stop emphasizing the paper so much.
What is needed, instead, is a system of publication that is more meritocratic in its evaluation of performance and productivity in the sciences. It should expand the record of a scientific study past an individual paper, including additional material such as worthy blog posts about the results, media coverage and the number of times that the paper has been downloaded."
It's true that more and more of this is being done out here on the internet, in public, and in real time. (I'm glad to say that some of it is done on this site). The new Crossmark system (now being tested) might be a way to keep up with all these extensions, and link them to the original paper. Such a system would have come in very handy indeed during the "arsenic bacteria" business, during which just finding all the useful comments was a real job in itself. There are authors who will not care for this sort of thing, but when you publish a paper, you're opening the door to public comment (and criticism). It's just that now we have the tools to do that more quickly and thoroughly.
I wanted to let people know that I've got a "Perspective" piece in ACS Medicinal Chemistry Letters, entitled "Nowhere to Go But Up?". The journal is starting to run these opinion/overview articles, and contacted me for one - I hope it's the sort of thing that they were looking for!
Since the year is winding down, I had a question for the readership: what's the best scientific paper you read this year? Let's restrict the field to chemistry and biology; I'm not ready for particle physics. And I don't necessarily mean "best written", although you're free to nominate some if you can find any that stood out. What I had in mind was more "Most interesting" or "Most unexpectedly useful", or "Most surprising". I've got a couple in mind myself - I'll turn the suggestions into another post or two.
There's a new paper coming to the defense of rhodanines, a class of compound that has been described as "polluting the scientific literature". Industrial drug discovery people tend to look down on them, but they show up a lot, for sure.
This new paper starts off sounding like a call to arms for rhodanine fans, but when you actually read it, I don't think that there's much grounds for disagreement. (That's a phenomenon that's worth writing about sometime by itself - the disconnects between title/abstract and actual body text that occur in the scientific literature). As I see it, the people with a low opinion of rhodanines are saying "Look out! These things hit in a lot of assays, and they're very hard to develop into drugs!". And this paper, when you read the whole thing, is saying something like "Don't throw away all the rhodanines yet! They hit a lot of things, but once in a while one of them can be developed into a drug!" The argument is between people who say that elephants are big and people who say that they have trunks.
The authors prepared a good-sized assortment of rhodanines and similar heterocycles (thiohydantoins, hydantoins, thiazolidinediones) and assayed them across several enzymes. Only the ones with double-bonded sulfur (rhodanines and thiohydantoins) showed a lot of cross-enzyme potency - that group has rather unusual electronic properties, which could be a lot of the story. Here's the conclusion, which is what makes me think that we're all talking about the same thing:
We therefore think that rhodanines and related scaffolds should not be regarded as problematic or promiscuous binders per se. However, it is important to note that the intermolecular interaction profile of these scaffolds makes them prone to bind to a large number of targets with weak or moderate affinity. It may be that the observed moderate affinities of rhodanines and related compounds, e.g. in screening campaigns, has been overinterpreted in the past, and that these compounds have too easily been put forward as lead compounds for further development. We suggest that particularly strong requirements, i.e. affinity in the lower nanomolar range and proven selectivity for the target, are applied in the further assessment of rhodanines and related compounds. A generalized "condemnation" of these chemotypes, however, appears inadequate and would deprive medicinal chemists from attractive building blocks that possess a remarkably high density of intermolecular interaction points.
That's it, right there: the tendency to bind off-target, as noted by these authors, is one of the main reasons that these compounds are regarded with suspicion in the drug industry. We know that we can't test for everything, so when you have one of these structures, you're always fearful of what else it can do once it gets into an animal (or a human). Those downstream factors - stability, pharmacokinetics, toxicity - aren't even addressed in this paper, which is all about screening hits. And that's another source of the bad reputation, for industry people: too many times, people who aren't so worried about those qualities have screening commercial compound collections, come up with rhodanines, and published them as potential drug leads, when (as this paper illustrates), you have to be careful even using them as tool compounds. Given a choice, we'd just rather work on something else. . .
Readers may remember a now-retracted paper that I first blogged about here, the one that claimed to have isolated the reverse transcriptase inhibitor nevirapine as a natural product. Moreover, it claimed that the isolated material was chiral, which would have been very interesting indeed if it were true. (And, as that last post says, would have been worth making a big point of, if the authors really had understood what they were claiming).
Now a group from Manchester has weighed in on that topic. And what they find is what anyone who'd examined the field should have expected: that the nevirapine molecule, although capable of existing in two chiral forms, equilibrates between them on a time scale of seconds at room temperature. Isolating the atropisomers by standard means is not possible.
So everything about that original Tetrahedron paper was wrong; it never should have made it through the review process. And that's why I highlight such things - not to heap scorn on the original authors, which doesn't do that much good, but to heap it on the people who let such papers into print. Reviewers and editors are supposed to notice when a paper has made very unusual claims, and they're supposed to ask the authors to back them up. But the folks at Tetrahedron were asleep at the switch when this one came through. It's important for them (and other editorial staffs) not to let that happen, and it's important for a journal's readers to realize that it can.
Addendum - as an aside, I note that one of this blog's entries (the second link above) is cited in the references of this latest paper. I'm glad to be a cite-able source!
Someone has to make fun of lousy graphical abstracts in science journals. And apparently this is that person. I've no idea who this is, but they're helping, in their way, to make the world a better place.
It seems that the credibility of the scientific literature has been taking a beating recently. This has come about for several reasons, and through several different motivations. I'll get one of the most important out of the way first - politics. While this has been a problem for a long time, there's been a really regrettable tendency in US politics the last few years, a split across broadly left/right lines. Cultural and policy disagreements have led to many on the left claiming the Dispassionate Endorsement of Settled Science, while others on the right end up complaining that it's nothing of the sort, just political biases given a quick coat of paint. Readers will be able to sort several ongoing controversies into that framework.
Political wrangling keeps adding fuel to the can-we-trust-the-literature argument, but it would still be a big issue without it. Consider the headlines that the work of John Ioannidis draws. And there's the attention being paid to the number of retractions, suspicions of commercial bias in the medical literature, the problems of reproducibility of cutting-edge results, and to round it all off, several well-publicized cases of fraud. No, even after you subtract the political ax-grinding, there's a lot of concern left over (as there should be). There are some big medical and public policy decisions to be made based on what the scientific community has been able to figure out, so the first question to ask is whether we've really figured these things out or not.
A couple of recent articles prompted me to think about all this today. The Economist has a good overview of the Duke cancer biomarker scandal, with attention to the broader issues that it raises. And Ben Goldacre has this piece in The Guardian, highlighting this paper in Nature Neuroscience. It points out that far too many papers in the field are using improper statistics when comparing differences-between-differences. As everyone should realize, you can have a statistically significant effect under Condition A, and at the same time a lack of a statistically significant effect under Condition B on the same system. But that doesn't necessarily mean that the difference between using Condition A versus Condition B is statistically significant. You need to go further (usually ANOVA) to be able to say that. The submission guidelines for Nature Neuroscience itself make this clear, as do the guidelines for plenty of other journals. But it appears that a huge number of authors go right ahead and draw the statistically invalid comparison anyway, which means that the referees and editors aren't catching it, either. This is not the sort of thing that builds confidence.
So the questions about the reliability of the literature are going to continue, with things like this to keep everyone slapping their foreheads. One can hope that we'll end up with better, more reliable publications when all this is over. But will it ever really be over?
Here's another article in the Guardian that makes some very good points about the way we judge scientific productivity by published papers. My favorite line of all: "To have "written" 800 papers is regarded as something to boast about rather than being rather shameful." I couldn't have put it better, and I couldn't agree more. And this part is just as good:
Not long ago, Imperial College's medicine department were told that their "productivity" target for publications was to "publish three papers per annum including one in a prestigious journal with an impact factor of at least five.″ The effect of instructions like that is to reduce the quality of science and to demoralise the victims of this sort of mismanagement.
The only people who benefit from the intense pressure to publish are those in the publishing industry.
Working in industry feels like more of a luxury than ever when I hear about such things. We have our own idiotic targets, to be sure - but the ones that really count are hard to argue with: drugs that people will pay us money for. Our customers (patients, insurance companies, what have you) don't care a bit about our welfare, and they have no interest in keeping our good will. But they pay us money anyway, if we have something to offer that's worthwhile. There's nothing like a market to really get you down to reality.
RetractionWatch has this gem from the Journal of Clinical Microbiology. My favorite part is midway through: "Moreover, we realized after our article had been published that major parts of the text had been plagiarized almost verbatim. . .".
Oh, yeah. There is that. But there's more at the link. The RetractionWatch people are trying to get more details, but I wish them luck. This looks like one of those things that no one is going to be very happy to talk about. . .
Nature has an article on the 20th anniversary of the ArXiv preprint server by its founder Paul Ginsparg. That's gradually worked its way through many parts of physics and mathematics to become a major route for releasing scientific results. (They're now getting about 7,000 submissions a month, with a million downloads a week). But many areas of science remain untouched:
Physicists were quick to adopt widespread sharing of electronic preprints, but other researchers remain reluctant to do so. Fields vary widely in their attitudes to data and ideas before formal review, and in choosing to share electronic preprints, each community will have to develop policies and protocols best suited to their users. A talk I gave in 1997 to a group of biologists helped catalyse the resource now known as PubMedCentral — run by the US National Institutes of Health. I served on the initial advisory board, which soon decided not to host any unrefereed materials, even carefully quarantined, in part for fear of losing essential publisher participation. There remain many legitimate reasons for individual researchers to prefer to delay dissemination, from uncertainty over correctness, to retaining extra time for follow-ups, to sociological differences in the way publication is regarded — in certain fields, the research somehow doesn't count until peer reviewed.
No community that has adopted arXiv usage has renounced it, however, so the growth has been inexorable. . .
But back in its early days, it looked like it would be even more inexorable than that:
The idea that print journals had outlived their usefulness was already in the air in the early 1990s. David Mermin memorably wrote in Physics Today in 1991: “The time is overdue to abolish journals and reorganize the way we do business.”1 By the mid 1990s, it seemed unthinkable that free and unfettered access to non-refereed papers on arXiv would continue to coexist indefinitely with quality-controlled but subscription-based publications. Fifteen years on, researchers continue to access both, successfully compartmentalizing their different roles in scholarly communication and reward structures.
The transition to article formats and features better suited to modern technology than to print on paper has also been surprisingly slow. Page markup formats, such as PDF, have only grudgingly given way to XML-based ones that support features such as manipulable graphics, dynamic views, linked annotations and semantic markup. . .
Well, in chemistry, that transition has been even slower. Our field is still very much dominated by the societies (ACS, RSC, etc.) and the commercial publishers like Elsevier, Nature, Wiley, etc. When's the last time - when's the first time - you heard of a significant organic chemistry paper appearing anywhere else? There's absolutely no equivalent of the ArXiv system, and although I know that the question has been asked before, I'll ask it again: why not? If someone had started such a thing back in the early 1990s, would it (could it) have taken off? Or are there other factors that would have kept it from doing so (the same ones as now?)
Here's a useful (and rather brave) editorial from Chinese chemist Nai-Xing Wang in Nature. He's pointing out that the government's funding agencies are taking a crude and harmful approach to who gets research support: publish, and publish according to their schemes, or flippin' well perish:
The biggest problem remains the obsession with journal impact factors. Generally speaking, articles in journals with high impact factors are judged to appeal most to readers, but not every paper published in a high-impact-factor journal is high quality, and papers published in lower-ranked journals are never worthless. Yet some administrators in China take a very crude approach: high-impact-factor publications mean excellent work.
Research proposals are judged according to the impact factor of a scientist's previous publications. (And referees are usually selected on these criteria too.) Worse, the salaries of my chemistry colleagues go up or down depending on a complex mathematical formula based on the impact factor of the journals in which we publish our work — which we must supply in a detailed list.
Now, this sort of thing has been going on around the scientific world for a while, and it's going to be hard to put a complete stop to it. But the Chinese system that's described is about the most blatant that I've come across. The effects are pernicious:
If a high impact factor is the only goal of chemistry research, then chemistry is no longer science. It is changed to a field of fame and game. There are other effects too. Administrators in almost every university and research institute like to evaluate researchers by their papers at the end of each year. As a result, chemists often choose easy research topics that can be written up inside a year. . .
You get what you subsidize; I don't think that law is ever broken. And if the Chinese government wants people to crank out lots of papers in what they feel are high-end journals, well, that's what people will do. But if they want something useful to come out of all that effort, well, things might need to be adjusted a bit. But "useful" is a slippery word. For the people who are gainfully employed in keeping the current system running, it's just about as useful as it can be the way it is.
The Wall Street Journal has an interesting article based on data from Thomson Reuters on the frequency of retracted papers. It seems to be increasing dramatically:
Since 2001, while the number of papers published in research journals has risen 44%, the number retracted has leapt more than 15-fold, data compiled for The Wall Street Journal by Thomson Reuters reveal.
Just 22 retraction notices appeared in 2001, but 139 in 2006 and 339 last year. Through seven months of this year, there have been 210, according to Thomson Reuters Web of Science, an index of 11,600 peer-reviewed journals world-wide.
They mention Retraction Watch, as well they should. But ten years ago, would there have been enough new material to keep that blog running? Pharmalot has some more from its founder about what might be going on. There are, of course, more journals than ever these days, and many of them are junk. But it's not the bottom-tier journals that are driving this trend, I'd think, since honestly, when does anyone ever retract a paper in one of them? Consistent with that view, this bar chart of PubMed retractions by journal is heavily weighted towards the big dogs. A lousy or nonreproducible paper in one of the top journals is more likely to be of enough interest to attract attention, but one in J. Whatever will just sit there.
No, when you look at this chart, it appears that retractions-per-papers-published have been climbing, so the answer must be some combination of more mistakes, more fraud, or better policing. Retractions due to fraud seem to be where most of the growth is, according to this study, so that takes us down to the latter two explanations.
Software has definitely made the lazier sorts of fraud easier to detect, automatically flagging copy-and-paste hack jobs. But those aren't the kinds of things that show up in the better journals, are they? We may be seeing a mix of greater incentive to commit fraud and a rise in skepticism among readers. There have been enough cases, enough highly-publicized retractions and scandals, that more people may be willing to wonder if some exciting new result is true at all.
That's not a bad thing. The rise in fraud is a bad thing, but a corresponding rise in scrutiny is the only thing that's going to cure it. There are always a few pathological types out there that kind of know that they're going to get caught and kind of don't care. Those we shall always have with us, and not much is going to discourage them. But as for the rest of the fraudsters, the thought that they have a better chance of being found out and punished should give them something to think about.
A perennial topic around here has been the state of scientific research in China (and other up-and-coming nations). There's no doubt that the number of scientific publications from China has been increasing (be sure to read the comments to that post; there's more to it than I made of it). But many of these papers, on closer inspection, are junk, and are published in junk journals of no impact whatsoever. Mind you, that's not an exclusively Chinese problem - Sturgeon's Law is hard to get away from, and there's a lot of mediocre (and worse than mediocre) stuff coming out of every country's scientific enterprise.
But what about patents? The last couple of years have seen many people predicting that China would soon be leading the world in patent applications as well, which can be the occasion for pride or hand-wringing, depending on your own orientation. But there's a third response: derision. And that's what Anil Gupta and Haiyan Wang provide in the Wall Street Journal. They think that most of these filings are junk:
But more than 95% of the Chinese applications were filed domestically with the State Intellectual Property Office—and the vast majority cover "innovations" that make only tiny changes on existing designs. A better measure is to look at innovations that are recognized outside China—at patent filings or grants to China-origin inventions by the world's leading patent offices, the U.S., the EU and Japan. On this score, China is way behind.
The most compelling evidence is the count of "triadic" patent filings or grants, where an application is filed with or patent granted by all three offices for the same innovation. According to the Organization for Economic Cooperation and Development, in 2008, the most recent year for which data are available, there were only 473 triadic patent filings from China versus 14,399 from the U.S., 14,525 from Europe, and 13,446 from Japan.
Starkly put, in 2010 China accounted for 20% of the world's population, 9% of the world's GDP, 12% of the world's R&D expenditure, but only 1% of the patent filings with or patents granted by any of the leading patent offices outside China. Further, half of the China-origin patents were granted to subsidiaries of foreign multinationals. . .
The authors are perfectly willing to admit that this probably will change with time. But time can make things worse, too: as this editorial in Science last year made clear, the funding of research in China has some real problems. The authors of that piece are professors at two large Chinese universities, and would presumably know what they're talking about. For the biggest grants, they say:
. . .the key is the application guidelines that are issued each year to specify research areas and projects. Their ostensible purpose is to outline “national needs.” But the guidelines are often so narrowly described that they leave little doubt that the “needs” are anything but national; instead, the intended recipients are obvious. Committees appointed by bureaucrats in the funding agencies determine these annual guidelines. For obvious reasons, the chairs of the committees often listen to and usually cooperate with the bureaucrats. “Expert opinions” simply reflect a mutual understanding between a very small group of bureaucrats and their favorite scientists. This top-down approach stifles innovation and makes clear to everyone that the connections with bureaucrats and a few powerful scientists are paramount. . .
Given time, this culture could be changed. Or it could just become more entrenched as the amounts of money become larger and larger and the stakes become higher. China could end up as the biggest scientific and technological powerhouse the world has ever seen - or it could end up never living up to its potential and wasting vast resources on cargo-cult theatrics. It's way too early to say. But if many of those Chinese patents are just being written because someone's figured out that the way to get money and prestige is to file patents - never mind if they're good for anything - then that's not a good sign.
I note via Retraction Watch that the Journal of Biological Chemistry has issued retraction notices for four papers published from the group of the late Maria Diverse-Pierluissi, at the Mt. Sinai School of Medicine. One of their readers looked over the papers (which had been cited a few times, without making any particular huge impact, it seems), and found that some of the figures (Western blots and so on) repeat, even though they're supposed to represent different things (e.g., Figure 3A and 3C here).
Mt. Sinai told the Retraction Watch people that an internal investigation turned up the evidence of misconduct, and that the matter has been referred to the NIH, which funded the work. What those duplicate figures make me wonder, though, is how long it'll be before we have a plagiarized-figure search tool, in the same way that we have plagiarized-text tools running? There's already something similar out there - TinEye - and I'm sure that much nicer systems are available for a fee. Have any scientific journals implemented something like this?
I couldn't resist mentioning this one: the Archives of Internal Medicine was set to publish a paper showing a benefit for transcendental meditation in heart attack and stroke. Word was already out in the press - in the UK, the Telegraph had already published a story, with a quote from one of the paper's lead authors (from, ahem, the Maharishi University of Management) that the effect seen was as great or greater than any pharmaceutical intervention.
I don't have a link up to that particular newspaper report; its URL is no longer valid. That's because twelve minutes before the paper was set to be published online, the journal pulled it. (Other sources still have their stories up). We still don't know quite what the problem was. Nature got this statement:
“It became apparent that there was additional data not included in the manuscript that was about to be published, and the editor of Archives thought that the information was significant enough that it needed to be included as part of the paper, and then re-analyzed and verified, so she made the last-minute decision not to publish it. . .It’s an unusual situation, but the bottom line is that our journal wants to make sure that the information we put out is as accurate as can be.”
I'm glad to hear it. Larry Husten at Forbeshas the data from the paper, and has a lot of questions. We'll see how things look when (and if) it ever appears. But for now, if you're looking for the latest anyone has ever pulled a paper before publication, we may well have the record.
I found this article in The American Scholar via Arts and Letters Daily, entitled "Flacking for Big Pharma". As you might have possibly guessed from the title, it's a broadside against the advertising practices of the drug industry, and particularly against its interactions with physicians and the medical journals.
And I'll say up front that the piece is not, in fact, completely wrong. It's probably not even mostly wrong. There really are big problems in these areas, such as too-aggressive promotion, minimization of side effects, too many payments to "key opinion leaders", too many studies that don't see the light of day, and so on. And these things really do lower the respect that people have for the drug industry - assuming, by this point, that there's much respect left. But overall, this article is sort of a summary version of Marcia Angell's book, for people who would like to hate the drug industry but find themselves pressed for time. And as such, it manages to get some important things wrong in the process of getting some things right.
For example, it makes much of subgroup analysis of clinical trials, but as a way for drug companies to pull the wool over readers' eyes. I wonder how much this really happens, though, since overzealous data mining of a trial that wasn't powered to generate such conclusion is (you'd think) a well-known pitfall by now. Perhaps not, though. But the example given in the article is BiDil:
BiDil proponents published studies that supported their claim of a racially mediated genetic anomaly that was addressed by BiDil, making it an ideal drug for blacks but not for whites.. . .
NitroMed won FDA approval of a new trial that included only 1,050 black subjects, with no white subjects to provide comparison data. Furthermore, BiDil was not tested alone, but only in concert with heart medications that are already known to work, such as diuretics, beta-blockers, and angiotensin-converting enzyme (or ACE) inhibitors. The published results of the trial were heralded as a success when subjects taking the drug combinations that included BiDil enjoyed 43 percent fewer heart-failure deaths.
. . .excluding whites was a medically illogical but financially strategic move because it eliminated the possibility that the drug would test well in whites, thereby robbing NitroMed of its already thin rationale for calling BiDil a black drug. The “black” label was crucial, because BiDil’s patent covering use in all ethnic groups expired in 2007, but the patent for blacks only allows NitroMed to profit from it until 2020. BiDil is a case study in research methodology “flaws” that mask strategies calculated to make a dodgy drug look good on paper, for profit.
Now, about those placebo-controlled trials. This article makes much of a British Medical Journal satire from 2003 on how to make a drug look good. But it's confused:
A placebo, such as a sham or “sugar” pill, has no active ingredient, and, although placebos may evoke some poorly understood medical benefits, called the “placebo effect,” they are weak: medications tend to outperform placebos. Placebo studies are not ethical when a treatment already exists for a disorder, because it means that some in the study go untreated. However, if you care only that your new drug shines in print, testing against placebo is the way to go.
Well, which is it? We can't, in fact, run placebo-controlled trials just to "shine in print" when there's a standard of care, you know. You can only do that when there's no standard of care at all. And in those cases, what exactly should we use as a comparison? Using nothing at all (no pills, nothing) would, in fact, make our drugs look even better than they are, because of that placebo effect. This is a specious objection.
And when there's a standard of care that a new drug will be added to (as was the case with BiDil), then you actually do have to run it with those therapies in place, at least when you get to Phase III. The FDA (and the medical community) want to know how your drug is going to perform in the real world, and if patients out in that real world are taking other medications, well, you can't pretend that they aren't.
In another section, the article makes much of the Merck/Elsevier affair, where Elsevier's "Excerpta Medica" division set up some not-really-journals in Australia (blogged about here). That was, in fact, disgraceful (as I said at the time), but disgraceful apparently isn't enough:
. . .Elsevier, the Dutch publisher of both The Lancet and Gray’s Anatomy, sullied its pristine reputation by publishing an entire sham medical journal devoted solely to promoting Merck products. Elsevier publishes 2,000 scientific journals and 20,000 book-length works, but its Australasian Journal of Bone and Joint Medicine, which looks just like a medical journal, and was described as such, was not a peer-reviewed medical journal but rather a collection of reprinted articles that Merck paid Elsevier to publish. At least some of the articles were ghostwritten, and all lavished unalloyed praise on Merck drugs, such as its troubled painkiller Vioxx. There was no disclosure of Merck’s sponsorship. Librarian and analyst Jonathan Rochkind found five similar mock journals, also paid for by Merck and touted as genuine. The ersatz journals are still being printed and circulated, according to Rochkind, and 50 more Elsevier journals appear to be Big Pharma advertisements passed off as medical publications. Rochkind’s forensic librarianship has exposed the all-but-inaccessible queen of medical publishing as a high-priced call girl.
Fifty journals? Really? As far as I can tell, that figure comes from this analysis at the time, and seems to be mostly nonce publications, one-off conference proceedings, and the like. There is a whole list of "Australasian Journal of So-and-Sos", which would be the same reprint advertorials as the other Excerpta Medica stuff, but do these still exist? (Did all of them on the list, in fact, ever actually publish anything?)
You'd get the impression that Elsevier is (or was, until Big Pharma came along) an absolute shining pinnacle of the medical establishment - but, with apologies to the people I know who work there, that is unfortunately not the case. They're big, and they're very far from the worst scientific publishers out there, but some of their titles are, in fact, not adding much to the total of human knowledge. Nor has the conduct of their marketing department always been above reproach. But no, this has to be built up to look even worse than it is.
The irritating thing is that there's plenty to criticize about this industry without misrepresenting reality. But does that sell?
Well, one day after writing an obit for the XMRV story comes this abstract from Retrovirology. The authors, from Cornell and SUNY-Buffalo, say that they've detected other murine retrovirus transcripts from CFS patients (but not in most controls), and that these are more similar to those reported in last year's Lo and Alter paper in PNAS than they are to XMRV itself.
So perhaps the story continues, and what a mess it is at this point. I continue to think that the XMRV hypothesis itself is in serious trouble, but murine retroviruses as a class are still worth following up on. This is tough work, though, because of the twin problems of detection and contamination, and it's going to be easy for people to fool themselves.
Meanwhile, Retraction Watch has more on Science's "Expression of Concern" that I wrote about yesterday. It appears that the journal asked the authors to retract the paper (so says the Wall Street Journal, anyway) but that co-author Judy Mikovits turned them down (as might have been expected from her previous stands in this area). Science released their editorial note early because of the WSJ piece.
I meant to blog on this late last week, but (in case you haven't seen it) the whole putative link between XMRV and chronic fatigue syndrome seems now to be falling apart. If you want to see the whole saga via my blog posts and the links in them, then here you go: October 2009 - January 2010 - February 2010 - July 2010 - January 2011. At that last check-in, the whole thing was looking more like an artifact.
And now Science is out with a paper that strongly suggests that the entire XMRV virus is an artifact. It looks like something that's produced by the combination of two proviruses during passaging of the cells where it was detected, and the paper suggests that other human-positive samples are the result of contamination. Another paper is (again) unable to replicate detection of XMRV in dozens of samples which had previously been reported as positive, and finds some low levels of murine virus sequences in commercial reagents, which also fits with the contamination hypothesis.
With these results in print, Science has attached an "Editorial Expression of Concern" to the original 2009 XMRV/CFS paper, which touched off this whole controversy. My take: while there are still some studies ongoing, at this point it's going to take a really miraculous result to bring this hypothesis back to life. It certainly looks dead from here.
There will be also be some people who ask whether Science did the world a favor by publishing the original paper in the first place. But on balance, I'd rather have things like this get published than not, although in hindsight it's always easy to say that more experiments should have been done. The same applies to the arsenic-bacteria paper, another one of Science's recent bombshells. I'm not believing that one, either, at this point - not until I see a lot more supporting data - but in the end, I'm not sad that it was published, either. I think we're better off erring a bit on the wild-ideas end of the scale than clamping down too hard. That said, you do have to wonder if Science in particular is pushing things a bit too hard, itself. While I think that these ideas deserve a hearing, it doesn't necessarily have to be there.
Your genome - destiny, right? That's what some of us thought - every disease was going to have one or more associated genes, those genes would code for new drug targets, and we'd all have a great time picking them off one by one. It didn't work out that way, of course, but there are still all these papers out there in the literature, linking Gene A with the chances of getting Disease B. So how much are those worth?
While we're at it, everyone also wanted (and still wants) biomarkers of all kinds. Not just genes, but protein and metabolite levels in the blood or other tissue to predict disease risk or progression. I can't begin to estimate how much work has been going into biomarker research in this business - a good biomarker can clarify your clinical trial design, regulatory picture, and eventual marketing enormously - if you can find one. Plenty of them have been reported in the literature. How much are those worth, too?
Not a whole heck of a lot, honestly, according to a new paper in JAMA by John Ioannidis and Orestes Panagiotou. They looked at the disease marker highlights from the last 20 years or so, the 35 papers that had been cited at least 400 times. How good do the biomarkers in those papers have to be to be useful? An increase of 35% in the chance of getting the targeted condition? Sorry - only one-fifth of the them rise to that level, when you go back and see how they've held up in the real world.
Subsequent studies, in fact, very rarely show anything as strong as the original results - 29 of the 35 biomarkers show a less robust association after meta-analysis of all the follow-up reports, as compared to what was claimed at first. And those later studies tend to be larger and more powered - in only 3 cases was the highly cited study the largest one that had been run, and only twice did the largest study show a higher effect measure than the original highly cited one. Only 15 of the 35 biomarkers were nominally statistically significant in the largest studies of them.
Ioannidis has been hitting the literature's unreliability for some time now, and I think that it's hard to dispute his points. The first thought that any scientist should have when an interesting result is reported is "Great! Wonder if it's true?" There are a lot of reasons for things not to be (see that earlier post for a discussion of them), and we need to be aware of how often they operate.
Yesterday's look into the Google Ngram data set brought up a discussion in the comments on how good the numbers are in it (and in other large datasets). "Garbage in, garbage out" is as true a statement as ever, so it's a real worry. (Even if the data were perfect, the numbers could still be misused and misinterpreted, of course).
An e-mail from a reader pointed me to another example of this sort of thing. The NIH Chemical Genomics Center (NCGC) has a collection of known pharmaceutically active compounds for use in screening and target ID. This is a good idea, and the same sort of thing is done internally in the drug industry. But the ChemConnector blog has some questions about how robust the dataset is. The rough estimate is that between 5 and 10% of the 7600+ structures are messed up in some way (stereochemistry, salt form, the dreaded pentavalent carbon, and so on).
Read the comments there for some interesting back-and-forthing with the NIH people. The NCGC folks realize that they have some problems, and are willing to put in the work to help clean things up. The problem is, they'd already published on this list, calling it "definitive, complete, and nonredundant", which now seems to be a bit premature. . .
A comment to the last post mentioned that if you search the word "biotechnology" in Google's Ngram search engine, something odd happens. There's the expected rise in the 1970s and 80s, but there's also a bump in the early 1900s, for no apparent reason. Curious about this, I ran several other high-tech phrases through and found the exact same effect.
Here's a good example, with some modern physics phrases. And you get the same thing if you search "nanotechnology", "ribosome", "atomic force microscope", "RNA interference", "laser", "gene transfer", "mass spectrometer" or "nuclear magnetic resonance". There's always a jump back in exactly the same period on the early 1900s.
So what's going on? I can understand some OCR errors, but why do these things show up in this specific Edwardian-age window? Can anyone at Google shed any light on this?
I was thinking the other day that I never remembered hearing the phrase "Big Pharma" when I first got a job in this business (1989). Now I have some empirical proof, thanks to the Google Labs Ngram Viewer, that the phrase has only come into prominence more recently. (Fair warning: you can waste substantial amounts of time messing with this site). Here's the incidence rate of "big pharma" in English-language books from 1988 to 2000.
It comes from nowhere, blips to life in 1992, doesn't even really get off the baseline until 1994 or so, and then takes off. (The drops in 2005 and 2008 remain unexplained - did the log phase of its growth end in 2004?)
Update: that graph holds for the uncapitalized version of the phrase. If you put the words in caps, you get the even more dramatic takeoff shown below:
To be fair, though, there seems to have been a general rise in Big Pharma-related literature during that period. Try out this graph, comparing mentions of Merck, Pfizer, and Novartis since 1970. The last-named, of course, didn't even exist until the early 1990s, but they (like the others) have spent the time since then zipping right up, with no apparent end in sight. (Merck, especially - what's with those guys?) And what accounts for this? Business books? Investing guides? Speculation is welcome.
Note: the above paragraph was written before realizing that the Google Ngram search is case-sensitive - so, as was pointed out in the comments, I was picking up on people not caring about capitalization more than anything else. Below is the correct graph, with initial capitals in the search, and it makes more sense. Merck still is the king of book mentions, though, for all the coverage that Pfizer gets.
I'll finish off with this one, using a longer time scale. Yes, folks, for better or worse, it appears that the phrase "organic chemistry" peaked out between book covers around 1950, and has been declining ever since. Meanwhile, "total synthesis" starting rising during the World War II era (penicillin?), and kept on moving up until a peak around 1980. Interestingly, things turned around in 2000 or so, and especially since 2003. And this can't be ascribed to some sort of general surge in chemistry publications - look at the "organic chemistry" line during the same period. Is there some other field that's adopted the phrase?
Chemists who don't (or don't yet) work in drug discovery often wonder just what sort of chemistry we do over here. There are a lot of jokes about methyl-ethyl-butyl-futile, which have a bit of an edge to them for people just coming out of a big-deal total synthesis group in academia. They wonder if they're really setting themselves up for a yawn-inducing lab career of Suzuki couplings and amide formation, gradually becoming leery of anything that takes more than three steps to make.
Well, now there's some hard data on that topic. The authors took the combined publication output from their company, Pfizer, and GSK, as published in the Journal of Medicinal Chemistry, Bioorganic Med Chem Letters and Bioorganic and Medicinal Chemistry, starting in 2008. And they analyzed this set for what kinds of reactions were used, how long the synthetic routes were, and what kinds of compounds were produced. Their motivation?
. . .discussions with other chemists have revealed that many of our drug discovery colleagues outside the synthetic community perceive our syntheses to consist of typically six steps, predominantly composed of amine deprotections to facilitate amide formation reactions and Suzuki couplings to produce biaryl derivatives. These “typical” syntheses invariably result in large, flat, achiral derivatives destined for screening cascades. We believed these statements to be misconceptions, or at the very least exaggerations, but noted there was little if any hard evidence in the literature to support our case.
Six steps? You must really want those compounds, eh? At any rate, their data set ended up with about 7300 reactions and about 3600 compounds. And some clear trends showed up. For example, nearly half the reactions involved forming carbon-heteroatom bonds, with half of those (22% of the total) being acylations. mostly amide formation. But only about one tenth of the reactions were C-C bond-forming steps (40% of those were Suzuki-style couplings and 18% were Sonogoshira reactions). One-fifth were protecting group manipulations (almost entirely on COOH and amine groups), and eight per cent were heterocycle formation, and everything else was well down into the single digits.
There are some interesting trends in those other reactions, though. Reduction reactions are much more common than oxidations - the frequency of nitro-to-amine reductions is one factor behind that, followed by other groups down to amines (few of these are typically run in the other direction). Among those oxidations, alcohol-to-aldehyde is the favorite. Outside of changes in reduction state, alcohol-to-halide is the single most favorite functional group transformation, followed by acid to acid chloride, both of which make sense from their reactivity in later steps.
Overall, the single biggest reaction is. . .N-acylation to an amide. So that part of the stereotype is true. At the bottom of the list, with only one reaction apiece, were N-alkylation of an aniline, benzylic/allylic oxidation, and alkene oxidation. Sulfonation, nitration, and the Heck reaction were just barely represented as well.
Analyzing the compounds instead of the reactions, they found that 99% of the compounds contained at least one aromatic ring (with almost 40% showing an aryl-aryl linkage) and over half have an amide, which totals aren't going to do much to dispel the stereotypes, either. The most popular heteroaromatic ring is pyridine, followed by pyrimidine and then the most popular of the five-membered ones, pyrazole. 43% have an aliphatic amine, which I can well believe (in fact, I'm surprised that it's not even higher). Most of those are tertiary amines, and the most-represented of those are pyrrolidines, followed closely by piperazines.
In other functionality, about a third of the compounds have at least one fluorine atom in them, and 30% have an aryl chloride. In contrast to the amides, there are only about 10% of the compounds with sulfonamides. 35% have an aryl ether (mostly methoxy), 10% have an aliphatic alcohol (versus only 5% with a phenol). The least-represented functional groups (of the ones that show up at all!) are carbonate, sulfoxide, alkyl chloride, and aryl nitro, followed by amidines and thiols. There's not a single alkyl bromide or aliphatic nitro in the bunch.
The last part of the paper looks at synthetic complexity. About 3000 of the compounds were part of traceable synthetic schemes, and most of these were 3 and 4 steps long. (The distribution has a pretty long tail, though, going out past 10 steps). Molecular weights tend to peak at between 350 and 550, and clogP peaks at around 3.5 to 5. These all sound pretty plausible to me.
Now that we've got a reasonable med-chem snapshot, though, what does it tell us? I'm going to use a whole different post to go into that, but I think that my take-away was that, for the most part, we have a pretty accurate mental picture of the sorts of compounds we make. But is that a good picture, or not?
Looking over the list, I think that there are some artifacts in it, and boy, don't metal-organic frameworks and nanotech just rate like crazy? But it's an interesting starting point for discussion, especially when you note how (relatively) few organic and synthetic organic chemists make the upper reaches. Thoughts?
I don't know if this DOI link resolves yet - or if the problem will be fixed by the time it does. But for now, in the "Articles in Press" queue over at Drug Discovery Today, they have one whose title reads like this:
Utility of protein structures in overcoming ADMET-related issues of drug-like compounds[1. AU: You use ADMET-relevant throughout manuscript. Would you like to change this to ADMET-relevant too?]
Well, would you? They're waiting for someone to answer them, apparently.
I've been browsing through my journal RSS feeds, and a question occurs to me. When you're scanning through the current literature, what sort of paper makes you most likely to keep scrolling? What kind of work are you least likely to actually read?
We'll stipulate that you're looking at a journal or subject that's relevant to your field - I read a lot of stuff, but I'm most certainly not going to slow down to look over (say) a theoretical paper calculating the stability of isomeric inorganic complexes. But that said, there are things that make you slow down while going through the abstracts, and there are things that absolutely made you speed up.
My particular biases are to walk more quickly past the following sorts of titles, which have been only slightly exaggerated for effect. And you?
"Synthesis of A Natural Product That You Don't Care About, Using Methods That Bore You"
"Slight Enantiomeric Excess Realized Through Use Of A Humungous Catalyst That Takes Nine Steps To Make All By Itself"
"Nanorods Attached to Nanoplates by Nanosprings: Progress Toward a Nanomattress"
"Green Chemistry, Part 87: A Novel Reagent to Prepare Nitriles from Oximes"
"Isolation Of Known Terpenoid Natural Products From Weeds In Our Back Yard"
Here's the cry of someone who's been jerked around by too many journal referee reports. Hidde Ploegh of the Whitehead Institute has a piece in Nature News called "End the Wasteful Tyranny of Reviewer Experiments". That could have just possibly have been phrased more diplomatically, but I know what he's talking about.
Too often, reviewers try to show that they're fulfilling their responsibilities by requesting additional work from the authors of a paper under consideration. This happens more and more as you move up the hierarchy of journals, as both the novelty of the work and the incentive to publish it increase. No one's going to exert themselves too much to get their paper into Acta Retracta, even if some rogue reviewer were to try it, but Science and Nature (among others) can really make you perform tricks.
What this reminds me of is a story about Steve Wozniak, of Apple fame. When he was in college, his dorm had an old TV down in the lobby with a rabbit-ear antenna, which had to be messed with constantly to get a good picture. Woz apparently built a gizmo to fuzz out the reception, and used to sit inconspicuously in the back of the room, trying to see what sort of crazy positions he could twist people into as they held the antenna in what was seemingly the One Perfect Spot.
The referee equivalent is Just One More Experiment, and it's not always justified:
Submit a biomedical-research paper to Nature or other high-profile journals, and a common recommendation often comes back from referees: perform additional experiments. Although such extra work can provide important support for the results being presented, all too frequently it represents instead an entirely new phase of the project, or does not extend the reach of what is reported. It is often expensive and unnecessary, and slows the pace of research to a crawl. Among scientists in my field, there is growing concern that escalating demands by reviewers for the top journals, combined with the increasingly managerial role assigned to editors, now represents a serious flaw in the process of peer review.
Ploegh's point is that too many referees aren't reviewing the paper that they have; they're suggesting a whole new project or phase of research. And some of these wouldn't even affect the results and conclusions of the paper under review very much - they're just "Gosh, wouldn't it be nice if you would also. . ." experiments. The benefit for science, he says, is nowhere near commensurate with the disadvantage of holding up publication, messing with the career prospects of younger investigators, spending extra time and grant money, and so on. His suggestion?
The scientific community should rethink how manuscripts are reviewed. Referees should be instructed to assess the work in front of them, not what they think should be the next phase of the project. They should provide unimpeachable arguments that, where appropriate, demonstrate the study's lack of novelty or probable impact, or that lay bare flawed logic or unwarranted conclusions.
He also suggests that reviewers provide an estimate of the time and cost involved for their suggested experiments, and compare that to their purported benefits. I wouldn't mind seeing editors crack down on this some, either. I've had useful feedback with my own manuscripts, which had identified things that really did need to be shored up. But submitting a paper should not routinely be an exercise in having other people tell you what experiments you should run before you can publish your. When there really is a gap or flaw, naturally, it's appropriate to ask for more, but I agree with Ploegh that a reviewer needs to make a case for such things, rather than just asking for them as a matter of routine.
Ploegh has a larger historical point to make as well. Looking back at the earlier days of , say, molecular biology, you get the impression that if someone sent in an interesting paper that seemed reasonable, it would just get published, without all these trips back to the bench. Somehow, the mechanics of science (and especially scientific publication) have changed. Has it been for the better? Or would we all be better off letting more things through as they stand, if they're clearly presented and logically consistent?
I wonder if journals might consider publishing in this style, while then adding an editorial note about what further experiments had been suggested by reviewers. This would fulfill the function of pointing out potential weak points or areas for further exploration, but without delaying things so much. I don't see this happening - but why not, exactly?
And now a brief note from the "trivial but annoying" department, since it's been a couple of years since I last complained about this. Is there any way that we can start a petition, or take up a collection, or do something to make K. C. Nicolaou stop drawing ring systems like this? Coloring the insides of them with gradient fills adds no information and actually obscures elements of the structure.
If all else fails, can we at least send the man a set of Ed Tufte books?
Remember that weird Tetrahedron paper from last December? The one that claimed that it isolated the reverse transcriptase inhibitor nevirapine as a natural product from an Indian plant? In chiral form, no less?
Well, the journal would now like to say "Never mind". The lead author has retracted the paper, "due to doubt created in the scientific community on the origin of nevirapine from the seeds of Cleome viscosa". That's an odd way to put it. Isn't it? Doubts that other people might have are irrelevant if you're right, aren't they?
No, this is similar to the classic weasel-word apology, the one that goes on about regretting the way that some people took offense rather than regretting the original action itself. The reason this paper was retracted, surely, was that those doubts in the scientific community were well-founded. This paper made no sense on several levels, and those problems should have been caught immediately. Its publication was an embarrassment for Tetrahedron and for Elsevier.
I think that the reason I get so worked up about these things is the laziness and sloppy thinking involved. Scientific research deserves more than that, and the rest of us deserve more from the people who publish it.
Well, this post needs updating. In it I mentioned never running a Prins reaction again since the 1980s, nor any photochemistry, and today what do I find myself doing? Both of them, although not at the same time.
I am, fortunately, not running the Prins this way. But even bringing it up at all recalls to me a key part of my education. When I first joined my graduate school research group, I was put to making some tetrahydropyran systems. I was handed a synthesis, drawn up before my arrival, of how to make the first one, and like most first-year grad students, I gamely dug and and started to work on it.
I should have devoted a bit more thought to it. I won't go into the details, but it was a steppy route that relied, in the final ring-closure step, on getting the cyclic ether to form where one of the partners was a neopentyl center. The organic chemists in the audience will immediately be able to guess just how well that went.
So I beat on it and whacked at it, getting nowhere as I used up my starting material, until I was finally driven to the library. In the spring of 1984, that was a different exercise than it is now, involving the 5-year Chemical Abstracts indices and an awful lot of page flipping. (I haven't so much as touched a bound volume of CA in I don't know how many years now). If you were a nomenclature whiz, you could try looking up your compound, or something like it, in the name index, but a higher-percentage move was often to look up the empirical formula. That gave you a better shot, because (if it was there at all) you could see how CA named your system and work from there.
To my great surprise, the second set of collective indices I checked (the good ol' 9th), yielded a direct hit on an empirical formula, and the name looked like exactly what I had been trying to make. The reference was in Tetrahedron, which we most certainly had on the shelf, and I zipped over to see if there was any detail on how to make the stuff.
There was indeed. A one-stepper Prins cyclization gave just the ring system I'd been trying to make, and that was one step from the intermediate I needed. I just stared at the page, though. I honestly couldn't believe that this was real (as I mentioned, I was in about my second month of grad school lab work). Surely the synthesis I'd been given was the way to make this stuff? Surely the people responsible for it had checked the literature before drawing it up? (After all, it had only taken my a few minutes to find the stuff myself). Surely I couldn't just make the ring in one afternoon using two starting materials I could buy cheaply from Aldrich?
Well, surely I could. And that's just what I did, and got my project moving along until the next interesting difficulty came up a couple of months later. But I still recall standing there in the Duke chemistry library, looking at that journal article "with a wild surmise" that perhaps I should check things out for myself next time instead of just taking everyone else's word. It took a couple more lessons for me to really grasp that principle (Nullius in verba!, but it's helped me out a great deal over the years. I have the 27-year-old photocopy I made that afternoon in front of me now. It's a good reminder.
A reader who's attending the International Congress on Heterocyclic Chemistry in Glasgow later this year sent me a note about it. Like many such meetings, they have guidelines for presentation and poster abstracts. But this one was done by someone who's been around the block a few times.
The sample abstract is from a team from the University of Utopia (and in case you're wondering, ".ut" is apparently the internet domain for Utopia). And the authors, in a nice touch, are Black, Schwartz, Nero, Fekete, and Čzerný. (Too bad the other students in the group - Siyah, Dubh, and Musta - couldn't make it onto the list). But here's the text of the thing itself:
Two fundamentally different but complementary transition metal catalyzed chemo-, regio-,diastereo-, enantio-, and grantproposalo-selective approaches to the synthesis of a library of biologically significant nano- and pico-molecules will be presented with the focus on reaction mechanism and egocentric effects. The role of the nature of the metal, ligand, solvent, temperature, time, microwave, nanowave, picowave, ultrasound, hypersound, moon phase, and weather in this catalytic, sustainable, cost-effective, and eco-friendly technology will be discussed in detail.
If nothing else, that's about as grantproposalo-selective as it gets, right there. . .
I wish that more journals did this! Environmental Microbiology, which I have never looked at before, has published its favorite reviewer comments from the year just passed. They're not tied to the papers that generated them, naturally, but then, many of these manuscripts didn't quite make the cut:
"The biggest problem with this manuscript, which has nearly sucked the will to live out of me, is its terrible writing style."
"I usually try to be nice, but this paper has got to be one of the worst I have read in a long time."
"I suppose that I should be happy that I don't have to spend a lot of time reviewing this dreadful paper, however, I am depressed that people are performing such bad science."
"It is sad to see so much enthusiasm and effort go into analyzing a dataset that is just not big enough."
There are plenty more, including many from people who are actually happy about what they had to read (and yes, there are some). Check 'em out!
Ben Goldacre has an excellent point here at Bad Science: when a paper gets retracted from a journal, shouldn't everyone know why it's been retracted?
He highlights the experience of the blog Retraction Watch (which I hadn't heard of until now), when they tried to find out why a paper had been pulled from the Annals of Thoracic Surgery. The journal's editor responded to their query by informing them that "it's none of your damn business".
Gotta disagree there, chief. I think that this is actually important information, and that it should be disclosed as much as possible. There are all sorts of reasons for papers to be retracted, ranging from benign to evil, and it's in the interest of readers to know what category things have fallen into. I understand that in some cases papers are the subject of ongoing investigations, so these details aren't always available, but in that case, why not say something like: "The data in Table II have not been reliably reproduced by other workers. While some of the co-authors of the original work have stated that they stand by the results as published, an investigation has begun into the methods and data of this paper, and the lead authors have asked that it be retracted until this matter is concluded".
But that's not the sort of thing we get. Goldacre cites another example from Retraction Watch, concerning this paper from JACS. When the bloggers contacted the lead author, he gave them more details than you could get from the journal about what was wrong with the paper. So why doesn't JACS tell us these things?
Thanks to the Retraction Watch people for taking the time and effort to do this sort of thing. I just wish that it weren't necessary for anyone to do it at all.
Angewandte Chemie recently ran a behind-the-scenes article about their journal, with several interesting bits of information. For one thing, they've gotten a lot more selective over the years, as the number of submissions has gone up. They publish many more papers, total, than they used to, but reject a much higher fraction at the same time. (I've added to that total myself a couple of times!).
Mind you, there are times when that rejection rate should have been even a bit higher, but as you might guess, the article doesn't bring up those awkward moments. There's no insight into the vile puns and other pop-culture references that continue to infest their abstracts, either. Can't have everything.
But I found this chart interesting. These are the download statistics for a particular (unspecified) communication in the journal over time. (Note that they've scrubbed the units on the Y-axis, the wimps).
This confirms what most scientists have figured, that your paper has a brief window to be noticed, and then back in the pile it goes. Back to the background rate, with people coming across it in literature searches once in a while.
Since we've been talking about peer review on and off around here, this paper in PLoS One is timely. The authors are putting some numbers on a problem that journal editors have long had to deal with: widely varying reviews from different referees for the same exact paper.
It's a meta-analysis of 52 studies of the problem reported over the last few decades. It confirms that yes, inter-reviewer reliability is low. The studies that report otherwise turn out to have smaller sample sizes and other signs of lower reliability. The question now is: to what extent is this a problem?
One of the studies they quote maintains that too high a level of agreement would also be the sign of a problem (that some of the reviewers are redundant, and that the pool of referees might have been poorly chosen). I'm willing to think that total agreement is probably not a good thing, and that total disagreement is also trouble. So what level of gentlemanly disagreement is optimal? And are most journals above it or below?
FIguring that out won't be easy. Some journals would really have to open their books for a detailed look at all the comments that come in. I assume that there are editors who look over their reviewers, looking for those that tend to be outliers in the process. (Um, there are some editors that do this, right?) But that takes us back to the same question - do you value those people for the perspective they provide, or do you wonder if they're just flakes? Without a close reading of what everyone had to say about the crop of submissions, it's hard to say. Actually, it might not be easy, even then. . .
So, should I apologize for the tone of yesterday's post? I'm not ready to yet, and I'll explain why.
Here's a general rule: if you find yourself having to make excuses for a scientific paper, for the key experiments that weren't done and the rationales that don't appear, then something is wrong. A paper shouldn't make you have to assume that the authors will get around to running Experiment X and Comparison Y and Test Z, and put that in their next manuscript. (If this makes you think of the recent arsenic bacteria controversy, you're right on target). Anything like that which immediately comes to mind should be in the first manuscript. If the authors haven't put it in there, then it's the job of the referees to tell them to go back and do it. And whoever refereed this paper did an incompetent job.
Now, a quick technical note. It's true that medium-sized rings can form isolable atropisomers in some cases. (An atropisomer, for those who aren't chemists - or chirality geeks - is a compound that can exist as two mirror-image forms just through some sort of constrained movement of its parts. A putatively flat compound that comes around and wraps over itself like a screw thread is one example, as is one with a bond that should be freely rotating but it blocked by large side chains from doing so).
Problem is, this compound sure doesn't look like one of those cases. It has no bulky groups that have trouble getting out of each other's way, and it has no helical chirality. The only thing it has, to my eyes, is a nitrogen that could only lead to new isomers if its barrier to inversion were really, really high - and it shouldn't be. For nevirapine, there really should be no way to isolate such an isomer at room temperature. If anyone can provide evidence for isolable atropisomers in a system as small and lightly substituted as this one, I'll certainly consider eating my words - but not until then.
But that brings up a larger point. This is actually one of the things that makes me think this paper is bogus: the presence of such a compound really would be the big selling point of the manuscript, if the authors had thought about it. Instead, their main focus is on how the structure turns out to be a natural product (which I have trouble believing, too),and not on the bizarre nature of it being chiral. The chirality, frankly, seems to be an afterthought, the way things are written. The word "atropisomer" does not make an appearance. References to the other dibenzo ring systems that have shown this interesting phenomenon (which were worth papers all their own) are not cited. There is no mention of a nitrogen inversion. (If that were the explanation, you'd also expect that heating up the sample would eventually start flipping the molecules past that barrier and removing the optical activity - but there's no mention of any such thing). You'd never know that there was something interesting going on, because the authors give us no reason to believe that they knew that, either.
It would also be quite interesting, if you could have such a thing as an optical isomer of nevirapine, to see what its activity would be on its enzyme target, reverse transcriptase. And if nevirapine could exist as enantiomers, how about running the synthetic material down some chiral columns to see if you could resolve it? Then show that your new optically active stuff is only one of those peaks; that would be pretty convincing. Not done, not done, not even mentioned, etc.
Here's another consideration: as mentioned yesterday, I don't think that this structure has been given very good characterization (the X-ray data seem insufficient to talk about chirality). One of the comments to yesterday's post wondered why the authors didn't show HPLC traces with and without a spike of the authentic drug material. That's an excellent idea, and it's something that would be worth showing in an NMR spectrum, too. Claiming that you found nevirapine in a plant is quite weird - you'd want to really hammer down the fact that everything is identical. But these experiments haven't been done, either.
Allow me to mention one more oddity. The authors actually make reference to "optically active nevirapine" (their footnote #10), but the reference they cite (the original paper from Boehringer Ingleheim) does not, as far as I can see, mention any such thing. And that's because no one has ever mentioned any such thing, and that's because I don't think it exists, outside of (just maybe) a low-temperature NMR experiment.
No, I'm still not buying this. I'm upset with the authors for having proposed such a thing with such thin evidence, but I'm really more upset with the editors of what is supposed to be a reputable journal for publishing it.
Thanks to an email from a reader, I can bring you this very weird paper from Tetrahedron. The authors claim to have extracted a local plant and isolated nevirapine, (sold as Viramune by Boehringer Ingleheim as a reverse transcriptase inhibitor for HIV).
That's kind of odd. I'm no natural products expert, but I've sure seen a lot of them over the years, and that framework (and the N-cyclopropyl) don't look so likely to me. But hey, plants do odd things. That's not what's really puzzling about this paper. No, what's had me staring at it this morning is the claim that, in contrast to the marketed drug, this stuff is optically active nevirapine.
Say what? Try as I might, I can't see any plausible way that that's a chiral compound. The authors seem to think it is, though. They claim optical rotation, somehow, and then say that "The detailed structure and stereochemistry of compound 1 was established unambiguously by single crystal X-ray crystallography." But hold on - that's not as easy as it sounds. Getting absolute configurations from the X-ray data of light-atom-only molecules takes special efforts, and I don't see any being taken (molybdenum X-rays, direct methods, no talk of anomalous dispersion, etc.)
I'm just not willing to see that nitrogen atom as a source of chirality - if it were, shouldn't that be the focus of this whole paper? Instead, the authors just blithely tell us how neat it is that they've isolated the chiral material. In fact, they find it so neat that they tell us two times in a row:
This is a very interesting discovery that naturally occurring optically active nevirapine has been biosynthesized in the seeds of C.viscosa and the optically inactive nevirapine was designed as a selective non-nucleoside inhibitor of HIV-1 reverse transcriptase. It is also a remarkable finding that the seed of C.viscosa is the source of optically active nevirapine, which was also designed and synthesized before its isolation from natural source.
This sounds like some sort of lunatic patent-busting exercise, to be honest. And it sounds as if someone doesn't know what a chiral compound is. And that whoever reviewed this for Tetrahedron was incompetent. And that the editor who let it through should be a least a little bit ashamed. Well?
Looking over the chemical literature with an RSS reader can really give you a sense of what the hot topics are, and what's cooling off. Remember when it seemed as if every third paper was about ionic liquids? You still see work in the area, but it's nowhere near as crazy as it was. (I had a colleague come by my office the other day and ask "Did anyone ever find out what to do with those things?") Similarly, gold catalysts have been all over the place in recent years, but seem, to my eye, to be calming down.
Some of these things are research areas that look promising, but die off when their limits become apparent. Some of them are almost sheer fads, with papers coming out from all sorts of odd places because the authors want to get in on the hot, publishable topics while they can. Others keep going because the topics themselves are important but ver hard to exhaust (metal-catalyzed couplings come to mind).
And there are areas that keep going in the literature because they look like they should be important and useful, and eventually will, but no one can quite get them to either work generally enough or get people to recognize that they do. The metal-catalyzed coupling literature was in this shape back in the 1970s and into the 1980s - there were a lot of disparate reactions that you could do with palladium, but none of them had exactly taken over the world. My vote for a current field in this protostar state is engineered solid-phase catalysis.
That may sound odd, since work on solid-phase catalysts has been going on for decades, and is of huge industrial importance. But many of the important catalysts have been arrived at either by luck or by an awful lot of hard slogging. The field is complicated enough - fiendishly so - that it's hard to draw general conclusions. If you have a good solution-phase catalyst, how do you make a solid-supported variety that works just as efficiently? Well. . .if you really want one, you make about a zillion variants and hope for the best, as far as I can see.
Part of the problem (as with the metal-catalyzed coupling world) is that there are just so many variables. The solid supports alone are enough to keep a person occupied for life, what with all the various aluminas, silicas, zeolites, polymers, mesoporous engineered thingies, and so on. Then you have the uncountable schemes for linking these surfaces to active catalysts - what functional groups to use, what density things should be on the surface, what distance you need between the surface and the catalyst, etc. And just linking up to the known catalysts is no light work, either, since most of these things were not made with convenient handles hanging off them.
As we get better at making (and characterizing) new kinds of surfaces and new kinds of macromolecular assemblies, we might start to get our hands around this subject. For now, though, it seems to be mostly in the descriptive stage: papers are of the "Hey, we made this thing and here's what it does" variety, with further work in the series being "Hey, remember that stuff we made? Turns out you can do this with it, too - who knew?" What you don't see, or not too darn often, is a paper describing the general principles of these processes. For the most part, we don't know them yet.
But if I had to pick an area that will eventually blossom into a host of applications, this would be high on the list. It's a mixture of surface chemistry, materials science, nanotechnology, and organic synthesis, and it's got a lot of promise. But then again, it's had a lot of promise for a long time now. . .
I have a larger comment, sparked by the controversy over the NASA-arsenic-bacteria paper in Science. But it's not just about that one. It's about the "reactome" paper (also in Science and now retracted), the hexacyclinol synthesis published in Ang. Chem., and others. There have been, I think it's fair to say, a number of very arguable papers published in very high-profile journals in recent years. What's going on?
I want to make it clear that I'm not upset about journals published "out-there" work. In fact, I wish that there were a bit more of it. But at the same time, if you're going to go out there on the edge, you'd better have some solid stuff to report when you come back and write up the paper. Extraordinary claims really do require extraordinary evidence, and that's where things seem to be breaking down.
Peer review is supposed to catch these things. That reactome paper had chemists rolling their eyes as soon as they saw the synthetic schemes in it, and asking if anyone at the journal had thought to call someone who knew organic chemistry during the review process. This latest arsenic paper has other specialists upset, for different reasons (and, to be sure, for reasons that don't require much scientific specialization at all, as detailed in my post after I'd given the paper a close reading). But that hexacyclinol paper appeared in a chemistry journal, and had (one assumes!) been reviewed by competent chemists. How, then, could it have been published to immediate howls of derision about the quality of the evidence in it?
I also want to make clear that I'm not talking about some of the other categories of bad papers, such as the things are are probably true, but of little interest to anyone. And in the probably-not-true category, lower-ranking journals let not-so-good stuff through pretty often. I've been hard on Biorganic and Medicinal Chemistry Letters here before, among other journals, for publishing things that appear to have been incompetently reviewed. But these journals aren't Science or Nature, and the whole point of prestigious journals is that the things that appear in them are supposed to be important, and they're also supposed to be thoroughly vetted.
Is it the push to land the big papers that will make a big splash? Does that cause people in the editorial offices to bend the rules a bit? The official answer from every journal editor that's ever lived to such questions has been "Of course not!", but you have to wonder. Is it a problem with how they're assigning papers for review - who they go to, or how seriously the reviews are taken when they come back? I really don't know. I just know that we seem to be seeing a lot of embarrassing stuff in the literature these days. It's not supposed to work that way.
Since we were talking about worldwide scientific productivity here the other day, this article in The Economist is timely. They're talking about the share of worldwide R&D (and papers published) by country, and pointing out that the longtime order seems to be changing.
For sheer scientific publications,that order is, of course, the US and Western Europe, followed distantly by everyone else. I've reproduced two graphs from the article, atrocious color schemes and all, and you can see how large the gap has been in the published-paper count.. But there are several interesting features. Note how back in the early 1980s, Russia and Japan were quite similar, but the old Soviet Union (and its successor Russian state) was on the decline even then. Meanwhile, China has come up from nowhere to overtake even Japan. India, South Korea, and Brazil are down in the single digits.
But that brings up some other questions. Take a look at the second graph, on R&D spending as a % of GDP. (This is over a shorter time scale than the paper graph, so adjust your perspective accordingly). Note that Japan has been leading the way here, with South Korea catching up. Neither of them (especially South Korea) publish as much, though, as you'd think, given this investment - is the rest of it going into patents? Or staying inside the labs? Looked at another way, though, the EU is publishing even more than you'd think, given their R&D spending.
You'll see that China is coming up in the spending world, although they're not rising as steeply as South Korea (no one is). India's pretty flat, though, and are being outspent, on this basis, by Brazil. (I hope I'm reading the various shades of aquamarine, teal, and blue-green correctly - you know, the Economist used to be good at presenting information graphically, but whoever let this one through should be whacked on the head).
Neither of these measures is an end in itself. I'd say that robust R&D spending is necessary (but not sufficient) for a country to produce good results. And there are probably a lot of different ways to count things as R&D or not, which we aren't seeing here. As for publications, they're an even rougher measure, since different countries have different cultures (and incentives) for this sort of thing. (Don't forget language barriers, either). And as everyone knows, there are papers and there are papers. Long lists of junk that no one ever reads would be one way to inflate things, but to what good?
Update: Richard Van Noorden at Nature runs the numbers on this paper, and comes to the same conclusions: you can't use it to say that US papers are more prone to fraud. . .
There's a new paper in the Journal of Medical Ethics looking at fraudulent publications. The author went back and studied the papers in the PubMed database that have been retracted during the last ten years (there are 788 of them), looking for trends. And so far, the press coverage that I've seen of his conclusions seems to be missing the point.
One thing that stands out is that retracted papers tend to come from higher-profile journals. That makes sense to me, because that's (a) where they're more likely to be noticed, and (b) where the editors are more likely to care. As you move down the list, people just seem to start shrugging their shoulders. Has a paper ever been retracted from Bioorganic and Medicinal Chemistry Letters, for example? I can't think of any, and readers are invited to try to find one themselves.
Another conclusion is that retracted papers tend to come from serial offenders. Over half the retracted papers had a first author who had retracted something else. I think that probably represents cases where someone had published a body of fraudulent work which all got exposed at once, but if that's not a serial offender, I don't know what is.
But a third conclusion is the one getting the headline writers going. The paper examines the retractions, looking for whether they were withdrawn for error or for fraud. The US is notable for having more retractions in the latter category, as compared to the rest of the world: one third of its 260 retracted papers are attributed to fraud, while the rest of the world comes in between 20% and 25%. So you get things like "US Scientists More Likely To Commit Fraud". But no, for that conclusion to be valid, you'd want to know how many fraudulent papers were published as a percentage of the whole output.
Even that wouldn't tell you the whole story. Remember, we're looking at papers that have actually been retracted. Most published fraudulent research never gets to that point. Lower-end journals have a terrible problem with plagiarized, derivative junk. Piles of it gets sent to them, then too much of it gets into print, and it just sits there, with no one ever paying any attention. Well, years later, some poor person might try to reproduce a prep, find that it doesn't work, sigh, and try something else, but otherwise. . .
No, in the same way that the prominent journals are over-represented, I think that the US might be a bit over-represented because slightly more fraud gets caught. More US papers appear in prominent journals than average, and fewer appear in the absolute bottom-rung journals. The United States accounts for at least one-third of the total scientific paper output, so those 260 papers out of 788 are exactly what you'd expect if all other things were equal. But other things aren't equal. We may have more prominent (and more harmful) frauds here, but I'd be willing to bet that as a proportion of the whole, we have fewer of them.
One last note: the figures in this paper seem to conflict with an earlier analysis, which seems to have found fewer retracted papers in PubMed, and a higher proportion of them tainted by fraud. Who's right?
Here's an attention-getting paper from Tomas Hudlicky (and his co-author Martina Wernerova), and I'd like to help it get some more. It begins:
One who has been reading the literature concerned with organic synthesis in recent years, be it methodology, catalysis, or total synthesis of natural products, may have noticed considerable inflation in the values reported for isolated product yields, ratios of diastereomers, and enantiomeric excess values. A comparison of papers published during the period 1955 to 1980 with those published between 1980 and 2005 reveals that those from the more recent period frequently report isolated product yields of reactions >95%. Such large values were rarely found in the older literature and are all but absent in Organic Syntheses, a journal that only publishes procedures that have been independently reproduced. . .
There, does that sound like the chemical literature you know? Just a bit? Hudlicky has tackled this issue before, and the reasons he advances for the problem remain the same: pressure to make your methods stand out (to the scientific community, to the journal editors, to the granting agencies), a decrease in scale in reactions (making accuracy and precision more difficult), and, finally, what he refers to as "deliberate adjustment". That's well put; the rest of us know it as fraud.
He identifies the mid-1980s as roughly the period when things really started to go to pieces, saying that most procedures in reputable journals before that era are reproducible by, as they say, one skilled in the art, while the numbers have been decreasing since then. And he puts some numbers on the problem, performing a series of test experiments with extremely careful weighing and analysis.
These confirm what every working organic chemist knows: the more manipulations, the more sample you lose. Filtration through a plug of silica gel, into one flask, can give you pretty much complete recovery. But if you cut fractions, you're going to lose about 1%. And if you have to do a separation, even between two widely separated compounds on silica, you're going to lose about 2%. So people who report a >98% yield after chromatography from a real-world crude mixture are kidding themselves. The same goes for extractions and other common methods. In general, every manipulation of a reaction is going to cost you 1 to 2% of your material, even with careful technique. Hudlicky again:
Given that most academic groups do not subject day-to-day reactions to serious optimization or matrix-optimization [6] as is done in industry, it is reasonable to assume that the vast majority of the reactions reported in the literature do not proceed with quantitative conversions. Such aspect would approximate our experiments with mixtures of pure compounds. Because a minimum of three operations (extraction, filtration, and evaporation) is required in working up most reactions, we conclude that yields higher than ca. 94% obtained by work-up and chromatography of crude reaction mixtures are likely unrealistic and erroneous in nature. Such values may arise as a direct consequence of not following correct protocols, which would be expected in the fast-paced academic environment. (An astute student of the organic literature may discover that this very author has been guilty of reporting yields in this range from time to time!)
He goes on to detail the limits of error in weighing, which depend greatly on the amount of sample and the size of the flask. (The smaller the sample-to-container ratio, the worse things get, as you'd figure). And he turns to analyzing mixures of diastereomers by NMR, LC, and the like. As it turns out, NMR is an excellent way to determine these up to about a ratio of 95:5 , but past that, things get tricky. And "past that" is just where a lot of papers go these days, with a precision that is often completely spurious.
Here's the bottom line:
The conclusion drawn from this set of experiments points to the prevalence of serious discrepancies in the reporting of values for yields and ratios in the current literature. We have demonstrated that the facilities and equipment available in a typical academic laboratory are not adequate to support the accuracy of claims frequently made in the literature. . .The current practice of reporting unrealistically high isolated product yields and stereoisomer ratios creates serious problems in reproducibility and hence leads to diminished credibility of the authors.
He recommends a rigorous disclosure of the spread of product yields over multiple experiments, calibration of LC and GC apparatus, or (failing that) at least admitting that no such analysis has been done. (He also recommends getting rid of the concepts of diastereomeric and enantiomeric excess, in line with my fellow Arkansan Robert Gawley's advice). But I think that these ideas, while perfectly reasonable, don't get at the underlying problems - the inflationary pressure to produce more and more noteworthy results. Hudlicky's rules should be adopted - but I fear that they might just push the self-deception (and outright fraud) into newer territories.
I'm glad he's published this paper, though. Because everyone knows that this is a real problem - we complain about it, we joke about it, we mutter and we grit our teeth. But "officially", in the published literature, it's never mentioned. Let's stop pretending, shall we?
Back in January, I wrote about the controversial "Reactome" paper that had appeared in Science. This is the one that claimed to have immobilized over 1600 different kinds of biomolecules onto nanoparticles, and then used chemical means to set off a fluorescence assay when any protein recognized them. When actual organic chemists got a look at their scheme - something that apparently never happened during the review process - flags went up. As shown in that January post (and all over the chemical blogging world), the actual reactions looked, well, otherwordly.
Science was already backtracking within the first couple of months, and back in the summer, an institutional committee recommended that it be withdrawn. Since then, people have been waiting for the thunk of another shoe dropping, and now it's landed: the entire paper has been retracted. (More at C&E News). The lead author, though, tells Nature that other people have been using his methods, as described, and that he's still going to clear everything up.
I'm not sure how that's going to happen, but I'll be interested to see the attempt being made. The organic chemistry in the original paper was truly weird (and truly unworkable), and the whole concept of being able to whip up some complicated reactions schemes in the presence of a huge number of varied (and unprotected) molecules didn't make sense. The whole thing sounded like a particularly arrogant molecular biologist's idea of how synthetic chemistry should work: do it like a real biologist does! Sweeping boldly across the protein landscape, you just make them all work at the same time - haven't you chemists every heard of microarrays? Of proteomics? Why won't you people get with the times?
And the sorts of things that do work in modern biology would almost make you believe in that approach, until you look closely. Modern biology depends, though, on a wonderful legacy, a set of incredible tools bequeathed to us by billions of years of the most brutal product-development cycles imaginable (work or quite literally die). Organic chemistry, though, had no Aladdin's cave of enzymes and exquisitely adapted chemistries to stumble into. We've had to work everything out ourselves. And although we've gotten pretty good at it, the actions of something like RNA polymerase still look like the works of angels in comparison.
Over at Ars Technica, here's an excellent look at the peer review process, which I last spoke about here. The author, Chris Lee, rightly points out that we ask it to do several different things, and it's not equally good at all of them.
His biggest problem is with the evaluation of research proposals for grants, and that has indeed been a problem for many years. Reviewing a paper, where you have to evaluate things that other people have done, can be hard enough. But evaluating what people hope to be able to do is much harder:
. . .Reviewers are asked to evaluate proposed methods, but, given that the authors themselves don't yet know if the methodology will work as described, how objective can they be? Unless the authors are totally incompetent and are proposing to use a method that is known not to work in the area they wish to use it, the reviewer cannot know what will happen.
As usual, there is no guarantee that the reviewer is more of an expert in the area than the authors. In fact, it's more often the case that they're not, so whose judgement should be trusted? There is just no way to tell a good researcher combined with incompetent peer review from an incompetent researcher and good peer review.
Reviewers are also asked to judge the significance of the proposed research. But wait—if peer review fails to consistently identify papers that are of significance when the results are in, what chance does it have of identifying significant contributions that haven't yet been made? Yeah, get out your dice. . .
And as he goes on to point out, the consequences of getting a grant proposal reviewed poorly are much worse than the ones from getting a paper's review messed up. These are both immediate (for the researcher involved) and systemic:
There is also a more insidious problem associated with peer review of grant applications. The evaluation of grant proposals is a reward-and-punishment system, but it doesn't systematically reward good proposals or good researchers, and it doesn't systematically reject bad proposals or punish poor researchers. Despite this, researchers are wont to treat it as if it was systematic and invest more time seeking the rewards than they do in performing active research, which is ostensibly where their talents lie.
Effectively, in trying to be objective and screen for the very best proposals, we waste a lot of time and fail to screen out bad proposals. This leads to a lot cynicism and, although I am often accused of being cynical, I don't believe it is a healthy attitude in research.
I fortunately haven't ever had to deal with this process, having spent my scientific career in industry, but we have our own problems with figuring out which projects to advance and why. Anyone who's interested in peer review, though, should know about the issues that Lee is bringing up. Well worth a read.
How reliable is the medical literature, anyway? This profile of John Ioannidis at The Atlantic is food for thought. Ioannidis is the man behind the famous "Why Most Published Medical Findings Are False" paper a few years ago, and many others in the same vein.
The problems are many: publication bias (negative findings don't get written up and reported as often), confirmation bias, and desire to stand out/justify the time and money/get a grant renewal. And then there's good old lack of statistical power. Ioannidis and his colleagues have noted that far too many studies that appear in the medical journals are underpowered, statistically, relative to the claims made for them. The replication rates of such findings are not good.
Interestingly, drug research probably comes out of his analysis looking as good as anything can. A large confirmatory Phase III study is, as you'd hope, the sort of thing most likely to be correct, even given the financial considerations involved. Even then, though, you can't be completely sure - but contrast that with a lot of the headline-grabbing studies in nutrition or genomics, whose results are actually more likely to be false than true.
Ioannidis's rules from that PLoS Medicine paper are worth keeping in mind:
The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.
The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.
The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.
The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.
The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.
The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.
And although he's talking about the published literature, these things are well worth keeping in mind when you're looking at your own internal data in a drug discovery project. Some fraction of what you're seeing is wrong.
After reading this piece on Chembark, I find that I have to help defend the whole chem-blogging enterprise. The latest Analytical Chemistry has an editorial from Royce Murray, on the subject. Unfortunately, it sounds like something written several years ago. He lays out the current model of scientific publishing, and along the way, briefly defends journal impact factors. Then he says that the lay public has traditionally received news of scientific discoveries through reports in magazines and newspapers, but that the finances in those areas has produced a "shrinkage" of the flow of reliable information to the public. And now we come to the part that really worries him:
In the above light, I believe that the current phenomenon of “bloggers” should be of serious concern to scientists. Bloggers are entrepreneurs who sell “news” (more properly, opinion) to mass media: internet, radio, TV, and to some extent print news. In former days, these individuals would be referred to as “freelance writers”, which they still are; the creation of the modern non-word “blogger” does not change the purveyor. An essential change is that these new freelancers, with the megaphone of the internet, can reach a much larger audience of potential clients than was possible in the past (and harness free “information sources”). . .blogging “agencies” are popping up that openly advertise “no formal qualifications are necessary” (as an internet search for “qualifications of bloggers” revealed). Who are the fact-checkers now? There are no reviewers in a formal sense, and writing can be done for any purpose—political, religious, business, etc.—without the constraint of truth.
He goes on to bemoan the lack of a system to qualify and fact-check these "bloggers" - I like the quotation marks, by the way - maybe we should stick those around every word that entered the language less than ten years ago, just to be sure. Now, it would be easy for me to spend a few paragraphs in the same mode as that last sentence, and as Murray accurately notes, there's no editor to stop me. But I won't, because some of his worries are well-founded.
There is indeed a lot of inaccurate nonsense on the internet. And everyone should read what they find online with a thought to who's written it, and why. But everyone should do the same with stuff that's printed on flattened sheets of dead trees, too, even if there are flattened-dead-tree-sheet editors and fact checkers. This is no place to list the stories that have been horribly messed up by even the most respectable of the old media. I'm thinking of a good list right now; any well-informed person should be able to. (If you can't, you're not as well-informed as you think you are). And there is indeed a lot of good science reporting in newspapers and magazines, although we can't ignore the fact that there's an awful lot of lazy and sloppy science reporting, too.
But there's a lot of inaccurate nonsense in the peer-reviewed literature, too. Without editors and reviewers there would surely be more, but too much junk gets through as it is. And if you want to see that stuff flagged, you'll do well to read the chemistry blogs. Murray's editorial doesn't seem to note that the most widely read ones are all written by chemists, not these unqualified people he's worried about. Would it be a cheap shot to point out that some editing and fact-checking might have caught that point before the editorial went out? Or (the other way around) to point out that a quick look through the scientific blogging world would have done just as well?
This, to me, is the real change that blogging has wrought, and I think it's for the better. Now, anyone who has the desire and ability can write about what they really know, about what they do for a living, and find an audience. I am most definitely not making a living as a journalist. My blog is a useful sideline to my real job, which is drug discovery. When I started in this industry, there was no way for me to self-publish my thoughts about it, but now there is, and I couldn't be happier about that.
Murray is suffering, I think, from a mental block, one that comes from his experience of journalism over his lifetime. He (and many other people) seem to feel that reporting is some sort of special profession, and that "real" journalists are the ones who write for the "real" news outlets. And it's a world where everyone knows which ones those are. It was fairly easy to believe that during the last half of the 20th century, but not so easy before it. (Or, as we're seeing, afterwards). All kinds of scruffy, opinionated people used to run newspapers in this country, and now they have internet sites. As do scientists, professors, lawyers, and anyone else with a keyboard and the desire. They're writing "for any purpose", just the way that Murray is worried about. And it's great.
A couple of postscripts: I should point out that I never would have seen Murray's piece at all had it not been for reading the scientific blogs - I hope that gives him some food for thought. He should also check out the above-linked Chembark post, as well as this response from Terra Sigillata (on an ACS site, no less), this one at InSightU, this one at Science 2.0, and this one at Cephalove. There will surely be more. Quite a lot of discussion! And it would be worth wondering how much of an impact this editorial would have had if it had only appeared in the print version of the journal, instead of being picked up by the Blogging Hordes. Could it be that he knows much more about the internet than he seems to, and he's successfully trolled us all?
Update: here's a trip report on this conference over at Practical Fragments
I'm back from Philadelphia and the FBLD conference. I'm not going to put a trip report up on the blog - although I'm certainly writing one up for my colleagues at work - but a number of people at the meeting asked me what I might say about it here.
Well, I enjoyed it. I tend to like more focused conferences like this one, anyway, where most of the people doing the best work in a field can attend what's still a fairly small meeting. It probably helps that this isn't a very old series of meetings, too. Over time, some sort of scientific entropy sets in, and the topics covered can begin to smear out a bit. Some of the longer-running Gordon Conferences are (to me, anyway) a little blurry about what they're trying to cover.
That same tendency can affect individual talks. We medicinal chemists are particularly guilty of that, since our discipline spreads over a pretty wide area. At a meeting like this one, which was all about fragment-based techniques, people had to resist the temptation to keep going past the fragment-based parts of their talk. Once you get up toward 400 molecular weight, you're not talking about fragments any more - you're doing good old medicinal chemistry. Maybe it's structure-based at that point, maybe not, and maybe you're using some of the biophysical techniques that help out with moving fragment leads forward - but the fragment techniques are what got you to that point, not what's carrying you forward through the concerns about PK, formulations, polymorphs, and all the other later-stage worries of a drug program.
The speakers at this meeting generally did a good job avoiding this pitfall, but I have to admit that the few times I saw PK data come up on the screen, I stopped taking notes. I didn't stop listening, on the chance that there might be something interesting, but it certainly wasn't what I was there to focus on. One could imagine a whole meeting about solving PK problems in drug development - there probably is one, actually. But at that one, you'd have to make sure that the speakers didn't spend time telling you about the neat fragment-based techniques that led to their drug candidate.
As I said, though, there were a lot of interesting speakers at this one, and not a single talk was anything close to a complete waste of time. How many meetings can you say that about? Things ran smoothly, and with notably better food than some of the other conferences I've attended. Some meetings just pitch a bunch of Wissenschaftlerfutter out onto the tables, figuring that people will deal with it - and to be honest, they're usually right. We'll eat most anything in this field, although I've been told that physicists are even less discriminating, so at least we have that.
Y'know, this is what I call an incremental improvement in the synthetic repetoire. I noticed this new paper in Tetrahedron Letters by its title, and read the whole thing just to make sure that I wasn't missing something.
Yep, that's right: someone has come up with a new way to form amides by reacting acid chlorides and amines. "But hold on," you say, "I thought that acid chlorides and amines form amides like an unstoppable juggernaut, which grinds to a halt only when enough HCl is given off to take the remaining amine out of contention". Well, you'd be right about that: but that's because you didn't think of using samarium metal as an acid scavenger.
Because that's what it seems to be here. The authors report that you have to pretty much use a full equivalent of samarium to get the high yields - control experiments with only 1/3 equivalent didn't work so well. What I wish they'd done is run the freaking control experiments with triethylamine. Or Hünig's base. Or pyridine. Or potassium carbonate, or aqueous 0.1N NaOH, or resin-bound nanocrystalline cesium complexes prepared in ionic liquids through renewable green chemistry whatchamacallits - in fact, with damn near anything else except stoichiometric metallic samarium, of all things. Well, OK: zinc and indium didn't work. I stand corrected. Give these folks another four or five Tet Lett papers, and they'll work their way back to baking soda. Only it'll be samarium bicarbonate, with any luck.
Perhaps I'm being unfair here. But really, amide formation is not a problem that is crying out for a new solution. It's really very, very, well worked out, and the number of options available for the experimentalist are nearly beyond counting. But now there's samarium metal. So if you're looking for the most expensive way you can think of to react an acid chloride with an amine, one that will make your labmates question your sanity and a reaction that will probably be a separate item all on its own come your next annual performance review, then go to it.
Oh, and one more thing: if you bother to read the experimental section, which apparently no one did, the procedure is titled: "General procedure for the homocoupling of terminal alkynes". Wrong samarium reaction, guys.
Many readers will remember the "sodium hydride as an oxidizing reagent" story from last year. (For the non-chemists in the audience, the problem here is that sodium hydride is most certainly not what you'd think of as an oxidizing reagent, quite the opposite, in fact. Seeing the paper's title was, for an organic chemist, a bit like reading about a new way to sweeten drinks with vinegar).
This was famously live-blogged over at Totally Synthetic and picked up on around the chemical blog world. The current thinking, though, is that adventitious oxygen is really doing the work here. If you run the reaction under strict inert atmosphere conditions, you get no more oxidation. (And it still doesn't appear that any note has been added to the original JACS paper). Update - make that no note added to the abstract page. The paper itself is still accessible, although it does have notes that it was withdrawn.
Well, now we have another one. This paper in press in Tetrahedron Letters claims oxidation of benzoins to benzils with good ol' sodium hydride. In this case, anyway, the authors (from Korea) did try running the reaction under inert atmosphere, and saw their yield go down. Their proposed mechanism involves molecular oxygen, in fact, and is quite plausible. (I've seen anion-oxygen chemistry myself - if you deprotonate Strecker amines of benzaldehydes, you'll convert them into amides via oxygen in your solvent, that is, if you don't saturate things with inert gas first). Still, I'd rather that they titled this paper differently, since it's not sodium hydride that's doing the oxidation here. You could probably get this to happen with NaHMDS, t-butoxide, or the base of your choice.
And, weirdly, the authors (as far as I can see by going over the PDF) manage not to cite the original JACS NaH oxidation paper at all. You'd never think that anyone had tried this before, especially not in one of the most high-profile chemical journals in the world, just last year, with plenty of added press coverage. What does it take a get a paper cited? Update: given the withdrawn-but-still-available status of the original, this becomes a trickier question. The earlier paper seems to have clearly gone through the same sort of chemistry, but the mechanism - and thus the point of the whole paper - was misassigned. Do you cite it, or not?
Not long ago I wrote about a Chinese journal that said that about a third of its submissions turned out to contain plagiarized material. Journal publishing in that country is apparently a real swamp, and the Chinese government has taken the publishers by surprise with an announcement that they're going to drain it:
Li Dongdong, a vice-minister of state and deputy director of the General Administration of Press and Publications (GAPP) — the powerful government body that regulates all publications in China — acknowledged that the country's scientific publishing had a "severe" problem, with "a big gap between quality and quantity", and needed reform.
Opening a meeting of scientific publishers in Shanghai on 7 September, Li announced that by January 2011, new regulations will be used to "terminate" weak journals.
Precisely how this reform will work is the subject of hot debate. . .News of the regulation startled many of the publishers at last week's meeting.
I'll bet it did, particularly those publishers who are turning out junk. And believe me, you know if you're publishing a crappy journal full of papers that no one reads. As that Nature News article goes on to detail, China is full of "campus journals", which exist only for the local grad students and the like to accumulate lines on their publication record. A colleague of mine used to call such titles "The Journal of Our Results", and that's right on target.
But while I can understand China's desire to upgrade its sometimes embarrassing scientific publishing world, I have to worry about the way that they're choosing to do it - not that it doesn't fit the best traditions of the Chinese government, mind you. This sit-still-while-we-fix-you approach may work in the short term, but if there's a demand for the Journal of Our Results (from authors, if not readers), then won't such titles just spring back up again under different names? As longtime readers here will easily be able to guess, I'd prefer a more market-oriented solution.
If the committees evaluating publishing records decided not to value such journals, much of their reason to exist would presumably vanish. While it's true that there's no perfect way to evaluate journals, a situation like China's - overrun with Journal of Whoozat Technical Colleges, 98% stocked with multi-part papers from the faculty of Whoozat - would seem to be waiting for everyone to just stop pretending. This let's-be-honest-here approach would let the people publishing this junk continue to exist, but they'd probably have to find a better business model.
As it is, just shutting these people down doesn't do anything for the let's-pretend side of the market, which I presume will continue to exist. And it'll probably be filled by the all-new Proceedings of the Whoozat Academy, Whoozat Letters, Acta Whoozatica, and who knows what else.
A comment to the most recent post on puns mentioned the famous JOC paper in verse from the 1970s, and prompted another comment that "If you have to report your results as a villanelle, I think we'll see fewer methyl, ethyl, butyl, futile papers. . ."
Well, it's not a whole paper, for sure. But here's the best that I can do in thirty minutes:
Put In Another Methyl Group: A Villanelle
I shouldn't have to put a methyl there
No matter what the modeling group might say
So it docks to perfection: I don't care.
The project head gave me an evil glare
When I spoke up at our review today.
I shouldn't have to put a methyl there.
"The glutamate will pick up that lone pair".
Who knows? That might be right; I couldn't say.
So it docks to perfection: I don't care.
How do these really bind? We don't know where.
It's not like we can get a good X-ray.
I shouldn't have to put a methyl there.
Quaternary chiral centers? I don't dare.
I'd need two months if I needed a day.
So it docks to perfection: I don't care.
But no one ever said research was fair.
I'm going to have to come up with a way.
I shouldn't have to put a methyl there.
So it docks to perfection: I don't care.
Update: yes, I'm going to give the molecular modelers their own poem. It's only fair!
Are we going to see this in all the Wiley-hosted European-based journals? Angewandte Chemie has specialized, as has been noted many times, in wince-worthy puns in its article abstracts.
Today I take a look at ChemBioChem and find this. No one is safe.
The topic of plagiarism in scientific journals has come up here several times. In recent years, automated systems for checking similar blocks of text have become available, and a number of journals now run their submissions past such software.
The first journal in China to sign up for the most well-known of these (CrossCheck) is the Journal of Zhejiang University–Science. I'll freely admit that I'd never heard of it, not that I've heard of a lot of the Chinese-language journals. But I also have to take my hat off to them, both for using the plagiarism-detection service and especially for writing in to Nature with the results.
Since October 2008, they've found "unoriginal material" in 31% of all their submissions, a number they themselves call "staggering". (Here's an earlier report on their progress). The letter mentions some possible cultural problems, such as Chinese students traditionally being asked to copy things word-for-word from authorities, but I'd guess that there's plenty of the good old publish-or-perish at work here, too. At any rate, congratulations to them for publicizing such problems; that's the only way they'll ever get any better.
Blogging time is short today, since I'm on a deadline to produce a couple of posters for presentation. These are for an internal hoe-down, unfortunately, so I won't be able to share the fruits of my labors with everyone out there in the readership. With any luck, though, they'll turn into public presentations/publications eventually, though.
As far as I'm concerned, posters are quite a bit harder to work up than a talk. They really should stand by themselves, for one thing, so you can't fill in any holes verbally. And narrative flow is harder: there's no chance to go back and re-emphasize or contrast with later slides, because the whole thing is sitting out there, with no guarantee of what order people will use to see its parts. (I find that narrative is one of my main weapons in a presentation, so going without it is always tough).
I care about design, too, probably more than I should, so a poster also presents complications there. If visual cues wander a bit from slide to slide through a presentation, that's not good, but it's not fatal, either. But when everything's up there on one sheet, the messages really have to be consistent: same fonts, same colors, same rotations, views, and angles, etc.
But at these times I try to remind myself of what happened to a friend of mine many years ago. She was working on a poster for an ACS meeting, and took it to her PI to look over. "Yes, yes, that looks good", came the word, "but could you perhaps take this part over to here? And emphasize this a bit more? And. . ." So she went back and made the changes, and took the poster back for a re-check. "Much better! Yes. . .but I wonder if maybe this part should be bigger? And did you find a way to include those results where. . ." Back for another round.
After another iteration of this, she caught on. She started taking an unchanged version back to the PI, and after another couple of rounds of seeing the exact same poster, it was finally pronounced ready for viewing. Saves time, saves effort - try it when you can!
Why doesn't anyone comment on scientific papers? Let's narrow that down: why doesn't anyone comment on them when they have a comments field attached to them on a scientific publisher's web site?
Nature has been wondering about this for a while. In a new item about "Web 2.0" tools in science, they mention:
But deposition in arXiv is about as far as the scientific openness of even astronomers goes. The discussion that ensues is private. As Nature's experiment in open peer review showed (http://go.nature.com/N67mFk), and as can be seen from the lack of commenting on papers in Nature and other journals that encourage it, researchers see little to be gained from open discourse before or after publication. Not only are they busy, as the above quotes attest, but there's no credit to be gained, and some risk if one makes an erroneous or critical statement in public. What is more, astronomers and biologists register active discouragement of blogging — a form of communication that in their eyes carries no stamp of reliability or prestige. That picture of resistance to interactive discussion of science on the Internet is further amplified in a new survey, If You Build It, Will They Come? How Researchers Perceive and Use Web 2.0, to be published later this month by the UK Research Information Network.
Contrast that with a comment left here the other day, where a reader suggested that a great business plan might be to start a web site where people could comment on scientific publications. How to reconcile these world views? I think that one word goes a long way: anonymity. The first reaction to that is often a mental picture of the comments pages to (say) YouTube videos, and a brief shudder. And it's true that comments sections that allow easy anonymous accounts can attract all sorts of nasty stuff, as newspapers found out when they opened up their sites.
But anonymity has a long, distinguished history in science. Peer review! People will speak frankly about a paper under review, as long as they know that their comments will remain untraceable by the manuscript's authors. What makes it acceptable is that the editors of the journal know all the names involved, which (usually) keeps things above the midnight-ninja-assassination level.
So what would be needed, I think, would be a site where real e-mail and contact information would be required. You would be free to post under your real name, if you wanted to, or to take on whatever nom de guerre you wished, with the total assurance this this would not be revealed. At the same time, sheer invective and libel would be tossed out immediately, with the line to be drawn at the discretion of the site's editors. Personally, I'd delete comments that said only "This paper sucked", but I'd leave in the ones that said "This paper sucked because of X, Y, and Z." And the comments that started that way but then went on to talk about how the authors sucked in similar fashion would be truncated with some sort of standard mark meaning "ad hominem deleted".
That would be a lot of work. And there's a good chance that it would never take off at all, given the amount of trust involved. Would I wish to run such a site? No way - I already have a full-time job, thanks. But I'd like to see someone try.
Yesterday morning I went on and on about the low quality of much of what gets published in the scientific literature. And indeed, the low end is very likely of no use to anyone, except (apparently) the people publishing it. But what to do with the rungs above that?
For organic chemistry, those are occupied by papers that report new compounds of little interest to anyone. But you never know - they might be worth someone else's time eventually. It's unlikely that any of these things will be the hinge on which a mighty question turns, but knowing that they've been made (and how), and knowing what their spectra and properties are could save someone time down the line when they're doing something more useful. These are real bricks in the huge construction of scientific knowledge, and while they're not worth much, it's more than zero. That's the value I assign to the hunks of mud that some people offer instead, or the things that look like real bricks but turn out to be made out of brick, yes, but about one millimeter thick and completely hollow.
So what to do with work that's mostly reference data for the future? It shouldn't have to appear in physical print, you'd think. How about the peer-reviewed journal part? Well, peer review is not magic. As it stands, that sort of information is the least-reviewed part of most papers. If someone tells you that they've made Compound X and Compound Y, and the synthesis isn't obviously crazy, you tend to take their word for it. It's a rare reviewer that gets all the way down to the NMR spectra in the supplementary material, that's for sure. And if one does, and the NMR spectra look reasonably believable, well, what else can you do? Even so, every working chemist has dealt with literature whose procedures Just Don't Work, and all those papers passed some sort of editorial review process at some point.
No, peer review is not going to do much to improve the quality of archival data. If someone really wants to fill up the low-level bins with junk, there's not much stopping them. You could sit down and draw out a bunch of stuff no one's ever made before, come up with plausible paper syntheses of all of it, use software to predict reasonable NMR spectra (which you might want to jitter around a bit to cover your tracks), and just flat-out fake the mass spec and elemental analyses. Presto, another paper that no one will ever read, until eventually someone has a reason to make similar compounds and curses your name in the distant future. The problem is, such papers will do you no real good, since they'll appear in the crappiest journals and pick up no citations from anyone.
Perhaps there should be a way to dump chemical data directly into some archives, the way X-ray data goes into the Protein Data Bank. That wouldn't count for much, but it would capture things for future use. Having it not count much would decrease the incentive for anyone to fill it full of fakery, too, since there would be even less point than usual. And before anyone objects to having a big pile of non-peer-reviewed chemical data like this, keep in mind that we already have one: it's called the patent literature, and it can be quite worthwhile. Although not always.
That's what this article at the Chronicle of Higher Education could be called. Instead it's headlined "We Must Stop the Avalanche of Low-Quality Research". Which still gets the point across. Here you have it:
While brilliant and progressive research continues apace here and there, the amount of redundant, inconsequential, and outright poor research has swelled in recent decades, filling countless pages in journals and monographs. Consider this tally from Science two decades ago: Only 45 percent of the articles published in the 4,500 top scientific journals were cited within the first five years after publication. In recent years, the figure seems to have dropped further. In a 2009 article in Online Information Review, Péter Jacsó found that 40.6 percent of the articles published in the top science and social-science journals (the figures do not include the humanities) were cited in the period 2002 to 2006.
As a result, instead of contributing to knowledge in various disciplines, the increasing number of low-cited publications only adds to the bulk of words and numbers to be reviewed. Even if read, many articles that are not cited by anyone would seem to contain little useful information. . .
If anything, this underestimates things. Right next to the never-cited papers are the grievously undercited ones, most of whose referrals come courtesy of later papers published by the same damn lab. One rung further out of the pit are a few mutual admiration societies, where a few people cite each other, but no one else cares very much. And then, finally, you reach a level that has some apparent scientific oxygen in it.
The authors of this article are mostly concerned about the effect this has on academia, since all these papers have to be reviewed by somebody. Meanwhile, libraries find themselves straining to subscribe to all the journals, and working scientists find the literature harder and harder to effectively cover. So why do all these papers get written? One hardly has to ask:
The surest guarantee of integrity, peer review, falls under a debilitating crush of findings, for peer review can handle only so much material without breaking down. More isn't better. At some point, quality gives way to quantity.
Academic publication has passed that point in most, if not all, disciplines—in some fields by a long shot. For example, Physica A publishes some 3,000 pages each year. Why? Senior physics professors have well-financed labs with five to 10 Ph.D.-student researchers. Since the latter increasingly need more publications to compete for academic jobs, the number of published pages keeps climbing. . .
We can also lay off some blame onto the scientific publishers, who have responded to market conditions by starting new journals as quickly as they can manage to launch them. And while there have been good quality journals launched in the past few years, there have been a bunch of losers, too - and never forget, the advent of a good journal will soak up more of the worthwhile papers, lifting up the ever-expanding pool of mediocre stuff (and worse) by capillary action. You have to fill those pages somehow!
If this problem is driven largely by academia, that's where the solution will have to come from, too. The authors suggest several fixes: (1) limit job applications and tenure reviews to the top five or six papers that a person has to offer. (2) Prorate publication records by the quality of the journals that the papers appeared in. (3) Adopt length restrictions in printed journals, with the rest of the information to be had digitally.
I don't think that those are bad ideas at all - but the problem is, they're already more or less in effect. People should already know which journals are the better ones, and look askance at a publication record full of barking, arf-ing papers from the dog pound. Already, the best papers on a person's list count the most. And as for the size of printed journals, well. . .there are some journals that I read all the time whose printed versions I haven't seen in years.
No, these ideas are worthy, but they don't get to the real problem. It's not like all the crappy papers are coming from younger faculty who are bucking for tenure, you know. Plenty more are emitted by well-entrenched groups who just generate things that no one ever really wants to read. I think we've made it too possible for people to have whole scientific careers of complete mediocrity. I mean, what do you do, as a chemist, when you see another paper where someone found a reagent to dehydrate a primary amide to a nitrile? Did you read it? Of course not. Will you ever come back to it and use it? Not too likely, considering that there are eight hundred and sixty reagents that will already do that for you. We get complaints all the time about me-too drugs, but the me-too reaction problem is a real beast.
Now, I realize that by using the word "mediocrity" I'm in danger of confusing the issue. The abilities of scientists are distributed across a wide range - I doubt if it's a true normal distribution, but there are certainly people who are better and worse at this job. But I'm complaining on the absolute scale, rather than the relative scale. I know that there's always going to be a middle mass of scientific papers, from a middle mass of scientists: I just wish that the whole literature was of higher quality overall. A chunk of what now goes into the mid-tier journals should really be filling up the bottom-tier ones, and most of the stuff that goes into those shouldn't be getting done in the first place.
I suppose what bothers me is the number of people who aren't working up to their potential (although I don't always have the best position to argue that from myself!) Too many academic groups seem to me to work on problems that are beneath them. I know that limits in money and facilities keep some people from working on interesting things, but that's rare, compared to the number who'd just plain rather do something more predictable. And write predictable papers about it. Which no one reads.
That's the University of California system versus Nature Publishing Group, in case you were wondering. As the Chronicle of Higher Education reports, there's a mighty dispute brewing about the cost of electronic access:
On Tuesday, a letter went out to all of the university's faculty members from the California Digital Library, which negotiates the system's deals with publishers, and the University Committee on Library and Scholarly Communication. The letter said that Nature proposed to raise the cost of California's license for its journals by 400 percent next year. If the publisher won't negotiate, the letter said, the system may have to take "more drastic actions" with the help of the faculty. Those actions could include suspending subscriptions to all of the Nature Group journals the California system buys access to—67 in all, including Nature.
The pressure does not stop there. The letter said that faculty would also organize "a systemwide boycott" of Nature's journals if the publisher does not relent. The voluntary boycott would "strongly encourage" researchers not to contribute papers to those journals or review manuscripts for them. It would urge them to resign from Nature's editorial boards and to encourage similar "sympathy actions" among colleagues outside the University of California system.
NPG's testy response is here, and this is the reply from California. The current points of dispute are how much the publishers are actually raising the prices (site license fees versus the base rate) and how much of a discount the UC system is getting already.
Could there really be a UC boycott? They're large enough (and productive enough) to make that a reasonably credible threat. The Nature journals will certainly survive without submissions from the UC system, although over the last six years they've contributed over five thousand papers to them. But the real danger, I think, is the damage that this could do to Nature's position, and to the whole idea of the high-prestige journals. The scientific publishing world has been feeling the earthquake tremors for some time now. The traditional model (1. Start an academic journal. 2. Charge whopping subscription fees. 3. Profit) seems to be breaking down, in the same way that many other traditional content-distribution pricing models have been.
Nature and its related journals, along with the other top-tier publications, have managed to stay on top (and to charge accordingly). But journal prestige is an artificial construct, a fiction by common consent. A journal has a good reputation because it's hard to publish in and can afford to reject all but the most high-impact papers that it's offered. If people stop offering such papers to it, its prestige will decline. The big-splash papers will go somewhere else, and will perhaps manage to signal their importance in some other way than by the name of the journal they appear in.
This dispute will be worth watching closely. Which side will give in? Will a UC boycott be effective, and could it spread? Remember, from one perspective, other journals have an interest in seeing this happen, since they'll now see the papers that NPG won't. But they might also fear the same thing happening to them if this succeeds. . .
Today's entry on an embarrassingly wrong structure in Bioorganic and Medicinal Chemistry Letters has a number of people in the comments talking about experiences they've had reviewing for the journal. I've done reviews for them myself, naturally, as well as for other journals. I have too few examples to judge from, but (so far) I have managed to kill off papers at other journals, but I have never managed to kill off a manuscript at BOMCL. I have - apparently like some other people - recommended in the past that a paper be published (if at all) only with major revisions, only to see it sail through basically untouched.
I hope I'm not being unfair here, because there are a lot of hard-working people at the journal. And similar stories can, I'm sure, be told about every other journal (in every other discipline). But I think that BOMCL gets so many manuscripts that their workload is very high. Unfortunately, the journal also publishes a great deal of what it gets. We're unlikely to see the real figures any time soon, but I'd have to guess that the percentage of papers rejected is definitely lower than average.
I also realize that I'm open to accusations of conflict of interest here, since I'm on the editorial board of a competing journal, ACS Medicinal Chemistry Letters. What I can promise is that I will, in fact, work to keep any papers that I consider inadequate out of those pages (and, at the same time, to encourage good work to go there). I've tried to do that with BOMCL, too, in my capacity as reviewer, but it just hasn't always worked out.
But perhaps this is something I can do for them: to point out, publicly, that their credibility as a venue for medicinal chemistry results has suffered recently. When people get the impression that work is being sloppily reviewed at a given journal, they wonder how much they should trust the other papers that get published. Bioorganic and Medicinal Chemistry Letters has been around for twenty years now, and has published some very useful stuff over the years. There's a real place for it in the publishing world. But it's been better than it has been recently, and it should be better than it is. I hate to say this. But someone should.
Something definitely went wrong with this paper: check out the thiophenes, which look through the whole paper just like they do in that abstract. It's another who-let-that-through moment for Bioorganic and Medicinal Chemistry Letters. Perhaps it's all a plot, to get you to read every paper in the hopes that something bizarre will turn up. . .
The fine people at Angewandte Chemie's English edition, who continue to have way too much time on their hands, appear to have declared this 70s and 80s music month. So far I've spotted The Beach Boys, Toni Basil (a particularly strained one), and Elvis Costello.
There are movies, too here's another James Bond reference - but if anyone can tell me where the "Beverley Hills" part comes into this one, other than the desire to reach for a joke, I'd be grateful.
Finally, I have proof that at least one of their English-speaking editors is not only American, but specifically from Baltimore. Number of non-English speakers who will get that reference? To be counted on the fingers of the hands. . .
Update: I get the reference to the movie "Beverley Hills Cop II", but I just don't understand reaching for that reference - other than, of course, to show that you can. And as for the Baltimore one, I hadn't noticed that the authors are from Johns Hopkins, so we have to presume that they're to blame - and that the editors at the journal let that one through, too. . .
I was talking with a colleague yesterday, and I suddenly had an insight into an opportunity in scientific publishing. We were discussing the various computational/modeling papers that you see out in the literature. Some of them are quite interesting, many are worth looking at if it's your particular field - but many others are, well, not so great. I should mention up front that the same objections apply - and how - to the non-computational literature, of course. But there are a number of second-tier (and lower) journals to soak up those sorts of papers in the other disciplines.
What surprises me is that there's no Computational Chemistry Letters or some such. Communications in Computational Chemistry? CADD Comm? This would be the dumping ground for the piles of unconvincing computer-driven stuff that gets sent around by people who have paid a bit too much attention to the sales brochures that came with their software packages.
The barriers for entry to such things have been getting lower and lower, while the real state of the art has been getting more and more complicated. That's created a gap into which too much stuff falls. Who will speak for the bottom-dwelling "We modeled it, therefore it's real" constituency? The advent of systems biology has created more opportunities than ever for these folks. Isn't it time that there was an expensive, low-impact, completely disregardable journal for them, too?
Since I'm on the editorial board, I should point out that the first full issue of ACS Medicinal Chemistry Letters is up, with free access for the occasion. I note with approval Dennis Liotta's lead-off editorial, which mentions that the journal will be introducing a section to highlight key patents - as far as I can tell, this is the first time a med-chem journal has explicitly acknowledged that this is where the real cutting-edge stuff often first appears in the field. (And no, I didn't have anything to do with the current batch of papers, in case anyone's wondering).
In keeping with my Modest Literature Proposal from earlier this year, I would like to briefly point out a Journal of Medicinal Chemistry paper on potential Alzheimer's therapies. Whose lead compound has a nine-carbon alkyl chain in the middle of it. And weighs 491. And has two quaternary nitrogens. Which structural features will, in all likelihood, lead to said compound demonstrating roughly this amount of blood-brain barrier penetration, assuming it reaches sufficient blood levels to get that far. That is all.
Science and Naturefinally decide to stop beating each other up in chasing after hot papers. The problem with the announcement is that it quotes magazine consultant Rick Rolling, but neglects to link to the video where he explains things in detail.
Well, now, this is a disappointment. In a new Angewandte Chemie paper, a French team reports synthesizing trinitropyrazole. And it's. . .well, it's well-behaved. Surprisingly insensitive. Not that touchy. Might actually be useful as a storable high-energy material that could actually be handled.
The fools! Don't they realize that Angewandte is the place to unload the barely-in-our-plane-of-existence compounds, the sweat-starting, nostril-flaring "How could it blow up? It's in liquid nitrogen!" stuff? Surely there's a better home for things with actual utility, the Journal of Not So Horrible Once You've Made Them, Really, or "That Wasn't So Bad Now, Was It" Communications. Sheesh.
A new paper in PLoS Biology looks at animal model studies reported for the treatment of stroke. The authors use statistical techniques to try to estimate how many have gone unreported. From a database with 525 sources, covering 16 different attempted therapies (which together come to 1,359 experiments and 19,956 animals), they find that only a very small fraction of the publications (about 2%) report no significant effects, which strongly suggests that there is a publication bias at work here. The authors estimate that there may well be around 200 experiments that showed no significant effect and were never reported, whose absence would account for around one-third of the efficacy reported across the field. In case you're wondering, the therapy least affected by publication bias was melatonin, and the one most affected seems to be administering estrogens.
I hadn't seen this sort of study before, and the methods they used to arrive at these results are interesting. If you plot the precision of the studies (Y axis) versus the effect size (X axis), you should (in theory) get a triangular cloud of data. As the precision goes down, the spread of measurements across the X-axis increases, and as the precision goes up, the studies should start to converge on the real effect of the treatment, whatever that might be. (In this study, the authors looked only at reported changes in infarct size as a measure of stroke efficacy). But in many of the reported cases, the inverted-funnel shape isn't symmetrical - and every single time that happens, it turns out that the gaps are in the left-hand side of the triangle, the not-as-precise and negative-effect regions of the plots. This doesn't appear to be just due to less-precise studies tending to show positive effects for some reason - it strongly suggests that there are negative studies that just haven't been reported.
The authors point out that applying their statistical techniques to reported human clinical studies is more problematic, since smaller (and thus less precise) trials may well involve unrepresentative groups of patients. But animal studies are much less prone to this problem.
The loss of experiments that showed no effect shouldn't surprise anyone - after all, it's long been known that publishing such papers is just plain harder than publishing ones that show something happening. There's an obvious industry bias toward only showing positive data, but there's an academic one, too, which affects basic research results. As the authors put it:
These quantitative data raise substantial concerns that publication bias may have a wider impact in attempts to synthesise and summarise data from animal studies and more broadly. It seems highly unlikely that the animal stroke literature is uniquely susceptible to the factors that drive publication bias. First, there is likely to be more enthusiasm amongst scientists, journal editors, and the funders of research for positive than for neutral studies. Second, the vast majority of animal studies do not report sample size calculations and are substantially underpowered. Neutral studies therefore seldom have the statistical power confidently to exclude an effect that would be considered of biological significance, so they are less likely to be published than are similarly underpowered “positive” studies. However, in this context, the positive predictive value of apparently significant results is likely to be substantially lower than the 95% suggested by conventional statistical testing. A further consideration relating to the internal validity of studies is that of study quality. It is now clear that certain aspects of experimental design (particularly randomisation, allocation concealment, and the blinded assessment of outcome) can have a substantial impact on the reported outcome of experiments. While the importance of these issues has been recognised for some years, they are rarely reported in contemporary reports of animal experiments.
And there's an animal-testing component to these results, too, of course. But lest activists seize on the part of this paper that suggests that some animal testing results are being wasted, they should consider the consequences (emphasis below mine):
The ethical principles that guide animal studies hold that the number of animals used should be the minimum required to demonstrate the outcome of interest with sufficient precision. For some experiments, this number may be larger than those currently employed. For all experiments involving animals, nonpublication of data means those animals cannot contribute to accumulating knowledge and that research syntheses are likely to overstate biological effects, which may in turn lead to further unnecessary animal experiments testing poorly founded hypotheses.
This paper is absolutely right about the obligation to have animal studies mean something to the rest of the scientific community, and it's clear that this can't happen if the results are just sitting on someone's hard drive. But it's also quite possible that for even some of the reported studies to have meant anything, that they would have had to have used more animals in the first place. Nothing's for free.
I realize that there have been about one Godzillion of these videos by now (the Michael Jackson one is a particular favorite), but at the same time, I'd be lying if I said that I didn't have similar thoughts the last time I had a manuscript rejected:
The movie is, of course, "Downfall" (Der Untergang), and probably many more people have been motivated to see the whole thing thanks to these YouTube clips. I have to turn the sound down a lot to really get into the spirit of the parodies; my German is still sufficient to make the reworked subtitles clash. Which reminds me of a distinctly weird sensation, watching a World War II documentary at home with my father after returning from my German post-doc stint. When archival footage of Hitler came up on the screen I suddenly found that I could follow his speech, and watched in amazement as he pounded the podium, hitting the compound verbs at the end of his sentence.
I was thinking the other day about the sheer number of reasonable chemical structures that have never been made. Chemical space is famously roomy - that's how we make a living in the drug industry, since we prefer to make things that have never been made before. And it still surprises non-chemists when I tell them that I make new compounds all the time - the feeling, I think, is that anything that's reasonably easy to make surely must have been mined out long ago. Not so. (It's worth remembering, though, that just because something's never been reported doesn't always mean that you can't buy it).
What brought this to mind was a steroid structure that I saw during a presentation. Looking at it like a medicinal chemist, I wondered idly if the carbons in the famous steroid backbone had ever been swapped out much with oxygen or nitrogen atoms. And in a few cases they have (more for oxygen, in some natural products), but for the most part, no. You can drop a tertiary amine into some spots on the steroid framework and immediately come up with no literature hits whatsoever. Many others yield only a handful.
It's worth noting that the partially-aromatized steroids have had some of this kind of work done on them - for example here and here. The aromatic rings give you a bit more of a handle to work with, but even here it's not like the literature is always packed with examples.
So there's as bioactive a scaffold as you could ask for, but many of the simple analogs still haven't been described. To be fair, these azasteroids aren't simple to make, and probably wouldn't have steroid-like activities in many cases. (Their natural receptors sure aren't expecting a basic amine in those spots). But many azasteroids do show biological activities, and I'd be quite surprised if these unknown compounds were pharmacologically inert. It's just that there's been no particular reason to make any of them yet. Chemical space is so huge, and our ability to explore it has been with us for such a relatively short time, that we just haven't gotten around to them yet.
There have been complaints that something is going wrong in the publication of stem cell research. This isn't my field, so I don't have a lot of inside knowledge to share, but there appear to have been a number of researchers charging that journals (and their reviewers) are favoring some research teams over others:
The journal editor decides to publish the research paper usually when the majority of reviewers are satisfied. But professors Lovell-Badge and Smith believe that increasingly some reviewers are sending back negative comments or asking for unnecessary experiments to be carried out for spurious reasons.
In some cases they say it is being done simply to delay or stop the publication of the research so that the reviewers or their close colleagues can be the first to have their own research published.
"It's hard to believe except you know it's happened to you that papers have been held up for months and months by reviewers asking for experiments that are not fair or relevant," Professor Smith said.
You hear these sorts of complaints a lot - everyone who's had a paper turned down by a high-profile journal is a potential customer for the idea that there's some sort of backroom dealing going on for the others who've gotten in. But just because such accusations are thrown around frequently doesn't mean that they're never true. I hate to bring the topic up again, but the "Climategate" leaks illustrate just how this sort of thing can be done. Groups of researchers really can try to keep competing work from being published. I just don't know if it's happening in the stem cell field or not.
A double complaint this morning, and both from the same literature item - if I were charging anything for the blog, I'd say that it's delivering value for the money. At any rate, the first kvetch is something that I know that many chemists have noticed when reading more biology/medical-oriented journals. You'll see some paper that talks about a new compound that does X, Y, and Z. It'll be named with some sort of code, and they'll tell you all about its interesting effects. . .but they don't get around to actually telling you what the damned stuff is.
As I say, this is a chemist's complaint. Many biologists are fine stipulating that there's a compound that will do these interesting things, because they're mostly interested in hearing about the interesting things themselves. It could just be Compound X as far as they're concerned. But chemists want to see what kind of structure it is that causes all these publication-worthy results, and sometimes we go away disappointed.
Or we have to dig. Take this PNAS paper on a broad-spectrum antiviral compound, LJ001. It looks quite interesting, with effects on a number of different viral types, and through a unique mechanism that targets viral membranes. But what is it? You'll look in vain through the whole paper to find out - that compound is LJ001 to you, Jack. You have to go to the supplemental material to find out, and to page 10 at that.
And that brings up the second complaint. LJ001 turns out to be a rhodanine, and regular readers will note that earlier this month some time was spent here talking about just how ugly and undesirable those are. It's very, very hard to get anyone in the drug business to take a compound in that class seriously, because they have such a poor track record. Looking over the small SAR table provided, I note that if you switch that thioamide group (the part that the chemists hate the most) to a regular amide, turning the thing into an thiazolidinedione, you lose all the activity.
TZDs aren't everyone's favorite group, but at least they've made it into marketed drugs. Rhodanines are no one's favorite group, and it would be a good thing of the authors of these papers would realize that, or at least acknowledge it if they do. It's not an irrational prejudice.
So I have a number of people trying to set me straight about Twitter. . .well, I'll see what I can do. For some time I've had it set up just to take 140 characters off the top of each post I do here, to serve as a sort of "I've posted something" alert, and that'll continue. I hardly follow anyone there, true. . .and many of them are non-chemical sources (Iranian politics and the like). What I probably need to do is set up more than one Twitter account, with one reserved for blogging and science. But even then, I don't see how I'll have time to look at it during the day, so people who fire messages back to me via Twitter are still going to come away disappointed.
My schedule is all over the place today - events at my kids' school, new projects at work, etc. But I do want to put a quick question out to people: I keep seeing various scientific journals, etc. proudly advertising that they're on Twitter, Facebook, etc. So, does anyone get any use out of that? I can't say that I do, but perhaps I'm just set in my ways, if reading journals by RSS feeds can be called "set in my ways".
I'm willing to be set straight on this, but whenever I see these logos and notices, I can't help but see some editorial meeting that I imagine went on. "Look, everybody's on Twitter, says one person around the table. "Well, I'm not," says an editor, "and I can't for the life of me figure out why I should be. Won't we look idiotic 'tweeting' at people, or whatever it is?" "Did I mention that it won't cost us anything to get all Web 2.0-ed up?" says the first guy, and the motion is carried. . .
I should also note that the Royal Society of Chemistry is starting its own med-chem communications journal. MedChemComm. Along with the new ACS journal, this now means that medicinal chemists have more places to publish their work than ever before.
Which is a bit of a sour thought, considering that the number of industry-employed medicinal chemists has been dropping for several years now, and the end does not appear to be in sight. We'll see how this affects the publishing world (admittedly, a minor worry). In the short term, people are probably trying to make their patent and publication records look as impressive as possible, so I would think that fewer and fewer publishable results are sitting around in desk drawers. In the long term, though, are we going to see fewer papers in general? (Or failing that, more from academic labs?)
There's an article out from a group in Australia on the long-standing problem of "frequent hitter" compounds. Everyone who's had to work with high-throughput screening data has had to think about this issue, because it's clear that some compounds are nothing but trouble. They show up again and again as hits in all sorts of assays, and eventually someone gets frustrated enough to flag them or physically remove them from the screening deck (although that last option is often a lot harder than you'd think, and compound flags can proliferate to the point that they get ignored).
The larger problem is whether there are whole classes of compounds that should be avoided. It's not an easy one to deal with, because the question turns on how you're running your assays. Some things are going to interfere with fluorescent readouts, by absorbing or emitting light of their own, but that can depend on the wavelengths you're using. Others will mung up a particular coupled assay readout, but leave a different technology untouched.
And then there's the aggregation problem, which we've only really become aware of in the past few years. Some compounds just like to stick together into huge clumps, often taking the assay's protein target (or some other key component) with them. At first, everyone thought "Ah-hah! Now we can really scrub the screening plates of all the nasties!", but it turns out that aggregation itself is an assay-dependent phenomenon. Change the concentrations or added proteins, and whoomph: compounds that were horrible before suddenly behave reasonably, while a new set of well-behaved structures has suddenly gone over to the dark side.
This new paper is another attempt to find "Pan-Assay Interference" compounds or PAINs, as they name them. (This follows a weird-acronym tradition in screening that goes back at least to Vertex's program to get undesirable structures out of screening collections, REOS, for "Rapid Elimination of, uh, Swill"). It will definitely be of interest to people using the AlphaScreen technology, since it's the result of some 40 HTS campaigns using it, but the lessons are worth reading about in general.
What they found was that (as you'd figure) that while it's really hard to blackball compounds permanently with any degree of confidence, the effort needs to be made. Still, even using their best set of filters, 5% of marketed drugs get flagged as problematic screening hits - in fact, hardly any database gives you a warning rate below that, with the exception of a collection of CNS drugs, whose properties are naturally a bit more constrained. Interestingly, they also report the problematic-structure rate for the collections of nine commercial compound vendors, although (frustratingly) without giving their names. Several of them sit around that 5% figure, but a couple of them stand out with 11 or 12% of their compounds setting off alarms. This, the authors surmise, is linked to some of the facile combinatorial-type reactions used to prepare them, particularly ones that leave enones or exo-alkenes in the final structures.
So what kinds of compounds are the most worrisome? If you're going to winnow out anything, you should probably start with these: Rhodanines are bad, which doesn't surprise me. (Abbott and Bristol Myers-Squibb have also reported them as troublesome). Phenol Mannich compounds and phenolic hydrazones are poor bets. And all sort of keto-heterocycles with conjugated exo alkenes make the list. There are several other classes, but those are the worst of the bunch, and I have to say, I'd gladly cross any of them off a list of screening hits.
But not everyone does. As the authors show, there are nearly 800 literature references to rhodanine compounds showing biological effects. A conspicuous example is here, from the good folks at Harvard, which was shown to be rather nonspecifically ugly here. What does all this do for you? Not much:
"Rather than being privileged structures, we suggest that rhodanines are polluting the scientific literature. . .these results reflect the extent of wasted resources that these nuisance compounds are generally causing. We suggest that a significant proportion of screening-based publications and patents may contain assay interference hits and that extensive docking computations and graphics that are frequently produced may often be meaningless. In the case of rhodanines, the answer set represents some 60 patents and we have found patents to be conspicuously prevalent for other classes of PAINS. This collectively represents an enormous cost in protecting intellectual property, much of which may be of little value. . ."
Looking through the latest papers to show up in the Journal of Medicinal Chemistry, this one on BACE-1 inhibitor compounds caught my eye. Perhaps I'm about to be unfair to it. At any rate, I'm going to ask of it something it doesn't provide: data in something that's alive. Doesn't have to be a person, a dog, or even a rat. A cell would do: something with a membrane to cross, with metabolic processes, and with the ability to accept or reject someone's new compound. Enzymes just have to sit there and take whatever you throw at them; living systems fight back.
I sometimes think that we'd be better served if each of the medicinal chemistry journals were split. In J. Med. Chem.'s case, we would then have the Journal of In Vitro Medicinal Chemistry and the Journal of In Vivo Medicinal Chemistry. The criteria for publishing in the two journals would be exactly the same, except to get into the latter one, you would have at least had to have tried your compounds out on something besides an in vitro assay. Doesn't mean that they have to have worked - you just have to have looked.
Although the case of compounds with molecular weights of 900 that have four amides and a sulfonamide in them, and are directed against a target in the central nervous system, might still be a bit of a stretch. I supposed what irritates me about this paper is that it starts off talking about Alzheimer's disease. And that's natural enough in a study dedicated to finding inhibitors of BACE-1, but the problem is, Alzheimer's disease occurs in human beings. And these compounds do not look to have much chance of doing anything inside any human's body. The best I can say for them is that they might give someone else an insight into something that they might be able to do to make something that might have a better chance of working.
Cranky folks like me would probably refer to the latter of my two new journals as just "J. Med. Chem.", and would refer to the former one by a variety of other easy-to-remember names. I offer this suggestion for free to the scientific publishing community, who will, I'm sure, reciprocate with things of equal value.
So, now that we're in 2010, the journal that introduced the whole idea of a graphical abstract to organic chemistry (Tetrahedron Letters) has finally started including them in their RSS feed. That's how I read journals these days, and I think I have a lot of company, so I'm grateful that they got around to it.
And on another literature note, I wanted to mention that I've accepted an invite to the editorial advisory board of the new ACS journal ACS Medicinal Chemistry Letters. You can tell, I guess, when you've been doing this stuff for a while - when I look at the rest of the advisory board, I see people I went to grad school with, people I used to work down the hall from, and so on. The journal has started publishing its first papers; we'll see how it works out, and how it competes with the other short-form outlet for this sort of work, Bioorganic and Medicinal Chemistry Letters. I promise not to let any anti-Bredt cyclobutenes get past me!
The crew at Angewandte Chemie has produced another head-shaking pun in one of their latest abstracts. Read only if (1) you know your 1980s music, and (2) you have a high tolerance for wordplay. When I was in Germany, this sort of joke was known as an "eiskalter" and was greeted with shivers.
Now here's a strange tale, courtesy of Science magazine, about some retracted work from Peter Schultz's group at Scripps. Twopapers from 2004 detailed how to incorporate glycoslylated amino acids (glucosamine-serine and galactosamine-threonine) directly into proteins. These featured a lot of work from postdoc Zhiwen Zhang (who later was hired by the University of Texas for a faculty position).
But another postdoc, Eric Tippmann, was later having trouble reproducing the work, and in 2006 he made his case for why he thought it was incorrect. Following that:
Schultz says the concerns raised were serious enough that he asked a group of lab members to try to replicate the work in Zhang's Science paper in addition to several other important discoveries Zhang had made. That task, however, was complicated by the fact that Zhang's lab notebooks, describing his experiments in detail, were missing. Schultz says that in the early fall of 2006, the notebooks were in Schultz's office. But at some point after that they were taken without his knowledge and have never resurfaced.
After considerable effort, Schultz says his students were able to replicate most of the work. The biggest exception was the work that served as the basis for the 2004 Science and JACS papers. "It was clear the glycosylated amino acid work could not be reproduced as reported. So we tried to figure out what was going on," Schultz says.
So far, so not-so-good. But here's where things get odd. Around this time (early 2007), Zhang started to get e-mails at Texas saying that unless he send $4000 to an address in San Diego, the writer would expose his "fraud" and cause him to get fired. The messages were signed "Michael Pemulis" - Science doesn't pick up on that pen name, but fans of the late David Foster Wallace will recognize the name of the revengeful practical joker from Infinite Jest.
That brings up another point: the e-mails quoted in the Science article are in somewhat broken English: "you lose job. ... Texas will fire you before you tenure. . ." and that sort of thing. But my belief is that no one who drops the second person possessive while writing would make it far enough into Infinite Jest to meet Micheal Pemulis and use him as an appropriate alias for an extortion plot.
At any rate, after the San Diego police got involved, they told Zhang that they had a suspect, but Zhang decided not to press charges. That fall, though, "Pemulis" dropped the bomb, with a hostile anonymous letter to everyone involved - officials at Scripps and UT-Austin, the editors at Science, etc. In 2009, Zhang was denied tenure. Eric Tippman (now at Cardiff) has published a paper in JBCdetailing the problems with the original work. (He denies having anything to do with the missing lab notebooks or the threats made to Zhang). And everyone involved is still wondering just what is going on. . .
I certainly have no idea. But I can say this: although I've spent a lot more time in industry than in academia, a disproportionate number of the people I've worked with over the years that I consider to have had serious mental problems are still from my academic years. Whoever "Pemulis" is, I'd put him or her into that category. Grad students and post-docs are under a lot of pressure, and some of them are at a point in their lives when their internal problems are starting to seriously affect them.
I missed this paper when it came out back in October: "Reactome Array: Forging a Link Between Metabolome and Genome". I'd like to imagine that it was the ome-heavy title itself that drove me away, but I have to admit that I would have looked it over had I noticed it.
And I probably should have, because the paper has been under steady fire since it came out. It describes a method to metabolically profile a variety of cells though the use of a novel nanoparticle assay. The authors claim to have immobilized 1675 different biomolecules (representing common metabolites and intermediates) in such a way that enzymes recognizing any of them will set off a fluorescent dye signal. It's an ingenious and tricky method - in fact, so tricky that doubts set in quickly about the feasibility of doing it on 1675 widely varying molecular species.
And the chemistry shown in the paper's main scheme looks wonky, too, which is what I wish I'd noticed. Take a look - does it make sense to describe a positively charged nitrogen as a "weakly amine region", whatever that is? Have you ever seen a quaternary aminal quite like that one before? Does that cleavage look as if it would work? What happens to the indane component, anyway? Says the Science magazine blog:
In private chats and online postings, chemists began expressing skepticism about the reactome array as soon as the article describing it was published, noting several significant errors in the initial figure depicting its creation. Some also questioned how a relatively unknown group could have synthesized so many complex compounds. The dismay grew when supplementary online material providing further information on the synthesized compounds wasn’t available as soon as promised. “We failed to put it in on time. The data is quite voluminous,” says co-corresponding author Peter Golyshin of Bangor University in Wales, a microbiologist whose team provided bacterial samples analyzed by Ferrer’s lab.
Science is also coming under fire. “It was stunning no reviewer caught [the errors],” says Kiessling. Ferrer says the paper’s peer reviewers did not raise major questions about the chemical synthesis methods described; the journal’s executive editor, Monica Bradford, acknowledged that none of the paper’s primary reviewers was a synthetic organic chemist. “We do not have evidence of fraud or fabrication. We do have concerns about the inconsistencies and have asked the authors' institutions to try to sort all of this out by examining the original data and lab notes,” she says.
The magazine published an "expression of concern" before the Christmas break, saying that in response to questions the authors had provided synthetic details that "differ substantially" from the ones in the original manuscript. An investigation is underway, and I'll be very interested to see what comes of it.
There's a commentary in the December issue of Nature Chemistry asking why our field has been comparatively slow to adopt web-based technologies like arXiv and GenBank:
"New web-based models of scholarly communication have made a significant impact in some scientific disciplines, but chemistry is not one of them. . .why do similar initiatives in chemistry fail to gain critical mass and widespread usage?"
The article considers several possibilities - among others, that (a) other fields aren't actually quite as techno-webby as we think they are, or (b) there might be a mismatch between chemistry as a discipline and the current tools, one that isn't found in some other fields of science, or (c) that there could be just a few defined issues that need to be addressed, then things will take off, or (d) that chemists already have the communication tools that they need, anyway.
The authors point out that technical hurdles can probably be ruled out as an explanation, and in many cases they can also rule out "because no one's ever tried". Elsevier, for example, tried to get an arXiv-type preprint server going a few years ago, but that bombed pretty thoroughly (not least, I think, because people were naturally a bit suspicious of such an effort being launched under Elsevier's banner, and because the ACS journals refused to take manuscripts that had appeared there). Nature has been trying something similar in the last couple of years with Nature Precedings, but I'm not sure if it's taking off or not. I've never really used it myself, if that's a data point worth mentioning.
One key point that the authors make is that totally new means of communication don't just pop into existence in a scientific discipline. The ones that catch on tend to build on things that the scientists are already doing. I think that physicists, for example, were already more used to sharing preprints of articles, and that the arXiv server just helped them do that more easily. Chemists, on the other hand Just Don't Do That, so announcing to them that Now They Can! isn't enough to bring in participants.
On the same chemistry-is-different front, the commentary also notes that our field has always had an emphasis on making stuff, although they don't put it quite that way. The computer is not usually the machine that produces our results; it's just the means by which we keep track of them. And we don't generate the piles of (sometimes) reusable data that physicists do, so much as we generate new substances and new ways of making and using them. The data are there to show that we did, in fact, make what we said we made. Those piles of data also tend to hold their value much longer than in other fields, too - after all, a compound is a compound, and its NMR spectrum doesn't change. If you want to know how some class of compounds behave, a paper from fifty years ago (or more) can be a perfectly good place to look.
Also in contrast to the physics community, chemistry is broken up into many smaller units. You'll never see a chemistry paper with as many co-authors as a high-energy physics paper, because we don't have to run our experiments on the One Big Machine In the Whole World. It may be that parts of the physics world have basically been forced to collaborate more widely, because that's the only way to get anything done. We also have a wide range of sub-disciplines, what with physics on one side of us and biology on another, and these all have their own idiosyncracies. (And, of course, many of us work in areas where we basically can't share some information until we're good and ready to).
One thing that the whole article doesn't quite address though, is: what would these wonderful new communication modes be, actually? And how would they improve my research life? Electronic literature searching certainly has, as has the availability of journals online. Electronic notebooks definitely have. What else would? I'm sure that there must be a few things, but I find that some of the Web 2.0 info-heaven visions that people outside the field talk about don't do much to excite me. It's like seeing some scientific abstract online, and then noting the little row of social-media icons below it, inviting me to submit the thing to Digg, Reddit, or what have you. Or to go visit the journal's page on Facebook, of all things. Why I'd do that is something I haven't quite figured out yet.
But hey, I'm not as much of a Luddite as that makes me sound. I also note this passage from the article (emphasis mine):
An increasing number of scientists have adopted blogging as a means of informal communication. Typicall, the writing style of blogs is conversational, and humorous content gets mixed with posts of a more serious tone. Some blogs are dedicated to educating lay audiences, others aim at an academic discussion, and many are like personal diaries. At this point in time, many science bloggers are assumed to be less than 30 years old, and are primarily journalists, teachers, graduate students, or young researcher. Hardly any established scientists maintain a blog - after all, blogging regularly is very time-consuming. The question remains open whether these will remain fringe phenomena or become part of the mainstream communication in science.
Readers may remember the incident a couple of years ago where a paper was published claiming the synthesis of some very odd-looking 12-membered ring compounds. Prof. Manfred Christl of the University of Würzburg noticed something odd about this reaction, though, namely that it had already been run over a hundred years ago and was known to give a completely different product. (As I pointed out here, though, you didn't need to unearth the ancient literature to know this; ten minutes of looking through the modern stuff would have done the trick, too).
Well, Christl's back with another takedown of some improperly assigned weirdo 12-membered rings. This time, it's Cheryl Stevenson of Illinois State that gets the treatment, with this paper from last year that claims several interesting ring structures from 1,5-hexadiyne and base. Christl had trouble believing the mechanism, and on closer inspection had trouble believing the NMR assignments. Then, on even closer inspection, he assigned the structure as a simple isomerization of one of the triple bonds, and found that this exact reaction (and product) had first been reported in 1961 (and several times afterwards). Not good.
As it turns out, I almost certainly made some of the compound myself, by mistake, back in mid-1983. That was the summer before I started my first year of grad school, and I was doing work in Ned Porter's lab at Duke. One of the starting materials I needed was. . .1,5, hexadiyne, which you couldn't buy. So I made it, in real grad-school fashion. I homocoupled allyl Grignard to get the 1,5-hexadiene (which even if you could buy back then, we didn't). Then I reacted that with bromine and made the only six-carbon molecule with four bromines on it that I ever hope to make. Reacting that with freshly prepared sodium amide in ammonia gave the smelly di-yne, in crappy yield after distillation. I can still see it: me heating up a column full of glass beads, then turning to Steve, the postdoc in the next hood, and making a bad joke about Herman Hesse while David Bowie's "Modern Love" played on the radio. . .ah, those days, they will not come again.
At any rate, I went on to react the compound with bases under different conditions, trying (in vain) to alkylate both of those terminal alkynes, and thus passed the last of my summer, in exactly the way my two previous summers of research had passed: unsuccessfully. This latest paper, though, makes me think that I was probably turning my starting material instead into exactly the diene that Christl is talking about. I should have hit the library harder myself, although (to be fair) there are references that tell you that you can do that alkylation, and digging through the literature was a good deal more time-consuming back then that it is now.
That lab accident, you say? Well, that happened when I was making a big batch of sodium amide. You start that prep off like a Birch reduction - condense a bunch of liquid ammonia into a flask, and start chucking sodium metal into it. The big difference is that you add a bit of ferric chloride to the mix, which kicks things over at the end. After you've dissolved your sodium, to give you the bronzy purple-blue of solvated electrons, you take the flask out of the cold bath to let the ammonia reflux. At that point, the whole thing suddenly clears, dramatically revealing grey lumps of sodium amide rolling around on the bottom of a pond of plain ol' ammonia, without a solvated electron in sight. (I have, in years since, seen a couple of people refer to the blue stage of the reaction as sodium amide, which it ain't, and I can get quite cranky and pedantic about it).
One afternoon I was whipping up a batch of this stuff, when something starting going on inside the flask. I don't recall what made me take a look at the ammonia solution, but since there was so much bronze gorp on the side of the one-liter three-neck, I had to lean in and look down near the central joint. Whereupon my hair wound itself immediately around the greased shaft of the overhead stirrer, pulling my head in toward the whole setup and jamming my nose into the side of the flask. I fumbled for the switch of the stir motor, feeling like George Jetson as I shouted for someone to give me a hand, and watch with interest as the dry ice bath bubbled along an inch away from my face.
Steve the postdoc came to my aid, shutting off the grinding motor which was doggedly trying to wind me headfirst around the stirrer shaft. We unreeled my hair from the whole contraption, with me cursing foully and Steve merrily making jokes of the "Hair today, gone tomorrow" kind, with side comments about me getting too wrapped up in my work. Those days, as I said, will not come again.
This one gets my Least Informative Abstract award - it crossed my RSS feed recently, and while it may be a very interesting paper, there's no way to tell from this illustration. The text below it, which shows up on the actual ACS abstract page (but not on the RSS feed) is somewhat more informative, and Zewail's name alone is enough to make me take a look at the full paper. But that illustration. . .to think that I was complaining about a bunch of colored dots and the way Nicolaou colors the inside of his aromatic rings. . .ah, those were simpler times.
Everyone has heard about the "Climategate" scandal by now. Someone leaked hundreds of megabytes of information from the University of East Anglia's Climatic Research Unit, and the material (which appears to be authentic) is most interesting. I'm not actually going to comment on the climate-change aspect of all this, though. I have my own opinions, and God knows everyone else has one, too, but what I feel needs to be looked at is the scientific conduct. I'm no climatologist, but I am an experienced working scientist - so, is there a problem here?
I'll give you the short answer: yes. I have to say that there appears to be several, as shown by many troubling features in the documents that have come out. The first one is the apparent attempts to evade the UK's Freedom of Information Act. I don't see how these messages can be interpreted in any other way as an attempt to break the law, and I don't see how they can be defended:
Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise. He's not in at the moment - minor family crisis. Can you also email Gene and get him to do the same? I don't have his new email address. We will be getting Caspar to do likewise.
A second issue is a concerted effort to shape what sorts of papers get into the scientific literature. Again, this does not seem to be a matter of interpretation; such messages as this, this, and this spell out exactly what's going on. You have talk of getting journal editors fired:
This is truly awful. GRL has gone downhill rapidly in recent years.
I think the decline began before Saiers. I have had some unhelpful dealings with him recently with regard to a paper Sarah and I have on glaciers -- it was well received by the referees, and so is in the publication pipeline. However, I got the impression that Saiers was trying to keep it from being published.
Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted. Even this would be difficult.
And of trying to get papers blocked from being referenced:
I can't see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow - even if we have to redefine what the peer-review literature is !
Two questions arise: is this defensible, and does such behavior take place in other scientific disciplines? Personally, I find this sort of thing repugnant. Readers of this site will know that I tend to err on the side of "Publish and be damned", preferring to let the scientific literature sort itself out as ideas are evaluated and experiments are reproduced. I support the idea of peer review, and I don't think that every single crazy idea should be thrown out to waste everyone's time. But I set the "crazy idea" barrier pretty low, myself, remembering that a lot of really big ideas have seemed crazy at first. If a proposal has some connection with reality, and can be tested, I say put it out there, and the more important the consequences, the lower the barrier should be. (The flip side, of course, is that when some oddball idea has been tried and found wanting, its proponents should go away, to return only when they have something sturdier. That part definitely doesn't work as well as it should.)
So this "I won't send my work to a journal that publishes papers that disagree with me" business is, in my view, wrong. The East Anglia people went even farther, though, working to get journal editors and editorial boards changed so that they would be more to their liking, and I think that that's even more wrong. But does this sort of thing go on elsewhere?
It wouldn't surprise me. I hate to say that, and I have to add up front that I've never witnessed anything like this personally, but it still wouldn't surprise me. Scientists often have very easily inflamed egos, and divide into warring camps all too easily. But while it may have happened somewhere else, that does not make it normal (and especially not desirable) scientific behavior. This is not a standard technique by which our sausage is made over here.
What I've seen in organic chemistry are various attempts to steer papers to particular reviewers (or evade other ones). And I've seen people fire off angry letters to journal editors about why some particular paper was published (and why the letter writer's manuscript in response had not been accepted in turn, likely as not). The biggest brawl of them all was still going early in my career (having started before I was born): the fight over the nonclassical norbornyl cation, the very mention of which is still enough to make some older chemists put their hands over their ears and start to hum loudly. That one involved (among many others) two future Nobel Prize winners (H. C. Brown and George Olah), and got very heated indeed - but I still don't recall either one of them trying to get journal editors fired after publishing rival manuscripts. You don't do that sort of thing.
And that brings up an additional problem with all this journal curating: the CRU people have replied to their critics in the past by saying that more of their own studies have been published in the peer-reviewed literature. This is disingenuous when you're working at the same time to shape the peer-reviewed literature into what you think it should look like.
A third issue I want to comment on are the problems with the data and its analysis. I have deep sympathy for the fellow who tried to reconcile the various poorly documented and conflicting data sets and buggy, unannotated code that the CRU has apparently depended on. And I can easily see how this happens. I've been on long-running projects, especially some years ago, where people start to lose track of which numbers came from where (and when), where the underlying raw data are stored, and the history of various assumptions and corrections that were made along the way. That much is normal human behavior. But this goes beyond that.
Those of us who work in the drug industry know that we have to keep track of such things, because we're making decisions that could eventually run into the tens and hundreds of millions of dollars of our own money. And eventually we're going to be reviewed by regulatory agencies that are not staffed with our friends, and who are perfectly capable of telling us that they don't like our numbers and want us to go spend another couple of years (and another fifty or hundred million dollars) generating better ones for them. The regulatory-level lab and manufacturing protocols (GLP and GMP) generate a blizzard of paperwork for just these reasons.
But the stakes for climate research are even higher. The economic decisions involved make drug research programs look like roundoff errors. The data involved have to be very damned good and convincing, given the potential impact on the world economy, through both the possible effects of global warming itself and the effects of trying to ameliorate it. Looking inside the CRU does not make me confident that their data come anywhere close to that standard:
I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!
I do not want the future of the world economy riding on this. And what's more, it appears that the CRU no longer has much of their original raw data. It appears to have been tossed over twenty years ago. What we have left, as far as I can see, is a large data set of partially unknown origin, which has been adjusted by various people over the years in undocumented ways. If this is not the case, I would very much like the CRU to explain why not, and in great detail. And I do not wish to hear from people who wish to pretend that everything's just fine.
The commentator closest to my views is Clive Crook at The Atlantic, whose dismay at all this is unhidden. I'm not hiding mine, either. No matter what you think about climate change, if you respect the scientific endeavor, this is very bad news. Respect has to be earned. And it can be lost.
I asked recently for suggestions on the best books on med-chem topics, and a lot of good ideas came in via the comments and e-mail. Going over the list, the most recommended seem to be the following:
There's a disturbing article out at the New England Journal of Medicine on studies conducted on Neurontin (gabapentin) for various unapproved indications. Parke-Davis (and later Pfizer) looked at a wide range of possible indications for the drug - migraine, neuropathic pain, bipolar disorder, and more. That in itself isn't unusual, since CNS drugs often have rather broad and poorly defined mechanisms, and it's not like we understand any of them all that well.
What is unusual is the pattern found when comparing the internal reports with the published versions that showed up in the literature. The authors found that:
"More than half the clinical trials that we included in our analysis (11 of 20) were not published as full-length research articles. For 7 of the 9 trials that were published as full-length research articles, a statistically significant primary outcome was reported, and for more than half these trials, the outcome specified in the published report differed from the outcome originally described in the protocol. Three of the four trials with an unchanged primary outcome had statistically significant results for the protocol-specified primary outcome. Secondary outcomes also frequently differed between the protocol and the published report. Thus, trials with findings that were not statistically significant (P≥0.05) for the protocol-defined primary outcome, according to the internal documents, either were not published in full or were published with a changed primary outcome. . .all the changes that took place between what was specified in the protocol, what was known before publication (as presented in the internal company research reports), and what was reported to the public led to a more favorable presentation in the medical literature. . ."
The authors go on to point out that changing a primary outcome after you see the data is, in fact, a statistical sin (although that's not quite the phrase they use!) You really can't go around doing that, because you can end up chasing after random chance (and avoiding that is the whole point of running well-controlled trials). This does not cover Pfizer and Parke-Davis with glory, but it's worth noting that there's plenty of blame to go around when it comes to this practice:
"Our study is based on a relatively small number of trials undertaken to test a single drug manufactured by a single company and its successors. Furthermore, if a major purpose of the studies we examined was to promote off-label uses of gabapentin, the selective reporting we observed could be more extreme than that observed for studies conducted for other reasons. Previous studies in different settings have shown evidence of these same biases, however. Indeed, selective outcome reporting does not appear to be limited to studies funded by drug companies. Chan and colleagues examined published trials funded by the Canadian Institutes of Health Research and found that 40% of stated primary outcomes differed between the protocol and the published report. In addition, we cannot be certain that selective reporting was a decision made by employees of Pfizer and Parke-Davis, since the authors of the published reports included nonemployees. We did not systematically assess the methodologic quality of the included trials as described in the publications we examined. Previous research has indicated that quality scores are higher for trials conducted by the pharmaceutical industry than for trials conducted by not-for-profit entities, although reports from industry-sponsored trials have potentially distorted the scientific record because of other, less easily measured study factors."
That doesn't get the folks who conducted these gabapentin studies off the hook, although I should note that Pfizer disputes the conclusions of this article (as you'd certainly think that they would). And it's also worth noting that some of its authors have done work for the plaintiffs in suits against Pfizer over gabapentin (thus all the familiarity with the internal company documents, which came to light during discovery proceedings). But again, I don't see how that negates the paper's conclusions, and if Pfizer has any hard data that would do so, I think they should produce it with all speed.
And no, it's just a coincidence that this post involve Pfizer, after I've been going on about their merger business all week. Unfortunately, I think that they're probably not the only company that could be pointed at. But we in the industry shouldn't have things like this for others to uncover in the first place. Should we?
You're supposed to disclose conflicts of interest if you're the author of a scientific paper. For the most part, everyone does, but it's those times that the system breaks down that cause all the trouble. Does this author actually earn a side income from Company X? Is that author actually about to start a new company based on the discovery that's being reported so breathlessly? And does this other author have a big stock position in company Y, whose price will be affected by this new paper? Journal editors want to know about these things, as do readers.
But how far do we go with this idea? An editorial in BioCentury (free PDF version) takes up arms against a new rule for non-financial disclosures from the International Committee of Medical Journal Editors. It requires authors to "report any personal, professional, political, institutional, religious, or other associations that a reasonable reader would want to know about in relation to the submitted work". And it's the inclusion of the words "personal", "political", and "religious" that could cause trouble.
Or maybe it's the inclusion of the word "reasonable". That's a common legal-argument adjective, but on the whole, people are not reasonable when it comes to their political or religious beliefs. (You may have noticed that there's been a debate for several centuries now about whether religious belief has anything to do with reason at all, but I think we'll try to stay out of that one). A dedicated atheist may consider it quite reasonable to want, say, any biological publication featuring Francis Collins of NIH to always feature a statement that Collins is a born-again Christian with a strong interest in reconciling his beliefs with scientific practice. An evangelical Christian reader, on the other hand, may want to have the biology papers flagged for the authors who do not see the hand of a Creator in their field of study. Which of these is "reasonable", if either?
The situation doesn't get any easier when you move towards politics. Do we really want to start listing party affiliations or the like? I realize that the journal editors have no intention of doing any such thing, but no one ever intends for the worms to get so far out of the can, either. When a really contentious issue comes up (such as global warming), plenty of reasonable readers (or perhaps I mean readers who are otherwise reasonable!) would want to see the complete political disclosure done on the authors of every paper, the better to sniff out Error, Self-Interest, and Collusion from either side of the debate.
How are we going to draw these particular lines, and how are we going to draw them in any kind of consistent fashion? Consistency is going to be very hard to achieve. The BioCentury piece points out a recent major disclosure glitch by the editors of the New England Journal of Medicine, and if we go into the full empty-out-your-pockets mode, I worry that the arguments may never cease.
And I've even made it to the last paragraph without mentioning the libertarian none-of-your-business objections to the whole idea. Your thoughts?
Back in late September I wrote about a controversial paper in the Proceedings of the National Academy of Sciences. It attracted comment for its way-out-there hypothesis: that caterpillars and other larvae arose through a spectacular interspecies gene transfer rather than through conventional evolutionary processes. And it may have been the last paper to make it into the journal by the now-eliminated "Track III" route, which allowed members to essentially cherry-pick their own reviewers. This paper may well have hastened the disappearance of that system, actually - it created quite an uproar.
At the time, I wrote that the paper's hypothesis seemed very likely to be wrong, but at least the author had proposed some means to test it. Now in the latest PNAS come a letter and a full article on the subject. Both mention the testability of the original paper, and go on to point out that such tests have already been done. The paper is written in a tone of exasperation:
Williamson suggested that "many corollaries of my hypothesis are testable." We agree and note that most of the tests have already been carried out, the results of which are readily available in the recent literature and online databases. Here, we set aside (i) the complete absence of evidence offered by Williamson in support of his hypothesis, (ii) his apparent determination to ignore the enormous errors in current understanding of inheritance, gene expression, cell fate specification, morphogenesis, and other phenomena that are implied by his hypothesis, and (iii) the abundant empirical evidence for the evolution and loss of larval forms by natural selection. Instead, we focus on Williamson's molecular genetic predictions concerning genome size and content in insects, velvet worms, and several marine taxa, and we point out the readily available data that show those predictions to be easily rejected.
And you know, they really should set aside those first three points. Entertaining as it is to read this sort of thing, the real way to demolish a paper like Williamson's is to rip it up scientifically, rather than hurl insults at it (however well-justified they might be). There seems to be plenty of room to work in. For example, Williamson predicts that a class of parasitic barnacle will be found to not be barnacles at all, and to have an abnormally large genome, with material from three different sorts of organisms. Actually, though, these organisms have smaller genomes than usual, and from their genes they appear to be perfectly reasonable relatives of other barnacles.
And so on. Williamson predicts that the genomes of insects with caterpillar-like larval stages will tend to be larger than those without, but the data indicate, if anything, a trend in the opposite direction. His predictions for specific insects don't pan out, nor do his predictions about the genome size of velvet worms and many other cases. If I read the paper right, not one of Williamson's many predictions actually goes his way. In some cases, he appears to cite genome size data that line up with his hypothesis, but miss citing similar organisms that contradict it.
So that would appear to be that. Indeed, as the authors of the latest PNAS paper mention, one might have thought so years ago, since these very authors have shot down some of Williamson's work before. That's the real problem here. I have a lot of sympathy for people who are willing to be spectacularly wrong, but that starts to evaporate when they don't realize that they've been spectacularly wrong. Williamson appears to have had a fair hearing for his ideas, and as far as I can tell, they've come up well short. And while we need brave renegades, cranks are already in long supply.
I've been meaning for some time to acknowledge whoever it is at Angewandte Chemie that works in so many odd musical references in the abstracts. There are the usual runs of weak puns - and don't you wish that Nature, among other journals, would consider how unintelligible those jokes are as headlines in their RSS feeds? Lukeward wordplay is the standard of wit for most scientific prose, and I've been guilty of it myself. (I have to say, though, this one rises above the pack, this one is fairly hard to take, and this one and this one definitely cross the pain threshold).
But I wonder how many readers have noted recent references to Mike Oldfield, Ace of Base, Offspring, and even the Sex Pistols? Have the editors noticed, for that matter? (This joke suggests a speaker of American English is at work, since I doubt most Germans have heard of the American Automobile Association).
Note: this has been going on for some time. Carbon-Based Curiosities adduces some other examples from about a year ago, including ricochet shots off "My Sharona", Chic, Jimi Hendrix, the Terminator movies, and the X-Files. . .
There seems to be some finger-pointing going on about conflicts of interest in the scientific and medical literature. According to this piece in Nature Medicine, a recent conference in Vancouver on peer review featured statements such as this:
"We absolutely should not let up on our scrutiny of industry," says Karen Woolley, a co-author of one of the new studies and chief executive officer of the professional medical writing company ProScribe, based in Queensland, Australia. "But why are we always pointing our finger over there? There's an elephant in the room, and that's the nonfinancial conflicts of interest in academia."
I hope that ProScribe wasn't involved in that Australian journal scandal. But even though the head of a medical writing company clearly has a gigantic axe to grind here, the point isn't invalid. Academia has pressures of its own to publish, and lot of shaky stuff gets sent out under them.
Under the auspices of (the Council on Publishing Ethics), (consultant Liz) Wager dug through PubMed files to see how many papers had been retracted between 1988 and 2008. She found 529, and, in a close study of a randomly selected set of 312, she judged that only 28% were due to "honest error". Among the rest, some of the largest chunks were due to authors found publishing the same results more than once (18%), plagiarism (15%), fabrication (5%) and falsification (4%) of data. Taking into account an additional 1% in the 'other misconduct' category, the unethical reasons stacked up to 43%.
Many, perhaps most, of these papers seemed to have been unlikely to have been funded by industry. And there are, of course, plenty of rotten papers out there that never get retracted at all, in many cases because no one reads them or notices that they're a rehash of what someone else has already published. The Deja Vu people are starting to cut into that pile, though, and it's a big one.
There's a danger of all this turning into an exchange of tu quoque arguments between industry, academia, and the publishers. I think there's common ground to agree, though, that all sorts of pressures exist to publish work that shouldn't be published, and that everyone has a common interest in making sure that this doesn't happen. And industry still has a bigger responsibility, since (1) it has more money to cause trouble, if it wants to, and (2) the sorts of things it works on often have more immediate relevance to the outside world. If some obscure faculty member somewhere published reheated work in a series of low-end journals, he's only wasting the time of a limited number of people. A publication involving clinical trial data, though, can send ripples out a lot farther and faster.
There's a letter in the latest Nature from two researchers in Halifax that makes a point which isn't made often enough. Why do so many papers in the literature ignore patent references?
Why are patent citations so conspicuously absent across academic journals, with most even omitting formatting instructions for these in their author guidelines? Patents present novel, rigorously reviewed unpublished work, as well as providing an unmatched resource for detail.
We randomly selected one month (December 2008) and reviewed all citations in the reviews, articles and letters/reports in Nature (1,773 citations) and Science (1,367). These citations included textbooks, http://arXiv.org preprints and abstracts — but no patents.
They go on to point out that searching the patent literature, which traditionally was rather a pain, is much easier now, as is access to the patents themselves. And they have a point. When I was in graduate school in the 1980s, getting a procedure out of a patent was really considered an absolute last resort - it was a special order in the library, and you had this vague feeling that there was some sort of trickiness going on, that none of those syntheses were ever actually supposed to work, anyway.
Not so. While the patent literature is indeed full of junk, the open literature is, too. They're not exactly peer-reviewed, true - but journal papers have a much lower chance of having to stand up in court, so things sort of even out. And as far as organic synthesis is concerned, patents are full of real procedures to make real things (and often enough, with real spectral data to support the claims). Most of the compounds I've made in my career that have seen any light of day at all have done so in patents, and they're real as can be.
I've complained several times when refereeing papers for publication about the lack of relevant patent citations in them. And I'd advise others to do the same - this branch of the scientific literature deserves its due.
I've written before here about how I actually like reading the Proceedings of the National Academy of Science (PNAS). It's a journal that has published a lot of groundbreaking work, and a fair amount of nonsense, and that mix is largely due to its unusual paths to publication.
Well, one of those paths is drying up. The academy has decided to stop the "communicated by" option (Track III), where someone can approach a PNAS member and ask them to send in a manuscript (each member could do this up to twice a year). Some members seem to mourn the passing of an old tradition, while others are glad that they don't have to pick and choose between manuscripts from their friends. Science has some details, and you can see the PNAS announcement here.
One of the things that may have either sped this along, or at least made people think about the decision more, was a recent paper by Donald Williamson, communicated by Lynn Margulis. Williamson presents an evolutionary hypothesis that is controversial to say the least, the idea that larvae (caterpillars, etc.) are the result of a wholesale gene transfer between completely different phyla. I think that this idea is very likely to be wrong, but in Williamson's defense, he proposes some ways to test it - and if by some chance he's right, he'll rewrite a big chunk of evolutionary theory.
Some people may look at the latest PNAS move and think "Good, now we won't have any more craziness like that caterpillar stuff". But I actually like to see a bit of such craziness, and I worry that there are already too few outlets for it to see publication. It may not have been an appropriate paper for PNAS - but where else would it have been published at all?
Maybe I'm just cranky, or cranky on the subject of design and presentation, but this sort of chemical structure drawing rubs me the wrong way. I'd have a much better chance of understanding the transformation without all the colored dots; they don't seem to be adding anything.
And while I'm in get-off-my-lawn mode, the similar trendlet of coloring the interiors of ring structures annoys me, too. Nicolaou seems to enjoy doing this (here's a recent example), and I can't say that it adds much to my understanding.
I was looking through my RSS feed of journal articles this morning, and came across this new one in J. Med. Chem.. Now, there's nothing particularly unusual about this work. The authors are exploring a particular subtype of serotonin receptor (5-HT6), using some chemotypes that have been looked at in serotinergic ligands before. They switch the indole to an indene, put in a sulfonamide, change the aminoethyl side chain to a guanidine, and. . .wait a minute.
Guanidine? I thought that the whole point of making a 5-HT6 ligand was to get it into the brain, and guanidines don't have the best reputation for allowing you to do that. (They're not the easiest thing in the world to even get decent oral absorption from, either, come to think of it). So I looked through the paper to see if there were any in vivo numbers, and as far as I can see, there aren't.
Now, that's not necessarily the fault of the paper's authors. They're from an academic med-chem lab in Barcelona, and animal dosing (and animal PK measurements) aren't necessarily easy to get unless you have a dedicated team that does such things. But, still. The industrial medicinal chemist in me looks at these structures, finds them unlikely to ever reach their intended site of action, can find no evidence in the paper's references that anyone else has ever gotten such a guanidine hydrazone into the brain, either, and starts to have if-a-tree-falls-in-the-forest thoughts.
Now, it's true that we learn some more about the receptor itself by finding new ligands for it, and such compounds can be used for in vitro experiments. But it's not like there aren't other 5-HT6 antagonists out there, in several different chemical classes, and that's just from the first page of a PubMed search. Many of these compounds do, in fact, penetrate the brain, because they were developed by industrial groups for whom in vitro experiments are most definitely not an end in themselves.
I don't mean to single out the Barcelona group here. Their work isn't bad, and it looks perfectly reasonable to me. It's just that my years in industry have made me always ask what a particular paper tells me that I didn't know, and what use might some day be made of the results. Readers here will know that I have a weakness for out-there ideas and technologies, so it's not like I have to see an immediate practical application for everything. But I would like to see the hope of one. And for this work, and for a lot of medicinal chemistry that comes out of academic labs, I just don't see it.
Update: it's been pointed out in the comments that there's a value in academic work that doesn't have to be addressed in industry, that is, training the students who do it. That's absolutely right. But at the same time, couldn't people be trained just as well by working on systems that are a bit less dead on arrival?
And no, I'm not trying to make that case that academic labs should make drugs. If they want to try, then come on down. If they don't, that's fine, too - there's a lot of important research to be done in the world that has no immediate practical application. But this sort of paper that I've written about today seems to miss both of these boats simultaneously: it isn't likely to produce a drug, and it doesn't seem to be addressing any other pressing needs that I can see, either.
And yes, I could say the same about my own PhD work. "The world doesn't need another synthesis of a macrolide antibiotic", I told people at the time. "But I do". Does it have to be like that?
Here's an interesting paper that some of you may have seen in J. Med. Chem.: "Heteroaromatic Rings of the Future". That's an odd title, but an appropriate one.
For the non-chemists in the crowd who made it to this paragraph, heteroaromatic rings are a very wide class of organic compounds. They're flat cyclic structures with one or more nitrogen, oxygen, or sulfur atoms in the ring - I'll leave out explaining the concept of "aromaticity" for now, but suffice it to say that it makes them flat and gives them some other distinct properties. These structures are especially important in medicinal chemistry. If you stripped out all the drugs that contain something from this class, you'd lose a bit under half of the current pharmacopoeia, and that share has lately been increasing.
The authors have sat down and attempted to work out computationally all the possible heteroaromatic systems. If you include a carbonyl group as a component of the ring, you get 23,895 different scaffolds (and only 2986 if you leave the carbonyl out of it). Their methods to define and predict that adjective "possible" are extensive and worth reading if you're curious; they did put a lot of effort into that question, and their assumptions seem realistic to me. (For example, right off, they only considered mono- and bicyclic systems, 5- and 6-membered only, C, H, N, O and S).
At any rate, only 1701 of those 23,985 have ever been reported in the literature. And it looks as if reports of new ring systems reached a peak in the late 1970s, and have either dropped off or (at the very least) never exceeded those heights since then. The authors estimate that perhaps 3,000 of their list are synthetically feasible, with a few hundred of them being notably more likely than the rest. Their paper, in fact, seems to be a brief to alter that publication trend by explicitly pointing out unexplored synthetic territory. It wouldn't surprise me if they go back in a few years to see if they were able to cause an inflection point.
I hope they do. I'm a great believer in the idea that we medicinal chemists need all the help we can get, and if there are reasonable ring systems out there that we're not exploiting, then we should get to them. Adventurous chemists should have a look.
As much as I defend the industry I work in, I have to talk about things that we do that I don't think are so defensible. Another one of those has come up thanks to the New York Times and PLoS Medicine, who obtained a pile of records from a current court case.
This article has the details. Wyeth seems to have contracted with a medical writing outfit (DesignWrite) to produce and place a number of review articles covering hormone therapy for menopausal women. (Wyeth, of course, was the main player in that market). The articles seem to have been entirely written by the staff at DesignWrite - authors are listed as "TBD", and then academics were recruited to serve as lead authors and to submit the papers to journals.
No mention was ever made in the published papers of the medical writing group's role, nor of Wyeth's (who were paying them for this service). As far as the readers could see, these were the standard sorts of review articles that show up in the medical literature all the time. And that's the part that bothers me. For all I know, these articles were reasonable reviews of the field - I'm no great expert in the field, so I can't judge if they're truly fair summaries. But even if they are, the readership of a journal is entitled to know that a drug company was the impetus behind them, and they're also most certainly entitled to know the actual authors (as opposed to the people who would appear to have been the authors, but just signed off on the stuff).
I think that drug companies are entitled to promote their products. But full disclosure should be the the standard to try to reach in any market: put it all out on the table, and let physicians make their own decisions. It doesn't help, not one bit, to get papers into the journals this way - because when a company goes to such lengths to hide its participation, it almost looks as if it has something to hide. . .
Someone's leaked an American Chemical Society memo to Nature, in which the VP of the publishing division talks about how the printed journals are going to be phased out. The ACS isn't confirming anything, but they're not denying it, either: it looks like the days of paper copies of their journals are numbered.
I've been expecting that. I used to have a print subscription to the Journal of Organic Chemistry back in the early and mid-1990s, and I took them with me in a move in 1997. I interrupted my subscription around that time, and never got around to renewing it. By then, online access was starting to become a more convenient way to locate old articles, and as the ACS improved their archives the advantages became overwhelming. Then I got used to following the new issues online, either by going to the journal's site or by RSS feeds.
So my boxed collection of several years of JOC sat in my basement, in bales of cobalt-blue-covered bricks of paper. I'd planned on moving them into my office, but didn't got around to it at first. That delay allowed the situation to turn into "Hmmm. . .not sure that I see the need to have these taking up the shelf space", which turned into "You know, I need to recycle these things". And gradually, that's just what I did.
When I joined the Wonder Drug Factory in '97, new print journals were still put out on a table in the library as they came in, for people to sit down and read. A few years later, the table was gone, and whole idea was sounding downright Victorian in retrospect. The company where I work now doesn't even have much of a real, printed-on-paper chemistry library at all. It's been years I last picked up a hard copy of any chemistry journal - when I see the cover illustration of a journal on its web site, I keep thinking of "Elegy Written in a Country Churchyard": Full many a flower is born to blush unseen / And waste its sweetness on the desert air. OK, I'm perhaps a bit weird in that respect. But you get the idea.
Printed copies of journals have some advantages. I used to read JOC in the laundromat when I lived in New Jersey, which kept the casual chit-chat down to a stark minimum, I can tell you. I think that the browsing effect of looking through a hard copy is only partially emulated by scrolling through an RSS feed - the old way, you could see all the details inside a paper as you flipped through, and often learned something. So in a way, I'll miss the bound versions. But then I think of those boxes in my basement, and I realize that there's really no other way.
Via a reader comes this article, which takes us to Elsevier's hard-hitting textbook publishing operation. The co-authors of a psychology text for the publisher were recently taken aback to get this e-mail from a publicist at the company:
""Congratulations and thank you for your contribution to Clinical Psychology. Now that the book is published, we need your help to get some 5 star reviews posted to both Amazon and Barnes & Noble to help support and promote it. As you know, these online reviews are extremely persuasive when customers are considering a purchase. For your time, we would like to compensate you with a copy of the book under review as well as a $25 Amazon gift card. If you have colleagues or students who would be willing to post positive reviews, please feel free to forward this e-mail to them to participate. We share the common goal of wanting Clinical Psychology to sell and succeed. The tactics defined above have proven to dramatically increase exposure and boost sales. I hope we can work together to make a strong and profitable impact through our online bookselling channels."
George Tremblay of Antioch U. blew the whistle on this one, which is a good deed. But, cynical person that I am, it makes me wonder how many others on the list might have been ready to pitch in. And given that this has apparently been done before (hey, this is a "proven" strategy), you also have to wonder about five-star reviews of other textbooks published by Elsevier. And other houses, too?
I ask because the company's director of public relations has come out to explain just where this latest tactic went too far - and I have to say, it's a bit further along the line than many people might have thought:
"Encouraging interested parties to post book reviews isn't outside the norm in scholarly publishing, nor is it wrong to offer to nominally compensate people for their time, some of these books are quite large," he said. "But in all instances the request should be unbiased, with no incentives for a positive review, and that's where this particular e-mail went too far."
So when you're encouraging people to write reviews, and offering them some baksheesh for doing so, that's fine. You just don't want to be so gauche as to actually come out and say that you want the reviews to be positive. This does not make Elsevier look good, of course, coming as it does after the reheated-tray-of-friendly-leftovers journal scandal in Australia. (And let's not forget the, um, unusualcase of El Naschie and his private Elsevier journal of nonsense). They either are the poor victims of widely scattered unethical promotions staff, or (just perhaps) there's a general culture in that department that allows people to think that these things are acceptable practice.
I spoke here about Scigen, the program that'll concoct a load of total nonsense for you and make it look - from a distance - like a journal paper. It's a surprisingly valuable tool, since the scientific publishing world apparently has a bigger demand for total nonsense than you might think, especially after the checks clear.
The latest example of this comes from The Scholarly Kitchen, where Philip Davis generated "Deconstructing Access Points", a paper that's nothing but a string of gibberish and non sequitars from first to last. It's here (in PDF form) if you want to try reading it. You won't get far; no human could.
Ah, but what if no human bothered to? That's what happened when Davis submitted this compost pile to the Open Information Science Journal, which is one of the new Bentham "open access" journals. You see, Bentham (like some other publishing houses) has heard that this open access stuff is like, the new trend, so they've started a line of their own journals. Once your paper's accepted, anyone can access it. Of course, there is a fee up front - to be fair, there pretty much has to be, if someone is actually going to do the back-end reviewing and editing work of a real journal. But what if you don't do any of that, and just charge the fee anyway?
Yes, the paper was accepted - of course it was accepted. It was accepted despite it being an unreadable mass of pseudo-English, and despite the fact that it was sent in under the banner of the Center for Research in Applied Phrenology. (Nice touch!) Here's the acceptance letter from an assistant manager at Bentham. All Davis had to do was send $800 to a tax-free zone in the United Arab Emirates and this manuscript would be inflicted on the world.
He pulled back at this juncture, but the point had been made. As he puts it, in milder tones than I would have: ". . .it does raise the question of whether, at least in some cases, the producer-pays-to-publish model may unduly influence editorial decision-making." Indeed it does, especially with a lower-tier publisher. Too much of the scholarly publishing world is involved in this sort of thing (and too much of the conference-organizing world, too, for that matter). I know that it's hard for many people to realize this, but it really is better not to publish at all than to abet this sort of thing.
I missed this a couple of months ago, but there was a paper withdrawn from the Journal of Organic Chemistry. The original is here, a contribution from the Indian Institute of Chemical Technology in Hyderabad on 2-aryl benzothiazoles.
This manuscript was withdrawn from publication by the Editor-in-Chief of The Journal of Organic Chemistry. The basis for the withdrawal was a violation of the Ethical Guidelines to Publication of Chemical Research of the American Chemical Society. . .
The reason given is plagiarism from a paper in Angewandte Chemie in 2008, which is from Carsten Bolm's lab in Aachen on S-arylation of thiols. And here we find the trouble. Below are two sections - the first from the JOC paper, and the second from the original Ang. Chem.:
Among the various intramolecular reactions, S-arylation is comparatively less studied.(14) Two factors make this process difficult: First, thiols are prone to undergo oxidative S−S coupling reactions which result in the undesired formation of disulfides, and second, organic sulfur compounds can be effective metal binders, which leads to catalytic modification (or deactivation).(15) However, given the prevalence of C−S bonds in a wide range of pharmaceutically active compounds and polymeric materials,(16) it is desirable to find novel procedures that provide efficient access to such highly useful organic products.
Among the various cross-coupling types, S-arylation is comparatively less studied.[3] Two factors make this process difficult: First, thiols are prone to undergo oxidative SS coupling reactions, which result in the undesired formation of disulfides, and second, organic sulfur compounds can be effective metal binders, which leads to catalyst modification (or deactivation).[4] However, given the prevalence of CS bonds in a wide range of pharmaceutically active compounds and polymeric materials,[5] it is desirable to find novel catalytic procedures that provide efficient access to such highly useful organic products.
There's no doubt that this is a copy-and-paste job. And I believe that the ACS policy cited doesn't leave much wiggle room - if you do this, you get slapped down. What's silly about it is that it didn't have to happen. People borrow such background material all the time, to greater or lesser extents. But word for word? Bad idea. Frankly, if the Hyderabad authors had spent twenty minutes rewriting those sentences, no one would have ever noticed a thing. The automated similarity searches that can be done now (which I presume led to this incident) would have passed right over.
But (as far as I know) the conclusions of the JOC paper are still valid. And if you care about 2-arylbenzothiazoles, you might even want to see them. I note that the paper is still on the JOC web site, even though it's been "withdrawn". Is this the middle ground, then, a way to discipline people without yanking the results completely from the literature?
I've been meaning to write about the latest advance in salesmanship, pioneered by Merck and Elsevier. As most of you will have heard, the two collaborated to produce something called "The Australasian Journal of Bone and Joint Medicine". This appears to have looked like a real journal, complete with the Elsevier logo and a board of review editors, but it apparently featured nothing but articles (complimentary article, needless to say) about Merck products.
Update: It appears that Merck and Elsevier actually set up a whole publishing division, Excerpta Medica, to handle these things. More here and many more details here.
The news broke about a month ago in The Australian, and the story has been rolling downhill ever since, getting larger all the way. Now Elsevier has issued a public apology for their part in the whole affair, as well they should.
As Orac points out, there are a lot of "throwaway" journals out there, particularly in the medical field. These are sort of once-over-lightly review journals, condensing the literature down into short reads. And that's not all bad, although you wouldn't want a physician to be getting all his or her news that way. But this latest venture was designed to look like a real journal, and was, in fact, full of real articles which had been reprinted from other Elsevier journals. That's well over the line.
I'm not sure who to be more mad at here: Merck or Elsevier. This one really looks like a team effort. If Merck wants to assemble a bunch of previously peer-reviewed studies and put them out under some banner to show how wonderful their drugs were, well, that's fine by me. But that banner shouldn't be something that's deliberately designed to look like a peer-reviewed journal itself. And the collection should have a disclaimer on the cover that it's being paid for by Merck, and the first page of every article should have another box: "As originally reported in (journal citation) - brought to you as a service by Merck". I wouldn't have a problem with that at all.
But that (completely above-board) style seems to be just what the company wanted to avoid, and they got Elsevier, a large and (apparently spottily) respectable scientific publisher to say "Yes, indeed!". Merck's marketing people should be ashamed of themselves, but they should be ashamed for doing what they're paid to do too vigorously. Elsevier, on the other hand, shouldn't be doing this sort of thing at all.
What a mess!Science has a retraction of a 2005 paper, which is always a nasty enough business, but in this case, the authors can’t agree on whether it should be retracted or not. And no one seems to be able to agree on whether the original results were real, and (even if they weren’t) whether the technique the paper describes works anyway. Well.
The original paper (free full text), from two Korean research groups, described a drug target discovery technique with the acronym MAGIC (MAGnetism-based Interaction Capture). It’s a fairly straightforward idea in principle: coat a magnetic nanoparticle with a molecule whose target(s) you’re trying to identify. Now take cell lines whose proteins have had various fluorescent tags put on them, and get the nanoparticles into them. If you then apply a strong magnetic field to the cells, the magnetic particles will be pulled around, and they’ll drag along whichever proteins have associated with your bait molecule. Watch the process under a microscope, and see which fluorescent spots move in which cells.
Papers were published (in both Science and Nature Chemical Biology), patent applications were filed (well, not in that order!), startup money was raised for a company to be called CGK. . .and then troubles began. Word was that the technique wasn’t reproducible. One of the authors (Yong-Weon Yi) asked that his name be removed from the publications, which was rather problematic of him, considering that he was also an inventor on the patent application. Early last year, investigations by the Korean Advanced Institute of Science and Technology came to the disturbing conclusion that the papers “do not contain any scientific truth”, and the journals flagged them.
The Nature Chemical Biologypaper was retracted last July, but the Science paper has been a real rugby scrum, as the journal details here. The editorial staff seems to have been unable to reach one of the authors (Neoncheol Jung), and they still don’t know where he is. That’s disconcerting, since he’s still listed as the founding CEO of CGK. A complex legal struggle has erupted between the company and the KAIST about who has commercial rights to the technology, which surely isn’t being helped along by the fact that everyone is disagreeing about whether it works at all, or ever has. Science says that they’ve received parts of the KAIST report, which states that the authors couldn’t produce any notebooks or original data to support any of the experiments in the paper. This is Most Ungood, of course, and on top of that, two of the authors also appear to have stated that the key experiments (where they moved the fluorescent proteins around) were not carried out as the paper says. Meanwhile, everyone involved is now suing everyone else back in Korea for fraud, for defamation, and who knows. The target date for all this to be resolved is somewhere around the crack of doom.
Emerging from the fiery crater, CGK came up with another (very closely related) technique, which they published late last year in JACS. (If nothing else, everyone involved is certainly getting their work into an impressive list of journals. If only the papers wouldn’t keep sliding right back out. . .) That one has stood up so far, but it’s only April. I presume that the editorial staff at JACS asked for all kinds of data in support, but (as this whole affair shows) you can’t necessarily assume that everyone’s doing the job they’re supposed to do.
The new paper, most interestingly, does not reference the previous work at all, which I suppose makes sense on one level. But if you just came across it de novo, you wouldn't realize that people (at the same company!) had already been (supposedly) working on magnetic particle assays in living cells. Looking over this one and comparing it to the original Science paper, one of the biggest differences seems to be how the magnetic particles are made to expose themselves to the cytoplasm. The earlier work mentioned coating the particles with a fusogenic protein (TAT-HA2) that was claimed to help with this process; that step is nowhere to be found in the JACS work. Otherwise, the process looks pretty much identical to me.
Let’s come up for air, then, and ask how well useful these ideas could be, stipulating (deep breath) that they work. Clearly, there’s some utility here. But I have to wonder how useful this protocol will be for general target fishing expeditions. Fluorescent labeling of proteins is indeed one of the wonders of the world (and was the subject of a recent a well-deserved Nobel prize). But not all proteins can be labeled without disturbing their function – and if you don’t know what the protein’s up to in the first place, you’re never sure if you’ve done something to perturb it when you add the glowing parts. There are also a lot of proteins, of course, to put it mildly, and if you don’t have any idea of where to start looking for targets, you still have a major amount of work to do. The cleanest use I can think of for these experiments is verifying (or ruling out) hypotheses for individual proteins.
But that's if it works. And at this point, who knows? I'll be very interested to follow this story, and to see if anyone else picks up this technique and gets it to work. Who's brave enough?
A minor theme around here recently has been bogus publications and bad conferences. Interestingly, I’ve had three invitations within the past few weeks to speak at meetings in China – once on antivirals, once on antibodies, and most recently on molecular diagnostics. Perhaps it’s my publication record in those areas that recommended me? Nope. I have no publication record in any of those fields. Maybe it’s because of this blog? That’s a lousy reason to invite someone to give a technical presentation, but hey, it’s better than no reason at all, right? Nope, not that either. The letters are make no mention of anything other than the fact that (on behalf of the organizing committee, natch) I’m invited to give a speech.
So let’s go with “no reason at all”, other than (1) I can breathe, (2) I can speak, (3) I can thus fill up a slot in the presentation schedule, and (4) I can presumably pay the conference registration fee. The invitations themselves all come from different contact people, with varying skills in English. The most recent one addresses me as “Dear Dr. Derek”, for example, and goes on through a tangle of subject-verb disagreements and dropped definite articles to invite me to “keep the moment going”. Now, I don’t object to bad grammar on the part of a non-native speaker (the letter’s English is a hell of a lot better than my Chinese), but then again, I’m not organizing a conference by sending out e-mails in the language, either. For which I’m grateful.
But all of the invites are from the same organization, BIT Life Sciences. These folks seem to have a real conference empire going over there, at least for the past two or three years. They have seven big ones scheduled for this year - perhaps I'll eventually be invited to present at all of them. (I wonder if anyone will make the whole circuit like that - do you get a special T-shirt or something?) I note that one of BIT's other sidelines is "Nobel Across China", where they invite various life-science Nobel winners to come and give presentations. I quote from BIT:
"As we all know, Nobel Prize is reputed as an international authorized prize throughout the world, which is a scientific reward for Technology Innovation & Creations. It is designed to bring creative minds as well as exploratory spirits here rightly and stimulate domestic scientists and young generations to pursue new exploration and creation in Life Sciences with surpassing spirits for the crest of Science & Technology."
Well, that's fine. I know, I know: I really shouldn't be making fun of the way that this is phrased. But it's a mystery to me why they don't run these things past a native English speaker. (I notice that James Fallows, a man with plenty of China experience, is baffled by the same thing). And for all I know, it's done some good for people in the audience to attend these presentations. My hopes for BIT's other conferences aren't as high, though, not if they're having to e-mail people like me out of the blue to fill out the program.
Has anyone out there had dealings with BIT, or attended one of their meetings? Are they mostly a chance to make some business contacts in China, or what? First hand experiences welcome. . .
And on the subject of odd Chinese conferences, I'm still looking around for a link to an article I read months ago - I believe on a link from James Fallows, although I can't track it down. A group of expatriates were recruited to attend a steel industry conference in a Chinese city, even though they knew nothing about the field. Bus transportation and free food were provided, along with fake business cards, so long as they spent the day wandering around giving the exhibition an international look. . .
A comment here the other day mentioned Scigen, which I hadn’t seen before. Some folks at MIT have whipped up a bit of code and a database of computer science topics, phrases, and graphs, and developed a quick paper generator. The paper will make no sense at all, of course, but it is quick. And what they’ve found is that making no sense isn’t as much of a handicap as you might think when it comes to some conferences and some journals.
Scigen papers have been accepted for presentation at some of the less prestigious meetings, and have been sent to various cheesy journals, which have cheerfully “reviewed” them once details of payment were cleared up. This is not a good sign for your field when total gibberish can be passed off like this, although one assumes that it says more about the sorts of conferences and journals that are accepting these things.
And yes, a comparison to the Sokal hoax comes to mind immediately. That one was even more damning, though, because the gibberish paper that Sokal came up with wasn’t sent to some sleazy fee-generating publication mill, but to what was considered one of the better journals in the field (Social Text). Who (famously) published it anyway. The editors later backtracked by saying that they thought the paper, you know, lacked originality, that it wasn’t well written, that they (ahem!) just accepted it as a favor to a physicist visiting their rigorous area of study, and so on – but the fact remained (and remains) that an editor should be able to distinguish a valid paper from a sticky pile of superglued nonsense.
The reason the Scigen papers aren’t picked up on, clearly, is that no one’s looking at them, at least no one with any knowledge of computer science. The editors and organizers who let them through are interested in collecting the registration and editorial fees first, and after that, well, that’s not really their department. A perfectly analogous example is the utterly crazed “Atlanta Nights” manuscript, whipped up by a loose team of authors to expose the “editing process” of a pay-to-publish operation (PubilshAmerica) for what it really was. The book is a bit hard to follow. Characters change names and/or genders, die and come back to life, and find themselves doing ridiculous things in impossible tangles of verb tenses. But hey, the manuscript was supposedly read through, and accepted without one solecism out of place. If the credit card number is valid, so’s the syntax. (Don't want to take my word for it? Here it is, under the byline "Travis Tea", published by a print-on-demand house after PublishAmerica hastily backtracked.)
No one’s tried (as far as I know) to submit a Scigen paper to a reputable comp-sci venue. I assume (and very much hope) that it would be sent back with a puzzled note attached. The same goes for the chemical literature, or at least it had better. A chemistry-focused version of Scigen would be an interesting experiment, but I think I know what the likely results would be. There are bottom-tier journals and conferences in every field. They’ll bite. As long as that check clears.
When a medicinal chemist starts digging around through the literature for help on some chemistry, there are several levels of results. The most welcome are recent papers that solve the exact problem you’re looking for, of course. We’re not in business to come up with new reactions (unless we have to,) so if someone else has done the work, that’s great.
Almost as good are similar reactions from the reliable literature. Different people have different borders drawn for that territory, but everyone would include solid publications like the Journal of Organic Chemistry, particularly if it’s a full paper. Interestingly, that’s because JOC is generally not the place (these days) for the hottest work to appear, which takes that out of the equation. As has been proven several times over the last few years, journals can stumble when they try to publish stuff that’s a bit too avant-garde: some of that work is so cutting-edge that it hasn’t even been done yet.
The communications-only journals vary widely in quality, so they’re generally a step down from the better full-paper publications. One big reason is that the communications often don’t include much in the way of full experimental details. One way to tell how useful a journal is would be to measure how surprised you are when you can’t repeat chemistry from it. If you can’t get some reaction from Tetrahedron Letters to work, you just say “Oh, well”, whereas a dud reaction from a long JOC paper gives you more of a feeling of betrayal.
Then there are patents. When I was a grad student or post-doc, I tended to just ignore patent references, but that was partly because I couldn’t get access to them very easily. Now there’s less of an excuse, and anyone who bypasses them is missing out on a lot of useful preparations. They don’t always work quite as wonderfully as advertised, but there are a lot of very interesting intermediate compounds that are described nowhere else. And when the patent goes on to prepare seventy-five enabled compounds from said intermediate, you can be reasonably sure that you’ll be able to make enough for your own needs.
But there are patents, and then there are patents. If there’s no spectroscopic data associated with a compound, you’d better step lightly. Similarly, there’s journal literature and there’s. . .well, there are an awful lot of journals out there. And some of ‘em are, in fact, awful. SciFinder and other such tools are perfectly capable of tossing out references, in the same list as everything else, to the Bulletin of Some Obscure Country’s Academy of Sciences, 1953, communicated from Unpronounceable University in Everyone Leaves Province. You’re on your own if you track these things down, and good luck to you.
In a category all its own is the Soviet-era Russian literature. There are a large number of compounds (particularly heterocycles) that are described nowhere else, and a wide range of these things are available in English translation. But (as with patents) you have to be careful. Some of this material is really worthwhile and unique, and some of it is. . .well, my theory has always been that people in the Soviet era were willing to do a lot to remain "academicians", considering what some of the other options were like. Can't say I blame them, either, but it means that the more obscure Communist-era references need to be approached cautiously. If you're depending on a reference from J. Siberian Oil Chemist's Soc., (a real journal, at one point), then you may need to start looking for some backup.
Update: On the other hand, here's a new mathematics journal that's made up, explicitly, of papers that have been rejected by peer review. Each includes a summary of why it was rejected, and why the original author thinks it's still important anyway. . .thanks to Marginal Revolution for the tip.
You know, this is a comparatively minor sin, but it's an irritating one. I was browsing through my Google Reader list of chemistry journals, and this paper caught my eye. It's from a group in Hyderabad, and describes a preparation of propargylamines using indium bromide. "I've seen that reaction before," I thought.
And sure enough, here's a 2005 paper from a group in Tokyo which describes the preparation of (among other things) propargyl amines using. . .indium bromide. The details are slightly different, but it seems clear that these reactions are proceeding through the same sorts of intermediates. The earlier Tokyo reaction uses N,O acetals, while the Hyderabad one starts from the amine and aldehydes (which, to give them their due, is more convenient).
So fine, the new paper is a reasonable way to make these compounds. But what gets on my nerves is that its abstract reads:
"Indium(III) bromide has been used for the first time for the synthesis of propargyl amines in a one-pot operation from aldehydes,
amines and alkynes. This is the first example on the use of InBr3 for the activation of both alkyne and aryl imine."
And what's more, it doesn't even reference the earlier Japanese work. What's more, the Japanese authors actually published a preliminary communication of their work in 2003, in Tetrahedron Letters, the same journal that the new paper appears in. As I say, I realize that this is a (comparatively) minor sin. And I realize that it goes on all, all, all the time. But it's still wrong. And someone should have called the authors on it when the paper was reviewed.
You don’t often get to see the sort of fistfight that’s detailed in the latest issue of Organic Process Research and Development. Patents whose procedures are hard to reproduce are familiar to every industrial chemist, unfortunately, but coming across one that seems completely mistaken in its most important details is rare. And this is the first time I’ve seen one of these dragged out into the open literature for a give-and-take with the original authors about whether they’re delusional or not. (The editors of the journal seem to be in new territory themselves on this one).
I should add here that the great majority of patent preps I’ve followed have worked pretty much as described, and I don’t think that my success rate in reproducing them is any worse than procedures from the chemical journals. Some journals more than others, of course, (another topic!) but OPRD is known to be very, very reproducible indeed. As it should be: it’s a journal for process chemists, whose livelihood is refining chemical routes until they’re scalable, economical, and (very importantly) until they work exactly the same way every time they’re run.
So here’s the situation. In 2007, the journal published a paper by a group from Dr. Reddy’s Laboratories, a large Indian company that does both generic drugs and has their own drug discovery operation. (There are, I should note, some academic co-authors who seem to have completely disappeared during this current food fight). The paper covered a synthesis of S-citalopram, and it caught the attention of the process chemists at Lundbeck, in Denmark. And well it might – citalopram (Celexa and other brand names), an antidepressant, was discovered there in the late 1980s, and has been generic since 2003.
The original paper (Eliati et al.) described a new alkylation reaction route to produce a key intermediate and a resolution of it (and of citalopram) into pure enantiomers by forming chiral salts. So far, so good – these sorts of things are the heart of process chemistry, and entirely appropriate for a paper in OPRD. But only if they work.
The Lundbeck group (Dancer and de Diego), had tried that exact resolution of citalopram many times themselves, though, without success, so they were rather taken aback to see it published as working just fine. They detail their attempts to reproduce the Eliati procedure, and demonstrate in great detail that it indeed does not work as written. I won’t go into their experimental work, which is very extensive and painstaking, but nothing the Lundbeck team could do resulted in anything better than a 55:45 mixture, which is a rather poor substitute for a pure compound. Midway through their paper, they start putting the word “resolution” in quotation marks when discussing the Eliati procedure, and the arm’s-length-and-holding-the-nose attitude is very successfully conveyed. The phrases “enormous disparity”, “effectively impossible”, “extremely unlikely”, and “not feasible in any meaningful, practical sense” all make appearances.
They also were surprised at the alkylation reaction reported in the Eliati paper, which is the only one of its kind reported in the literature – well, other than a patent by the same team from Dr. Reddy’s, that is. The weird thing about it is that it uses 3-chloropropylamine, apparently as the isolated free base. My chemistry audience will now be raising their eyebrows, because this is not a compound that you’d expect to be very happy as anything but a salt. It should, in fact, start reacting with itself quite vigorously, with plenty of HCl being given off in the process. But the Eliati procedure doesn’t have enough base to allow for anything else, and they use (supposedly) 12 grams of the stuff in 2.5 mL of DMSO. Since no paper or patent has ever reported isolation of this free base, it’s a rather odd compound to drop into your manuscript without explanation.
Another example of the same reaction in the Eliati paper is even weirder. Not only do they use this never-before-seen chloropropylamine, but this time they do the reaction in acetone, at 60 to 65 degrees C, by first adding 7.5 grams of potassium t-butoxide to 40 mL of the acetone. Now that prep should get the attention of the organic chemists in the audience, because that sounds like an excellent way to make a bunch of hot polymerized gunk. For one thing, acetone boils at 56, so how you get it to 65 is a real stumper. And adding a strong base to it is a surefire way to deprotonate it and start the famous aldol condensation (and every other base-catalyzed ketone reaction you can think of, for that matter). The Lundbeck group tried it, out of sheer curiosity, and got:
”. . . a vigorous/violent reaction. . .with the formation of a quantity of a white solid. (It had) an odor of higher ketones/alkenes, and analysis by NMR indicated that it was a complex mixture of products, with peaks consistent with condensation products of acetone.
A solid majority of the chemists reading that sentence, you can bet, finished reading that and added a “No shit” to the end. This is the sort of thing a sophomore undergraduate should be able to spot, and my guess is that whoever reviewed the Eliati paper for OPRD has had some interesting correspondence with the journal. The resolution is one thing – that’s impossible to spot if you haven’t worked with that exact reaction. But this alkylation step is ridiculous.
The journal gave Eliati and co-workers a chance to respond to all this, and followed that with a last word from Dancer and de Diego at Lundbeck. These things are all published back to back; it's like watching a boxing match. The Dr. Reddy’s group runs up the white flag immediately on the chiral salt resolution, actually, agreeing that their published procedure doesn’t work. But they claim that a modified version of the procedure does work, and that they “inadvertently missed incorporating a few words in the text” of the article which would have made this clear. The Lundbeck group isn’t buying this for a minute. They point out that the manuscript would have been had to have been substantially reworked to make it into this different procedure, for one thing. And even worse, the details of it as reported by Eliati are internally inconsistent, with the masses and ratios not even adding up. And finally, they report their own attempts to reproduce the new procedure, and find that it, too, is basically impossible.
And as for the alkylation, Eliati et al. claim that if you work quickly, you can use the chloropropylamine free base as they described. They also present a table showing how long it lasts under different conditions and in different solvents, and claim to have done the best variation of the reaction on a six-kilo scale. The acetone reaction, they admit, wasn’t as clean, but they didn’t spend much time talking about that because their “aim was to isolate the desired product instead of the aldol product.” Dancer and de Diego aren’t very happy with that either, continuing to insist that the acetone procedure is “completely unworkable”. As for the chloropropylamine, they welcome the clarifications in the second Eliati paper, but point out that said details contradict themselves at one point, and at any rate, none of them are to be found in the corresponding Dr. Reddy’s patent application, which continues to talk about using only the free base, and (on top of everything else) in a way that makes no sense.
The final Lundbeck reply has a telling line in the acknowledgements, which is, in its way, even more pointed than anything else in their paper: “One of us (R.J.D.) thanks Sir John Cornforth for inspiration derived from a series of his articles in a similar case some years ago.” That’s the famous “Some Comments on a Paper by Samir Chatterjee” affair, Tetrahedron Letters1980 709 and 1982, 2213. Cornforth completely demolished some heterocyclic chemistry work by the unfortunate Chatterjee, pointing out by several lines of evidence that the whole thing had to have been faked. Name-dropping this example is about as direct a statement of your opinion as the scientific literature will allow. . .
I'm taking the day off from cranking out the medicines of tomorrow (OK, the day after tomorrow), so there will be no post today.
I did want to add something about yesterday's post on the La Clair/hexacyclinol controversy. I'd like to ask that people not fill up the comments with ad hominem remarks or potentially libelous statements about La Clair himself. I don't mind saying that the evidence so far makes it very hard for me to believe his original paper, and I also have to say that I haven't seen any convincing explanations for all the discrepancies that have turned up. And I think that those opinions are shared by many people who've followed the story.
But let's keep it on a scientific plane, if possible. Opinions on NMR spectra and the like are one thing, but personal insults are another, and those we don't need. I try not to have to go in and hose out the comments sections around here.
Remember hexacyclinol? Some readers are probably groaning and thinking “Oh, yes, indeed”, which may make up for the ones who are saying “Remember what?”
Hexacyclinol is a complex natural product, but after that statement the arguing begins. James La Clair published a synthesis of it in 2006 in Angewandte Chemie, one of the most prestigious chemistry journals, but the reception of the paper did nothing to help the prestige of either La Clair or the journal. Readers immediately seized on odd spectral data and experimental details to ask whether the molecule had been made at all and just how well the manuscript had been refereed.
The story got even messier later in the year when synthetic organic chemist Scott Rychnovsky weighed in with a paper suggesting that the structure of the natural product had been misassigned to start with. This was followed by a synthesis by John Porco and his group of his proposed structure, which turned out to match the NMR spectra of the original natural product. Since they also had an X-ray crystal structure, you would think that this would have ended the argument, at least at the level of what hexacyclinol looks like. The argument about what La Clair actually made, though, continued. And La Clair himself suggested that he and Rychnovsky had made two different molecules that just happened to have very similar NMR spectra.
Now a paper in Organic Letters is trying to clear that part of the story up, saying that ” indeed, the possibility that two molecules as complex as 1 and 2 may have indistinguishable NMR spectra carries an uneasy feel.” The authors, Giacomo Saielli and Allesandro Bagno from the University of Padova, return to the two proposed structures and calculate both their carbon and proton NMR spectra using what appear to be the best methods available for estimating their shifts and coupling constants.
The second structure fits much, much better, and the authors conclude that there is “hardly any doubt” that it’s the correct structure for hexacyclinol. In fact, they go further:
”The structure of hexacyclinol is confirmed to be 2. Furthermore, if 1 had been synthesized or was formed from an unforeseen reaction, its NMR spectra are sufficiently different from those of 2 as to guarantee their distinction.”
Note the “if it had been synthesized”. As far as I can tell, the remaining questions in this case aren't chemical. They're psychological. The original Ang. Chem. synthesis is certainly incapable of generating the real structure of hexacyclinol, but it also appears incapable of making the structure it claims to have made. Taken together with its original odd features, you have to wonder just what it was: a hoax? An odd and pointless work of fiction? Self-deception? We’ll probably never know. All we know is what it isn't.
A reader reminded me of this paper, which I meant to blog on when it came out last year. The authors looked over the entire Chemical Abstracts Service registry file – in theory, every compound that’s ever been reported in the chemical literature – and asked how many different chemical scaffolds make up the organic chemistry part of the collection. (That ran to a bit over 24 million compounds at the time the paper was written).
You’d expect a power-law (“long tail”) distribution in a data set like this, and that’s just what they found. Among heteroatom-containing scaffolds, the most common 5% were found in about 75% of the compounds. In fact, it was even steeper than that – the most common 0.25% of the heteroatom frameworks made up half the compounds! The flip side of this is that about half of the known scaffolds occur only once, which is about as long a tail as you can get.
That’s almost completely accounted for by (1) the availability of certain starting materials, largely from petroleum and from natural products and (2) the interest in preparing a given framework. Put more crassly, it depends on how much it’ll cost (in time and money), and how much you expect to get back. As the authors put it:
” We believe the presence of this power law is quantitative evidence that the minimization of synthetic cost has been a key factor in shaping the known universe of organic chemistry.”
Tiny variations can send a given scaffold diving off the charts. Think, for example, about the usual steroid framework – there have been a huge number of variations worked on that, since they’re of medical interest and the starting materials are available (thanks, in the early days, to some Mexican yams and their biggest fan). But imagine going in and replacing one or two of those carbon atoms with nitrogens: whoosh, down you go. Many of those frameworks have hardly been touched at all, partly because they’re quite difficult to make. You’d have to have a very good reason to go after them, and that hasn’t presented itself. Meanwhile, the vast numbers of indoles, piperazines, and piperidines in drug molecules help to perpetuate themselves.
The same goes, and even more so, for general compound shapes (heteroatoms or all-carbon). The authors found 836708 different framework shapes, but that breaks down rather sharply: half the compounds are accounted for by 143 frameworks, and the other 836565 make up the other half. I’ll let the authors have the last word:
”It seems plausible to expect that the more often a framework has been used as the basis for a compound, the more likely it is to be used in another compound. If many compounds derived from a framework have already been synthesized, these derivatives can serve as a pool of potential starting materials for further syntheses. The availability of published schemes for making these derivatives, or the existence of these derivatives as commercial chemicals, would then facilitate the construction of more compounds based on the same framework. Of course, not all frameworks are equally likely to become the focus of a high degree of synthetic activity. Some frameworks are intrinsically more interesting than others due to their functional importance (e.g., as a building block in drug design), and this interest will stimulate the synthesis of derivatives. Once this synthetic activity is initiated, it may be amplified over time by a rich-get-richer process. . .”
So I see that Nature Chemistry looks to be fueling up to launch. I’m curious to see how that one will do. The other Nature journals have done pretty well at preserving the prestige of the name, but I hope that they haven’t reached the point of diminishing returns just when they get to my specialty.
And this will be unusual, since you don’t see many attempts to launch a high-end journal from scratch. Most new journals aim for the middle (or worse), figuring that that’s where the papers will come from. But Nature Chemistry will presumably try to compete with JACS and Angewandte Chemie. My guess is that they’ll be able to do that, which means that the journals one tier down will be the ones that feel the consequences. On the other hand, it’s not like Nature has a reputation for paying much attention to chemistry over the years, so perhaps the cachet won’t carry over in the same way that it did for, say, Nature Medicine. We shall see. . .
I say unkind things about various scientific journals here on occasion. But I have to say that I've never encountered a situation in chemistry that matches the affair of M. S. El Naschie, editor of the Elsevier journal Chaos, Solitons, and Fractals. It's nice to see the editor-in-chief show up with an occasional paper in his own journal - keeping one's hand in and all. But El Naschie has published three hundred and twenty-two papers in the journal since he assumed editorship. He has five in the December issue alone!
The N-Category Cafe, a math blog from the University of Texas, has the full story here. Briefly, El Naschie seems to have been running this journal as his own private kingdom for some time now. While I'm not qualified to referee his works, those who are report that his papers don't make much sense - "undisciplined numerology larded with impressive buzzwords" is one review at the UT site. (That's a phrase I'm going to have to remember for future use; it's bound to come in handy).
Would you like to subscribe to this fine journal, and get the latest updates from El Naschie-land? That'll run you $4520/year. As a library, you'll be getting that as part of a bundle of other presumably more useful journals, so you won't be paying full whack. But still. Why pay anything for a vanity press full of nonsense? (And if there are some real papers in there from other groups, then I pity them for having to appear alongside the gibberish).
Elsevier seems to have finally caught on, after many recent protests. The journal's home page now states:
The Founding Editor for Chaos, Solitons and Fractals Dr El Naschie will retire as Editor-in-Chief. This will be announced in the first issue of 2009.
The publisher will work with the editorial board and other advisors to identify a new editor, as well as reviewing the aims and scope of the journal, as well as the editorial policies and submission arrangements.
I'll bet they will. What's puzzling is why this took so long - isn't anyone at Elsevier paying attention? And why did it take some math bloggers to get things rolling - where has everyone else been in the field all this time? Just rolling their eyes and chucking the issues into the circular file, apparently. As I say, I don't know of anything close to this in chemistry - if anyone has info to the contrary, let's get on it ourselves. . .
A colleague and I got to talking yesterday about something that I'm sure many chemists have noticed. Have you ever chased down some reaction or compound in the literature, only to find yourself wild-goosing back to some obscure journal that no one has ever read - just because no one can be bothered to publish a modern procedure?
Here's how that typically works. You run a SciFinder search on Molecular Structure X. A list with a dozen references comes up. There's a Tet. Lett. from 2002, but what are the chances it'll have any spectral data (or anything useful at all?) Ah, there's one from Tetrahedron in 1995, that should do. So you look over the PDF, search for your compound. . .there it is, number 17. Now to the experimental. . .and you find in the first paragraph that "Compound 17 was prepared according to a published procedure", footnote thirty-eight. And the footnote is to. . .ay, it's to a Chem. Ber. paper from 1932. Ausgezeichnet!
Oh-kay. Back to that SciFinder reference list. How about that Tet. Lett. paper? Nope, on inspection, it turns out to reference the 1995 paper you just looked at. What else? There's a JOC from 1984, let's try that. Good ol' JOC, solid stuff. Well, digging up that PDF, you find that it refers to a 1980 paper from the same group from Synthesis. Hrm. So you chase that one down, there it is, compound 9, and the experimental for it is. . .footnoted to the 1932 paper. Again.
And that's how it goes. Like as not, you can go through the whole list and find that it's made of tissue paper where your compound of interest is concerned. The whole presence of the compound in the literature is, in the end, based on some obscure German university's report from the last days of the Weimar Republic. What's irritating is that while those 1932 folks clearly must have made the compound, it's not always easy to get those papers immediately. And chemistry has, in fact, changed a bit since those days. Papers from that era rely on distillation and crystallization: there are no chromatographic purifications, because there was (by our standards) no such thing as chromatrography. Spectral data? Hah! UV/Vis was cutting edge back then. You'll get a melting point, an adjective-laden description of the appearance of the crystals, and maybe even a note about how the stuff tastes. Great.
You know that the people who re-made the stuff during the last 25 years didn't steam-distill their product or fractionally crystallize it from some mixture of benzene and carbon disulfide or whatever. They ran a quick column and they took an NMR. So why can't they publish that data? The only reason I can usually see is laziness. Why bother? It's a known compound; just reference it and get that manuscript out the door. . .
Fellow chemists, let’s address a housekeeping problem. I know that not all of us are thrilled at all times with the American Chemical Society, but I think that we can agree that the journals it runs are (for the most part) a valuable resource.
And in these days of modern times, as the Firesign Theatre used to say, the presence of such journals on the internet is crucial. As many of you know, the ACS recently revamped the web pages for its journals, adding a number of new functions. Unfortunately, they seem to have taken away some at the same time.
Initially, on too many of journal sites, the graphical abstract was presented at a default size of “itsy bitsy”. If you wanted to browse the ASAP in-press articles, you had to squint at tiny, fuzzy blobs to see what the authors of each article wanted you to know about it. And that’s clearly not what a graphical abstract is for, is it? I'm relieved to see that this seems to have improved in the last few days.
But I'm not sure when the change took place, because I hardly ever visit the front doors of the ACS journals I read. That’s because I, like many other scientists, follow these things through RSS feeds. (I use Google Reader myself – if anyone’s just setting up an RSS reader and would like my OPML file, just e-mail me and I’ll send it along and save you some time).
And what does the RSS feed look like now? Well, for several days, there were no graphical abstracts at all, which made the ACS feeds look just as ugly and nonfunctional as the Elsevier ones. Way to go! (Isn’t it odd that Tet Letters, the journal that pioneered the whole idea of a graphical abstract in chemistry, doesn’t include its graphics in its RSS stream? I don’t get any, at any rate). Which reminds me - if you don't care for the Wiley feeds, either, specifically Angewandte Chemie's, you can get a better version of that one here.
I wrote the ACS support people and complained about the problem, and was told that they were fixing things. Well, their idea of a fix is apparently to include the tiny, blurry graphic in the feed – those have started showing up this week, and an irritatingly useless sight they are. This is one of those examples of taking something that was perfectly useful before – the RSS feeds they way they were last month – and improving them into junk. The ACS pages will be down again on Sunday, presumably for more fine-tuning. We'll see if the feeds return to functionality next week. If they don't. . .well, please consider adding your voice to the chorus asking that they do. Thanks!
As I’ve mentioned here before, publication in the scientific journals is not necessarily a big priority in the drug industry. Patent filings, on the other hand, are very serious business indeed. If you’re an organic chemist looking to see if some particular compounds have been made, you ignore the patent literature at your peril. That’s where most of our med-chem procedures and analogs end up, and there’s a lot of good chemistry in there that never sees the light of day otherwise. Most of it’s even reproducible! (And that’s a crack that you can make about a number of open-literature journals, too, for that matter).
But we do write up papers from time to time. The problem is, it’s often only after the project has been finished for a while. Sometimes it takes that long to decide that the work is safe to publish, and sometimes there are just too many other (more important) things going on. But the result is the same, and I’ve experienced it myself: you go back to the old project data, ready to assemble it into a manuscript. . .and large sections of it appear to make no sense at all.
It’s disconcerting. By “no sense”, I mean that while the chemistry is fine, and the compounds are what they’re supposed to be, it’s nonetheless hard to see why some of them got made in the first place. Whose idea was it to react that amine with every single isocyanate on the whole shelf? None of the resulting ureas were all that good, so why did we decided we needed seventy-nine of them? And there are always gaps in the story that weren’t so apparent while things were going full speed: how come we never made any more of those N-alkyl compounds? And didn’t we resolve that series of racemates at some point? Somebody was supposed to do that.
All this makes it hard to turn many med-chem projects into coherent stories, and a coherent story is what you'd like for a journal publication. Ideally, you want a narrative, something along the lines of: ”We started with this screening hit – promising, but lacking so many key things. By careful, thorough experimentation, we solved those problems one after the other. Moving from strength to strength, and hardly wandering down any blind alleys at all, the analogs became more potent, more selective, and their physical properties and PK fell right into line. In the end, we prepared the wonderful clinical candidate shown in the last table of data on the last page. Not too obvious, is it? Bet you wouldn’t have gotten there yourself. But that’s how good we are.”
Right. The problem is, no projects ever work like that. Or if they do, I've somehow missed seeing them over the last nineteen years. A more realistic story would go something like: “We started out with this screening hit, and decided that we’d change the right-hand side of it – well, the left-hand side, for those guys down the hall who always drew the thing upside down and drove the rest of us crazy. That’s the part of the molecule that had the easiest chemistry, naturally. And naturally, everything we did to it over there made things worse. So the weeks went by, with management tapping their feet, and finally some of the guys said the heck with it and started changing the back end of the molecule. You thought those other compounds were less potent? You should see these! But one of the changes actually worked, for some reason. Then when we went back and starting messing with the easy side of the molecule, two things happened: for one, the chemistry wasn’t so easy any more. But now those changes actually made things better. So we cranked out a whole pile of these things, hoping for the best, and finally got down to two compounds: one with great potency and selectivity, but iffy blood levels, and one with great PK, but not so great on the potency. Never could bridge the gap. We put ‘em both into two-week tox, and they both flunked out for the same completely unexpected reason. They're not gonna be drugs, we've filed the patents already. . .so, here they are!”
Well, you can’t quite say that, not even in Bioorganic and Medicinal Chemistry Letters. So you put things into the most coherent shape you can, and trust your fellow medicinal chemists – those of them who might actually read your paper – to understand. They generally do.
Science has been writing on and off about scientific publishing, which naturally leads to a discussion of the ways that publication records are evaluated. Fortunately, I haven’t had to deal with this sort of thing myself, but if the reports are accurate, the whole “impact factor” business seems to be well out of control.
Impact factors, for those who haven’t had to worry about them, are an attempt to measure how good different journals are by how often papers in them are cited. The rankings that result are fairly well correlated with the way people have “good” journals ranked in their heads, although review publications get over-ranked by a straight citation count. There have been all sorts of refinements introduced, but the basic principle is the same: to quantify the publication list in someone’s c.v.
And that’s how it’s used in tenure evaluations. There are all sorts of tales of needed at least so-and-so many papers in journals of such-and-such impact factor and above. And in the cases where such things aren’t flatly written down, they’re widely felt to be calculated quietly behind the closed doors. As you’d imagine, not everyone thinks that this is a good thing. One of the letters that came in to Science this time, from Abner Notkins of NIH, says that:
”. . .many scientists are now more concerned about building high-impact factor bibliographies than their science.
The adverse effects of the impact factor culture must be reversed before more damage is done to the orderly process of scientific discovery. Although there may be no way of stopping computer-generated evaluation of journals and published papers, the scientific community certainly can control its use. . .each institution should make it clear, in a written statement, that it will not use the impact factor or the like to evaluate the contributions and accomplishments of its staff. Second, the heads of laboratories should prepare similar written statements and in addition discuss in depth with their fellows the importance of solid step-by-step science. Third, the editors of journals published by professional societies, joined by as many other journal editors as are willing, should indicate that they will not advertise, massage, or even state the impact factor score of their respective journals. By means such as these, it might be possible to put science back on the right track.”
Strong stuff, and to some extent I agree with it. The thing is, there’s nothing wrong per se with publishing in good journals. Aiming your research high is a good thing, as long as good publications are the by-product and not the entire goal. Now, I think that the advertising of impact factors by journals is irritating, especially when they trumpet things down to the sccond decimal place. But I think that a statement that impact factors will not be considered for academic evaluations would be useless. After all, these numbers just put a quantitative coat of paint on a process that everyone engaged in anyway. Papers in Science, Nature, and the like already counted for a lot more on a publication list than did papers in many other journals, and saying that you’re not going to use someone’s numerical rating for them won’t change that. Every scientist in every field has an idea of which journals are harder to publish in (and publish more high-impact work); getting a paper into one of them will always count for more.
As it should. We have to remember what the opposite situation looks like. Everyone’s seen publication lists with page after page of low-quality stuff that’s been turned out for quantity, not quality. Communication after communication in high-acceptance-rate journals, obscure conference proceedings, every poster session noted – you know the sort of thing. It’s supposed to look impressive (why list all this stuff, otherwise?) but ends up looking pathetic. We don’t want to end up rewarding this kind of thing.
So what to do? Perhaps a realistic compromise: tell junior faculty and staff that their publication records will be a part of their evaluations, of course. But tell them that they’re not the most important part, and that a short publication list can be balanced out by other factors (and a long one balanced out in the other direction, too!) Someone who’s doing really good work, but who declines to slice it up into publishable bits, or whose research is just not on a schedule for lots of publications no matter what, should know that they’ll be evaluated with these things in mind. Likewise, someone who runs every single experiment to slot into the next manuscript had better also be running the ones that they’d set up even if journals didn’t exist, and we all still communicated by handwritten letters. Good science is still good science, whether it’s published (or even if it’s published!) in Science or not.
You know, every time I point out a paper from PNAS, there are always a few comments to the effect of "Why do you bother reading that garbage heap, anyway?" Since I keep citing papers from the journal, it's obvious that I disagree, but I suppose I should take a minute to explain why.
The reason people are down on PNAS is the way that members of the National Academy can, if they choose, sort of jam things into the journal through a side entrance. Here are all the details. The unusual thing about the journal is the existence of "Track I". Basically, a member of the NAS can publish up to four of their own papers per year. Each of these have to be submitted with the comments of two qualified referees, but the author gets to pick them. So a reasonable member should be able to get any sort of interesting or at least non-insane paper in there, by judicious choice of colleagues for review. Members can also pass along up to two papers a year by others in their field, with a similar review process (Track III). Some NAS members take full advantage of these privileges, and some hardly ever do, even (so I'm told, in some cases) for their own papers.
It's a lot less rigorous than the open (Track II) submissions, that's for sure. For those, you're supposed to name three editorial board members, three NAS members, and five external referees, and the editorial board can still do whatever it wants with your paper or with the lists you've sent. (To be sure, they can also reject those direct-submission papers from members, although no figures are available on how often that happens). Two thirds of the Track II submissions are rejected before being sent out for review at all.
But hold on: according to the journal, 80% of the submissions are via Track II, but those make up only 40% of the published contents. Doing the math, that means that the most of Track I and Track III submissions have to get in. Assume 80 Track II manuscripts and 20 of the others. Rejecting two-thirds of the first group will give you about 27 papers to send out for review. If you've accepted all 20 of the others, that means that about half of those 27 will have to get canned during the later review process, to make that 40/60 proportion come out right. So the overall acceptance rate for open submissions has to be, at most, 16%.
But if you ditch some of the 20 member-track papers, you have to come down even harder on the open submissions, of course. If you only (only!) take 75% of the member submissions, that gives you 15 manuscripts. Now you have to reject not half of the open submission papers that made the first cut, but 63% of them, to knock it down to ten published Track II submissions. So with an acceptance rate of 75% for member submissions, it has to be about 12% for everyone else. And so on.
So much for the numbers - it's clear that NAS members must put a lot of things of their own (or from their friends) into PNAS. The real question is: what does this do to the quality of the journal? As far as I can see, it's still a very interesting read, and definitely cannot be safely ignored. And the publication routes are out on the table: if you want to keep score and adjust your perceptions accordingly, the Track I papers are identified as "Contributed by" the member, and the Track IIIs are "Communicated by". I think, myself, that the advantage of letting members publish unusual or possibly controversial work outweighs the temptation to fill the journal with junk.
The Boston Globehas a piece on the open-source science movement. Many readers here will have come across the idea before, but it’s interesting to see it make a large newspaper. (Admittedly, the Globe is more likely to cover this sort of thing than most metropolitan dailies, given the concentration of research jobs around here).
The idea, as in open-source software development, is that everything is out in a common area for everyone to see and work on. (Here's one of the biggest examples). Ideas can come from all over, and with progress coming more quickly as many different approaches get proposed, debated, and tried out. I like the idea, in theory. Of course, since I work in industry, it’s a nonstarter. I have absolutely no idea of how you’d reconcile that model with profitable intellectual property rights, and I haven’t seen any scheme yet that makes me want to abandon profit-making IP as the driver of commercial science. Of course, there's always the prize model, which is worth taking seriously. . .
Even for academic science, open source work runs right into the traditional ideas of priority and credit, and the article doesn’t resolve this dilemma. (As far as I can tell, the open-source science advocates haven’t completely resolved it, either). There’s always the lingering (or not-so-lingering) worry about someone scooping your results, and for academia there’s always that little question of grant applications. There have been enough accusations over the years in various fields of people lifting ideas during grant proposal reviews or journal refereeing to make you wonder how well a broader open-source system would work out, given the small but significant number of unscrupulous people out there.
On the other hand, maybe if things were more open in general, there would be less incentive to lift ideas, since the opportunities to do so wouldn’t be so rare. And if someone’s name is associated from the beginning with a given idea, on some open forum, it could make questions of priority easier to resolve. A subsidiary problem, though, is that there are people who are better at generating ideas than executing them – some of these folks, once unchained, could end up with their fingerprints on all sorts of things that they’ve never gotten around to enabling. Of course, that might be a feature rather than a bug: people who generate lots of ideas are, after all, worth having around. And over time, there might well be less of a stigma than there is now for someone else to follow up on these things.
The thing is, science has already been a form of open-source work for hundreds of years now. It’s just that the information has been shared at a later stage, though presentations and publications, rather than being put out there right after it’s been thought up or while it’s being generated. That’s why I always shiver a bit when I read about how long Isaac Newton waited before writing up any of his results – if Edmund Halley hadn’t pressed him to do it, he might never have gotten around to it at all, which would have been a terrible tragedy.
And it’s why stories like those told of physicist Lars Onsager strike me as somehow wrong. Onsager was famous for only publishing his absolute best work – which was pretty damned good – and putting the rest into his copious file cabinets (example here). (A related trait was that he was also apparently incapable of lecturing at any comprehensible level about his work). Supposedly, younger colleagues would come by once in a while and tell him about some interesting thing that they’d worked out, and ask him if he thought it was correct. Onsager would pause, dig through his files, pull out some old unpublished work that the new person had unknowing duplicated, and say “Yes, that’s correct”. It seems to me that you don’t want to do that, withholding potentially useful results for the sake of what is, in the end, a form of vanity.
And although I'm not exactly Lars Onsager, this is as good a time as any to mention that my summer student, who’s finishing up in the lab this week, has been able to generate a lot of interesting data, and that I’m going to be trying to write it up this fall for publication. Readers may be interested to know that this work is based on more ideas I’ve had in the vein of the “Vial Thirty-Three” project detailed here, so with any luck, people will eventually be able to see some of what I’ve been so excited about all this time. And that’s about as open-source as this industrial scientist can get!
Here’s an appropriate topic for a Friday, although at first many of you may think I’ve lost my mind. What would happen if you combed the full text of the experimental sections of the chemistry journals, looking for how long people ran their reactions?
I’m pretty sure that I know what you’d see: there would be a lot of scatter in the short time periods, with some peaks at the various half-hour and hour marks just for convenience. But as you went out into the multiple-hour procedures, I feel sure that you’d see pronounced spikes in the data at around sixteen to twenty hours and again at around 72 hours.
Some readers have doubtless started nodding their heads, having done the math. Those times correspond to "overnight" and "over the weekend", and I'm willing to bet that they're over-represented (and how) in the data set. I'll go on to predict scarce examples in, say, the 14-hour or 38-hour ranges - there's not much way to run a reaction for those intervals and not be in the lab too early in the morning or too late at night.
A second-order prediction is that when such reactions are found, that their origins will skew heavily toward academia rather than industry. And I'm also willing to bet that patent procedures will tend to follow the working-day timelines more than the general literature, for the same reasons. My last higher-order prediction is that the reaction times would not, in fact, obey Benford's Law, as many other data sets of this kind do.
As far as I know, no one's ever done this sort of analysis, but I suppose it would be possible, especially for someone at Chemical Abstracts or at one of the scientific publishers. If someone wants to try it, please let me know what comes out. And if the results follow my predictions, please feel free to refer to the title of this post or something similar. I won't object.
I sketched out a rather small molecule the other day, a perfectly reasonable looking thing, which nonetheless had absolutely no references in Chemical Abstracts. (I’d certainly like to be able to put up a drawing of the structure, but it’s something that I have a work-related interest in, so it has to stay under wraps). But it’s something with only a dozen or so heavy atoms, most of them flat and aromatic – you’d certainly expect something to have been made like it, but apparently not.
This has happened to me many times over the years. Now, you can obviously get into unknown territory immediately if you start looking for bizarre compounds: I don’t happen to have SciFinder access here on the train this morning, but I’m willing to bet that (for example) three-membered rings with one carbon, one boron, and one silicon are pretty wide open for some brave weirdo to explore. Enjoy!
But you don’t have to go that crazy to leave the paved roads behind. Many reasonable low-molecular-weight areas are only very lightly explored. You can get out of the universe of known compounds very quickly, for example, by searching for spirocycles, particularly with an oxygen or nitrogen or two scattered into the rings. Most of these would surely be interesting scaffolds for drug discovery libraries, if there were reasonable chemistry to explore them with. Even some perfectly normal looking substitution patterns of monocyclic compounds haven’t been looked into – I dreamed up a series of oxazole derivatives not long ago that no one’s ever made, and there’s nothing odd about them at all.
As you’d expect, there’s a commercial niche here. Novelty is a key requirement for patentability, so seeing no references turn up around your interesting structure is good news from an IP standpoint. (It may be bad news from a laboratory standpoint, though, because sometimes these things are unknown for a reason). But not always: there are companies that pride themselves on being able to supply such unknown scaffolds and libraries. The perfectly reasonable-looking diazabicyclo compound shown here, for example, has no references in SciFinder, but can be purchased on a multigram scale. (There are about fifty derivatives of that bare scaffold known in the literature, which makes it pretty much uncleared ground compared to the absolutely pulverized IP landscape around, say, piperazine). Next time you're searching for such things, refine your answer set to give only those compounds with no references, and take a look at how many of them are commercially available anyway. . .
File this under “does no one any good”. As many of you will have seen, JAMA just published a report on various studies that Merck has conducted and published over the years on Vioxx. The conclusion was that the company basically wrote the papers, and then went shopping for well-known academic names as authors. No, this one isn’t going to be good for anyone involved.
There seems little doubt that this practice does go on. I’ve never been in a position to see it happen, but it’s been reported for years. There are whole companies whose business is “scientific writing and communication”, and some of these seem to be in the business of turning studies into manuscripts, with no mention of their work in the final version. (The JAMA article found evidence of this sort of thing as well).
Scientific authorship is a messy business, true, and there are a lot of journal articles whose entire list of authors might have trouble with a pop quiz on the details of the paper. It is, in my mind, perfectly acceptable for one or two people on the author list to do most of the writing, with everyone else contributing suggestions and revisions. That’s how every paper I’ve been on (or written) has been done. But the worst of these Merck cases look like a search for a lead author or co-author, which is just unacceptable.
At least one of the authors named in the article is disputing its conclusions. Stephen Ferris of NYU says that he was no figurehead, and calls the JAMA paper “egregious” for having done no follow-up with the people it names. I suspect that there will be others in his category – the JAMA offices are getting a lot of testy e-mails this week, I’m sure. Of course, even the guilty are going to be sending them, since no one wants acquiesce to the label of “paid shill for publication”.
And that’s the problem. I can believe that the JAMA authors (Joseph Ross of Mt. Sinai et al.) could have cast their net too widely as they dug through the piles of discovery documents from the Vioxx litigation. But, unfortunately, I can’t believe that all their examples are mistaken. Enough chicanery goes on with authorship in purely academic settings – I can well believe that it happens in industry/academic collaborations.
But that’s the problem right there: the idea behind such a collaboration is, at least partly, to lend credence to the study’s results. Rightly or wrongly, industry studies on marketed drugs are perceived as needing the help. It’s the money involved, of course. When an industrial group publishes a paper on cell physiology or on a new method for cleaning up palladium-catalyzed reactions, no one doubts the results. But when it’s something that might have a direct and immediate effect on millions of dollars in revenue, doubts naturally set in. They always will, even if the research is beyond reproach.
And that’s why this ghostwriting business just makes the problem worse. I haven’t seen anyone suggesting that the Merck studies themselves are bogus – they had damn well better not be – but by playing games with the external author list, the company invites suspicion. I’m willing to bet that many people outside our industry who have just read the headlines on this story have assumed that the results were cooked up, just like the authorship. This is not what the industry needs. It never has been, and we need it less now than ever.
If we’re going to win back the trust of the general public – which we’ve lost, in case anyone hasn’t noticed – we’re going to have to cut out the shortcuts, stop the doubletalk, and act as if what we’re doing (drug discovery) is something to be proud of. Sure, this is a business – we sell improved health for money, and since it sure costs money to do it, there’s nothing in that transaction to be ashamed about. So why are we acting as if the only way to do business is under the cover of darkness?
We’re not going to have much of a business if these practices keep going on. Want price controls, real industrial-strength ones? Want lots and lots of marketing restrictions? Want the FDA to raise the bar for approval to levels never before seen? Want flocks of lawyers beating their wings, circling around our every move? Just keep it up, just keep this stuff up. We’ll get all that and more.
So, what’s easier: writing a blog entry every working day, or writing one scientific publication? The blog, the blog, no doubt about it. I write quickly, and pretty much always have, but putting a paper together is still slow work.
One difficulty is the length restriction, especially for a communication. Working in all the necessary details while still telling a coherent story are not always compatible goals, and doing it within four printed pages can be a real challenge. Many med-chem projects are pretty shaggy by the time it comes to publish, and there’s no way to get in all the twists and turns (nor would anyone want to read about them, in most cases).
So you have to decide how the work is going to be presented to give a readable but accurate account. The problem is, almost any project can be turned into a flowing narrative if you’re willing to throw away enough work and to lie about the rest. If you’re not going to do that (and I recommend against it!) then you have a harder job on your hands. You’re going to have to leave out something, but you’re going to have to be able to recognize what’s left.
It’s for sure that you’re not going to be able to talk about every single analog, so one good technique is to narrow down to representative compounds. That won’t always be popular with your co-authors, though. Odds are that some of the people who worked on the project will end up feeling slighted when their contributions make it onto the bottom few rows of a table, or are just mentioned in passing in the text: “Substitution with groups larger than methyl led to rapid loss of activity (data not shown), so our attention then turned to. . .”
Another difficulty is that series of analogs aren’t often made in the order that makes sense in hindsight. I think it’s acceptable to mess around with the timelines a bit in presenting the data, as long as you aren’t rearranging things that had an impact on the main flow of the SAR. That course of events you’re stuck with, and you just have to find a way to make your decisions seem reasonable. It helps if they actually were reasonable, of course. That condition does not always obtain.
As for writing style, I recommend a difficult one: the kind that you hardly see at all. Keeping someone reading along while you deliver the dry, concentrated, chewy news isn’t easy to do, but it’s a goal worth struggling for. Most papers scan as if they’ve been sprayed with light coating of eye repellent: you slide right off of them after a paragraph or so. If you can avoid that, you’re already well out from the pack. As for extra touches, I actually enjoy seeing a bit of personality and humor come through in a scientific paper, but getting that bit right is very difficult. Getting it wrong is very easy, though, and the results are unpleasant. If you’re not sure of your touch, keep your hands off the spice rack. This isn’t the time to be Henry James (is there ever a time?), William Faulkner, or Marcel Proust. If you’re going to emulate a novelist, think Hemingway. Early Hemingway. If you want a journalistic role model, you can aim for Orwell, but that’s a high mark – he had style to burn, but managed not to call attention to it. Good luck!
I wanted to follow up on the post the other day about Pfizer's attempts to open up the editorial files in various scientific journals. The decision on the New England Journal of Medicine motion hasn't come down yet, but two others have.
And Pfizer's lost both of them. The district court in Chicago rejected the company's arguments to compel JAMA and the Archives of Internal Medicine to open up their records on papers concerning Celebrex or Bexxtra. The ruling held (correctly, in my opinion) that the possible value of these documents to Pfizer's case was more than outweighed by the harm that would be done to the journals by allowing access.
And as this story at the Science web site mentions, the NEJM case may well be about to go the same way. According to the journal's attorneys, Pfizer narrowed its request to just the peer-review comments returned to the authors of the manuscripts. That seems, at least to me, to weaken the argument that these documents are of such great value to their legal case, while leaving the problem of breaching confidential peer review.
At least I think it does - I assume that Pfizer wants names attached to these things, unless they can use them in their case without attribution. Even so, that still doesn't sound like something that'll make people enthusiastic about reviewing such papers - the prospect of having their comments read off in open court. No, I think that argument that sank Pfizer's requests in Illinois still obtains, and that the Massachusetts court will rule the same way.
So if this whole issue goes away, we can relax until the next legal inspiration hits. In the interim, I still think that Pfizer should at least be vaguely ashamed of having taken this road. A confidential poll of the company's own scientists would surely find that a solid majority of them would be opposed to the whole idea of legal discovery of peer review documents. (I say that because I've hardly talked to a single chemist or biologist who didn't think the same way). That said, there aren't many companies that size whose business decisions would all survive after polls among the scientific staff. . .
Today (March 13) at 3 PM EST, there's a hearing scheduled on a legal motion that could change the way scientific results are published in this country. Pfizer is being sued over injuries that plaintiffs believe came from their use of Celebrex, one of the world’s only remaining Cox-2 inhibitor drugs. (I saw a Celebrex tv ad the other day, a surreal thing which was basically a lengthy recitation of FDA-mandated side effect language accompanied by jazzy graphics). Everyone with a Cox-2 compound is being sued from every direction, as a matter of course. The company is, naturally, casting around for any weapon that comes to hand for its defense, as did Merck when that same sky began to come down on them.
But Pfizer’s lawyers (DLA Piper LLP of Boston) are apparently (your choice, multiple answers permitted) more aggressive, more unscrupulous, or more clueless than Merck’s. Among the points at issue are several papers from the New England Journal of Medicine. According to the motion, which I paid to download from PACER, two of the particularly contentious ones are this one on complications after cardiac surgery and this one on cardiac risk during a colon cancer trial. So Pfizer has served the journal’s editors with a series of subpoenas. They’re seeking to open the files on these manuscripts – reviewer comments, reviewer names, editorial correspondence, rejected submissions, the lot. What are they hoping to find? Oh, who knows – whatever’s there: ”Scientific journals such as NEJM may have received manuscripts that contain exonerating data for Celebrex and Bextra which would be relevant for Pfizer's causation defense” say the lawyers. The journal refused to comply, so Pfizer has now filed a motion in district court in Massachusetts to compel them to open up.
What's particularly interesting is the the journal has, to some extent, already done so. According to Pfizer's "Motion to Compel", the editors "produced a sampling of forms identifying the names of manuscript authors and their financial disclosures, correspondence between NEJM editors and authors regarding suggested editorial changes and acceptance and rejection letters". The motion goes on to say, though, that the editors had the nerve to ignore the broader fishing expedition, only releasing documents for authors specifically named in the subpoenas, not "any and all" documents related to Celebrex or Bextra. They also withheld several documents under the umbrella of peer review and internal editoral processes. Thus, the request to open up the whole thing.
I’ve never heard of this maneuver before. Staff members of the NEJM gave depositions in the early phases of the Merck litigation, since the journal was in the middle of the Vioxx fighting. (They’d “expressed concern” several times about the studies that had appeared in their own pages and passed through their own review process). But even then, I don’t think that Merck wanted to open up the editorial files, and you’d think that if anyone had something to gain by it, they would.
Pfizer’s motion seems to me more like a SLAPP, combined with standard fishing expedition tactics. Their legal team doesn’t seem to think that any of this will be a problem, at least as far as you can tell from their public statements. They say in their motion that they don’t see any harm coming to the NEJM if they comply – heavens, why not? Reviewers will just line up to look over clinical trial publications if they think that their confidentiality can be breached in case of a lawsuit, won’t they? And the rest of the scientific publishing world could look for the same treatment, any time someone published data that might be relevant to someone’s court case, somewhere. Oh, joy.
Pfizer’s motion states that ” The public has no interest in protecting the editorial process of a scientific journal”. Now, it’s not like the peer review process is a sacred trust, but it’s the best we’ve been able to come up with so far. It reminds me of Churchill’s comment about democracy being the worst form of government until you look at the alternatives. I realize that it’s the place of trial lawyers and defense teams to scuffle around beating each other with whatever they can pick up, but I really don’t think that they should be allowed to break this particular piece of furniture.
And I can’t see how the current review process won’t get broken if Pfizer’s motion is granted. The whole issue is whether the journal's editors can claim privilege - if so, they don't have to release, and if not, they most certainly do. This can't help but set a precedent, one way or another. If there's no privilege involved in the editorial process, a lot of qualified and competent reviewers will start turning down any manuscript that might someday be involved in legal action. (Which, in the medical field, might be most of them). The public actually does have an interest in seeing that there is a feasible editorial process for scientific journals in general, and I hope that the judge rules accordingly.
In the meantime, for all my friends at Pfizer and for all the other scientists there with integrity and good sense: my condolences. Your company isn’t doing you any favors this week.
(One of the first mentions of all this was on the Wall Street Journal’s Health Blog. The comments that attach to it are quite interesting, dividing between the hands-off-peer-review crowd and a bunch of people who want to see the NEJM taken down a few pegs. I can sympathize with that impulse, but there has to be a better way to do it than this. And there’s more commentary from Donald Kennedy, editor of Science, here (you can pretty much guess what he thinks about this great idea).
Thinking about that plagiarizing Indian professor brings up the same thought I always have in these situations: what on Earth is going through the heads of these people?
I can tell you, honestly, that I have never faked any data. (That phrase makes me remember, though, that one of the most crazed fabulists I’ve ever known started a good number of his sentences with the phrase “I tell you honestly”). I would feel nervous and guilty about making up so much as an NMR coupling constant – I freely admit to having put down “10 Hz” for something that might well be 9 on closer inspection, but making it up without having even looked? No way. It’s not like I have a halo over my head, but hey, these things are real numbers that people can check. You’d think that if a person feels the need to lie about things that they’d pick something else to lie about. I can see telling people that the check is in the mail, or that yes, I did indeed read every word of your insightful memo, but I can’t see telling someone that I made some compound that I didn’t make.
So, then, faking up a whole publication? How can you do that and sleep at night? Even if it’s just some obscure analytical method, published in a journal that no one has ever read an issue of front to back, how can you do that? Well, then, how about sixty or seventy of the damn things over a period of a few years – that’s what this guy did, after all.
And I think that, other than the (to me) incomprehensible mental angle, what I feel about this sort of thing is anger. Although I work in a very applied research field, I think that scientific research is generally a good thing in and of itself. I’m signed up with Francis Bacon and his program “for the effecting of all things possible”. (Peter Medawar's thoughts on this are well worth reading). So this sort of cynical fakery really gets to me, because it’s the work of someone who, in the end, figures that science and data are just stuff to use to get what you want. They’ve no intrinsic value. It’s not like anyone cares, right?
It’s like watching a pastry shop mix ground cardboard into their muffins – hey, you get more muffins that way, and what good are the damn things anyway if not to unload them on the idiot customers for cash? So for anyone who came to Chiranjeevi’s work for anything useful (God help ‘em), well, his message to you is to stick it in your ear. “Useful for you” isn’t anything he cares about. What he’s interested in, of course, is “useful for him”, and that’s what the whole enterprise of science comes down to for someone like this: a means to an end. And what mighty end is that? Why, advancement at Sri Venkateswara University, of course. And some pocket money. And a longer CV. Noble stuff, isn’t it?
Since I had a blog entry here recently talking about plagiarism, I thought I should point out a whopping case of it that’s come to light. One Pattium Chiranjeevi, a professor of chemistry at Sri Venkateswara University in Triupati, India, has been accused of cranking out dozens of forged publications over the last few years.
I don’t see how there can be any doubt about the guy. He published 60 or 70 papers in under four years, which is enough to make you wonder right there. Unless you’ve got a monster research group, and you’re constantly breaking everything down into the tiniest bites and repeating lots of stuff to boot, that’s just not possible. But these papers, mostly on analytical methods development, are just too similar to things that were already in the literature. Elsevier has already retracted thirteen papers from the list, and no doubt other publishers are working on doing the same. A panel at his university has concluded that he plagiarized data and included “unjustified co-authors”. My favorite part of the whole affair is that some of his publications include data from instruments that don’t even exist at SVU.
We owe P. K. Dasgupta at UT-Arlington for catching on to all this. As detailed here in C&E News, he realized that one of Chiranjeevi's papers sent in for review was identical to something he'd seen last year. Well, mostly identical - Chiranjeevi had gone so far as to substitute the word "arsenic" for the word "chromium", but other than that demanding find-and-replace job, the manuscripts were identical. That should give you some idea of the level this guy was working on. Interestingly, he doesn't seem to show up in that Deja Vu database I linked to earlier, even though some of the journals he published in are in PubMed - is this because of these sorts of word games?
Science managed to get ahold of Chiranjeevi for comment, and his response does not inspire visions of a man unjustly accused. He blames colleagues and journal editors for the whole thing, says the charges are “baseless”, and (you won’t see this one coming) says that he plans to take action in an “international court of justice” to clear his name. Science left that last phrase in quotes, too, even though it’s a perfectly recognizable English term, which is the equivalent of putting “sic” after it: “That’s really what he said, folks; we’re not making that one up”. What sort of person starts blowharding (no offense!) about international courts of justice in a situation like this? Quite possibly the sort of maniac who’s capable of, well, plagiarizing up a new publication every three weeks or so without even bothering if the experimental section includes equipment that he’s ever seen or used. What goes through the heads of these people is a mystery that the rest of the population may never solve.
That Science news article tries to tie this to the recent scandals in stem cell research and low-temperature physics, but I don’t think the comparison holds up. For one thing, those two weren’t plagiarism, but featured results that had been completely made up. And they were spectacularly focused on hugely popular fields of research while Chiranjeevi’s papers are small and relatively obscure. It’s doubtful that anyone was led down the wrong path by reading them – in fact, it’s doubtful if anyone read them to any great extent at all, which is how something like this can go on so long. These sorts of papers are specialized reference material, not breaking news. Actually, it makes more sense to plagiarize that kind of work than to claim to have performed groundbreaking work in stem cells or superconductivity. If Chiranjeevi had cut back to a few papers per year, he probably could have made a career out of it. For some values of the word “career”.
Note: if I'm lucky, maybe one of the professor's defenders (!) will show up in the comments section, as one seems to have here and here!
There’s an analysis in the latest Nature that puts some numbers on a problem that scientists the world over have suspected for some time: the number of duplicate papers that show up in the literature. The authors used this online text-similarity tool to go through papers in Pubmed, and found a small (but not as small as it should be) percentage of papers that seem to be the same damn things, recycled.
As it turns out, the “most similar papers” function over on the right-hand side of the Pubmed results was a good starting point for tracking these down, and this shortcut allowed them to search the entire Pubmed database. The authors have set up a web site where they've deposited their data and their lists of duplicate papers. Out of about 7 million abstracts, some 70,000 were flagged as being highly similar to their corresponding "most related article" on Medline. Manual checking suggests that about 50,000 of these are going to be true duplicates - they've gone through about 2700 by hand so far (statistics here).
They have drawn some preliminary conclusions from their data set. For one thing, duplication seems to have been steady or trending down in the database during the 1990s, but has been increasing since 2000 (and is currently at the highest level). Their explanation - the rising number of print and online journals, making copying easier to perform and harder to detect - seems right to me. Another interesting graph is the frequency of duplicates by country of origin, versus that country's relative contribution to the Medline database as a whole. Looked at that way, the US is under-represented in the duplicates (which is good to know), and Japan and China are quite over-represented. Several explanations for this are considered – original publication in a language less used for scientific publication, followed by a chance to expose the same work to a wider audience, for one. But the authors don't hesitate to cite "differences in ethics training and cultural norms" as a factor, too.
A further fascinating detail is that the papers which seem to have been duplicated in different journals by the same author (or authors) very often appear too soon after the first publication to have gone through the reviewing process sequence. In other words, they were most likely submitted simultaneously to both journals, which isn't a nice thing to do. By contrast, when the same stuff appears under someone else's name, there's generally an appropriate time lag.
This study notes that their manual inspections have, so far, found over seventy cases of what looks like outright plagiarism, and that they're starting to contact journal editors and universities for more details. And they also seem to have found a number of what they term "serial offenders", and are investigating those cases as well. They don't go into details, but my guess is that some of those people could possibly be found here.
Their hope is that if such authors realize that such tools exist, that plagiarism and duplication will be seen as more risky. Thus all the publicity. Want to try it out yourself? The list of potential duplicates can be found here. Here's the list of journals, and you can plug those into this search page and see what you come up with. Here are some of the manually checked papers - click on the left-hand side ID number to see a side-by-side comparison.
I really should call attention to this blast against journal impact factors from the folks at Rockefeller University Press. They have a number of complaints, among them that the folks at Thomson who put the numbers together are inconsistent about what gets counted as a citable article and what gets tossed aside as “front matter” (editorials, news updates, etc.)
They went so far as to purchase the Thompson data to check the impact factors for their own journals, but despite several attempts, they could never get anything that matched up with the official figures. The explanations they received for this problem were of gradually decreasing credibility.
” When we requested the database used to calculate the published impact factors (i.e., including the erroneous records), Thomson Scientific sent us a second database. But these data still did not match the published impact factor data. This database appeared to have been assembled in an ad hoc manner to create a facsimile of the published data that might appease us. It did not.
It became clear that Thomson Scientific could not or (for some as yet unexplained reason) would not sell us the data used to calculate their published impact factor. If an author is unable to produce original data to verify a figure in one of our papers, we revoke the acceptance of the paper. We hope this account will convince some scientists and funding organizations to revoke their acceptance of impact factors as an accurate representation of the quality—or impact—of a paper published in a given journal.”
There’s now some competition for journal ratings, at any rate. You can search this list for free – they have their own system for ranking journals, which I don’t completely understand, but the order of the chemistry titles is not obviously crazy. It’s hard to keep the review journals from dominating any such list, and that’s what they do here. Accounts of Chemical Research, for example, usually come out pretty high, although its actual let’s-see-what’s-in-there readership is probably not too impressive.
What fascinates me, though, are the lower reaches. I have a grim curiosity about the least impactful titles, the unciteable compost bins whose pages never flip. Sorting things out that way, you find journals in which, it's safe to say, every single author wishes that their paper could have gone somewhere else. In this better-than-nothing league (an arguable assertion, in many cases) you find what you expect to find: Egyptian Journal of Chemistry, the evocatively abbreviated J. Chem. Soc. Pak. andBull. Chem. Soc. Ethiop., among others. You may have seen these things come up in particularly diligent online searches.
But have you ever seen anything from Chemical Papers? There's an exciting name for you - that one's from Slovakia. Or Oxidation Communications? Did you even know that a journal with that title existed? (It's Bulgarian). Journal of Natural Gas Chemistry? Chemical Journal on Internet, from Switzerland, of all places? Maybe I don't get around enough.
Another thing you learn is that some journals you've heard of are not doing very well. It seems safe to say, for example, that the Beilstein Journal of Organic Chemistry is not working out so far, since it's face-down in the mud at number 468 out of 470 titles. The Journal of Structural Chemistry could apparently vanish from the earth without many people noticing, as could Chemistry of Natural Compounds, and there's a whole list of Russian and Chinese journals that publish large numbers of manuscripts to very little effect.
There are plenty of chemical reagents and reactions that go in and out of fashion over the years, and even entire elements. For the last couple of years, it’s been gold – ten years ago, gold-catalyzed reactions were a backwater, and now they’re all over the literature. (Catalysts are the way to go; reactions that need excess gold to run are unlikely to catch on). Hardly an issue of Organic Letters goes by these days without some gold-catalyzed cyclization in it. But there are some elements that have never been in fashion, and odds are that they’re never going to be.
Tellurium comes to mind. It does some interesting reactions, and if it wasn’t rather poisonous and if its compounds didn’t stink beyond the ability of anyone to stand them, I’m sure that we would have discovered even more. But it is and they do, and there’s no way to stop either one, so no one’s going to make the effort any time soon. It’s the stench that really seals the deal, actually. Poisonous we work with all the time, but you don’t come across stuff that smells like organotelluriums very often, or so I hear. I’ve never had the pleasure myself.
And as for lab fashions, it’s also safe to say the day of the heavy metals is past. Mercury has a long, long pedigree in both organic and inorganic chemistry – back to the alchemists, actually. Everyone figured that there must be something special and/or magical about a metal that’s liquid at room temperature. They were right, in a way. Mercury does a lot of interesting reactions which are still taught in sophomore organic classes and are still run once in a while. I’ve done a few organomercurations myself, but most of them were years ago in grad school. I’ve only reached for the mercuric chloride once or twice in the last twenty years. That’s doubtless because I’m in the drug industry, but I think that the general use of the element has been trending down because of waste disposal issues. Lead, for its part, never had as much use in the art as mercury, and will probably never get the chance.
It’s not just the heavy metals, either. Beryllium is probably one of the most underused elements in the whole periodic table, as far as organic chemistry is concerned. Considering its spot up near the light end of the periodic table, where all its neighbors are on every lab shelf, you’d think that there’d at least be something you could do with the stuff. But I can’t think of a single reaction I’ve ever seen that uses it. The element’s peculiar toxicity (which mostly seems to be a problem by inhalation) helps keep it out of the spotlight: no organic chemist has ever found a need for it that outweighs its disadvantages, and not many are motivated to try.
None of these are going to be the next hot thing. But what is? Gold’s turn in the organic chemistry spotlight will end at some point – for all I know, things are already slowing down. If I had to guess, I’d pick another candidate from the precious-metal crowd, and I’ll nominate iridium. There are plenty of iridium-based catalysts, but none of them are the absolute first thing a chemist reaches for. It wouldn’t surprise me a bit if the element turned out to have a number of tricks in it that haven’t been discovered yet. They should at least be worth some JACS and Org Lett papers, that’s for sure. . .
As pointed out in the comments to this post, the Menger / Christl pyridinium incident made the mainstream German media in the form of an article in Der Spiegel, which is about as mainstream as it gets. Even a news magazine with a page count as high as that one doesn’t cover organic synthesis very often, so I was curious to see how they handled it. Here’s my own translation of the last part of the article, after Christl sends along the bad news that Menger had rediscovered the century-old Zincke reaction:
Menger reacted reasonably, says Christl. “Fifteen minutes after my e-mail he wrote to me that he would immediately review things”. The American at first believed himself to be in the right: that he really had prepared a new 12-membered ring. “I hate to disappoint you”, wrote Christl, equally sure of himself – and referred to a mass-spectroscopic analysis of the products. These had given a higher molecular mass than a 6-membered ring would. But Menger had overlooked an important detail.
“I am no specialist in analytical methods, but I knew the mass spec method that Menger had employed clusters fragments together", says Christl. During the mass spec analysis, a 6-membered ring could appear to be a 12-membered ring. Menger did not know this effect – “A point we did not realize at the time”, he said remorsefully in the journal Nature. Christl reproaches Menger for imprecise work: “He should have asked a specialist in mass spectroscopy”.
Menger must now send in a correction to his work, and Zincke will be acknowledged. That puts Yamaguchi and Menger hard on Zincke’s trail, says Christl with amusement. Both of them used a particular salt, actually called Zincke salt, in their experiments to prepare the supposed 12-membered rings, but they were obviously completely clueless as to the origin of the name.
But another question remains: How could a 102-year-old reaction simply be overlooked, even though every article in a journal is proofread by an external reviewer?
Zincke had naturally published his work at the time in German. At that time, Germany was a center of research, and English was not yet the official language of science.
But was it really just the language barrier that made the entry into Zincke’s work difficult for Yamaguchi and Menger? Christl says: “This literature is so important, that it’s also given in English”. Maybe not Zincke’s complete original article from the “Annalen”, but at least a description of the reaction in the chemical handbooks. These remain in the libraries.
And here lies the problem, that hardly anyone just looks in the books, complains Cristl. “Young researchers just don’t get up any more from their computers. Most of them don’t even know that such a handbook exists”.
Lack of time and overloading of the reviewers are likely to blame. “They have to proofread a publication, bit by bit, every day. How is that supposed to work?” It seems, over and over, that old reactions are unknowingly rediscovered – because scientists simply don’t do their homework, says Christl. “For well-known scientists, the reading of the literature has become a luxury that they can no longer afford”.
I think we missed a golden opportunity in the penultimate paragraph to learn how to say “young whippersnapper” in German. Applying the term to Fred Menger does require a bit of an imaginative leap, admittedly. I’m glad that Spiegel turned down the chance to make this an “if only people knew German” article, though. But even if Christl’s points about literature searching are valid, I’m not sure that this case illustrates them.
I think this reaction would (should) have been picked up by newfangled tools available to those young ‘uns sitting at their desks. Can you get any more newfangled than Wikipedia? You don't even need SciFinder: a Google search for ("primary amine" pyridinium reaction) will give you this, which should be enough to follow up on. But as for hard-core literature searching, a few minutes of reading the pyridinium + amine literature should have turned up this from 1976 or this from 1970. But why go back that far? How about this review in Angewandte Chemie itself from those far-off days of. . .2006?
No, I don't think the problem here is that people don't know how to turn off the computer and go seek out the good ol' dusty handbook. Those have a lot of good information in them, certainly, and no one's ever cranked out better ones than the Germans have. But the problem here is that people didn't apparently didn't spend any time at all checking the literature. And what's more, the folks who send papers to Angewandte Chemie, and especially the ones who review them, don't even seem to be able to find a key reference published last year in the same damn journal.
The lessons of the recent pyridinium follies are old ones. We’re going to have to relearn them again and again – doomed, if like. That’s because as scientists we’re pulled toward two opposite sorts of error when it comes to new ideas, and because in science, everything comes down to new ideas. We’ll have these problems with us always.
The first error, for which the recent retracted papers are the latest posters on top of a thick, stapled, stack, is to become too infatuated by one’s own ideas. It’s a very easy emotion to yield to. To use an unexpectedly R-rated metaphor, it’s the intellectual equivalent of sexual excitement. Under either influence, potentially dangerous decisions and courses of action can begin to seem reasonable and natural, in contrast to how they might appear in less agitated states of mind. Objections, even quite real and forceful ones, are swept aside as being trivial, fit to deal with later after the important business at hand has been concluded.
The problem is, the best scientific ideas induce this state of mind, and in proportion to their scope. I’ve been hit by a few of these, at my own level, and it’s difficult enough. Think about what goes on up in the heights! Can you imagine what it must have been like for James Clerk Maxwell to tie all of electromagnetism up into a perfectly wrapped gift box with three bows on it? Or for Watson and Crick, looking at their DNA model when they were the only two who’d seen it? That intense joy of discovery, of being right, causes people to behave in strange ways. But it’s one of the driving engines of science and always will be.
By the standards of the great discoveries, these latest cases are trivial – as is most work by most scientists, and all of mine, I hasten to add. But the same principles apply. You look at these things and think “Why didn’t they look into known pyridinium chemistry more? Spend some extra time in the library? Some of those salts are surely crystalline – why didn’t they get an X-ray structure as soon as possible?” All perfectly good questions, from outside, and in retrospect. But any of us could end up brushing aside similarly good questions about our own work, and we shouldn’t forget it.
Now for the other error. The excitement of a new idea has a flip side: the depressed (and depressing) feeling that it must have been done before. Surely this can’t be as good as it seems, otherwise it would be known, right? Most new ideas die. Actually, punishingly near all the new ideas in science die, and most of them die quickly. This spectacle horrifies and numbs many scientists, especially if they have sensitive or fearful natures, and causes them to keep their heads down. No breakthrough, no cry.
If you stay in this mindset long enough, the problem takes care of itself: you’ll train yourself to no longer have many new ideas at all, and you need not face the prospect of watching what happens to them. Unusual, potentially interesting things may happen to your experiments, but you won’t be fooled: into the red waste can they’ll go, along with all the other stuff that didn’t give you what you wanted. Nobel prizes have been poured into red waste cans.
Transportation metaphors are safer than copulatory ones. Discovery, then, is a road with ditches on both sides of it, and the hard part is steering between them. Too much optimism and you go whooping off after junk – or worse, catching it and publishing it after writing your name all over it. Too much pessimism, though, and you never accomplish anything at all. I’ve got mud from both sides of the road on my lab coat – how about you?
There’s a rather embarrassing note leading off the latest issue of Angewandte Chemie. Two recent papers (2007, 2006) had reported the synthesis of some rather weird 12—membered rings, the diazaannulenes shown here. They made them from dinitrophenyl pyridium salts and primary amines, with the pyridine ring unraveling oddly along the way. Too oddly.
Professor Manfred Cristl of Wurzburg, who apparently knows his pyridinium chemistry pretty well, recognized this as an old way to make further pyridinium salts, not funky twelve-membered rings. He recounts how over the last couple of months he exchanged awkward e-mails with the two sets of authors, pointing out that they seem to have rediscovered a 100-year-old reaction, and have they really looked at their spectral data closely, eh? Both groups have admitted their mistake – the data match up wonderfully with the known pyridinium compounds, unfortunately, so there’s really no other way out – and retractions are appearing.
He raises some broader points, though: first, there’s the obvious problem that this whole thing should have been caught by better literature searching and analytical chemistry. These arresting structures deserved more than a quick NMR and LC/MS, and they deserved more than what appears to have been a not-very-thorough look through the prior art. There’s a bigger problem, though, which fans of the LaClair imbroglio will enjoy. Note the exasperated tone of the following, which comes across in a very German fashion:
”A further question refers to the reviewing of the above papers. Presumably, at least four referees were entrusted with this duty, two of Angewandte Chemie and two of Organic Letters. They have provided conclusive evidence for their lack of knowledge of heterocyclic chemistry. However, the referees are probably chosen by the editorial offices according to the specialization of the corresponding authors and, thus, have the same gaps in the knowledge as the authors. In consequence, if the authors present results remote of their main projects, extreme misjudgments are inevitable. . .”
So, once again, Angewandte Chemie's reputation is upended by sloppy refereeing and editing. This time, though, they run an article berating themselves. Progress, I'd say. . .
(Note: update on this story here, and why it might have happened, here).
Now we open the sedate, learned pages of Nature Methods, a fine journal that specializes in new techniques in molecular and chemical biology. In the August issue, the correspondence section features. . .well, a testy response to a paper that appeared last year in Nature Methods.
“Experimental challenge to a ‘rigorous’ BRET analysis of GPCR oligimerization” is the title. If you don’t know the acronyms, never mind – journals like this have acronyms like leopards have spots. The people doing the complaining, Ali Salahpour and Bernard Masri of Duke, are taking issue with a paper from Oxford by John James, Simon Davis, and co-workers. The original paper described a bioluminescence energy transfer (BRET) method to see if G-protein coupled receptors (GPCRs) were associating with each other on cell surfaces. (GPCRs are hugely important signaling systems and drug targets – think serotonin, dopamine, opiates, adrenaline – and it’s become clear in recent years that they can possibly hook up in various unsuspected combinations on the surfaces of cells in vivo).
Salahpour and Masri take strong exception to the Oxford paper’s self-characterization:
“Although the development of new approaches for BRET analysis is commendable, part of the authors’ methodological approach falls short of being ‘rigorous’. . .Some of the pitfalls of their type-1 and type-2 experiments have already been discussed elsewhere (footnote to another complaint about the same work, which also appeared earlier this year in the same journal - DBL). Here we focus on the type-2 experiments and report experimental data to refute some of the results and conclusions presented by James et al.”
That’s about an 8 out of 10 on the scale of nasty scientific language, translating as “You mean well but are lamentably incompetent.” The only way to ratchet things up further is to accuse someone of bad faith or fraud. I won’t go into the technical details of Salahpour and Masri’s complaints; they have to do with the mechanism of BRET, the effect on it of how much GPCR protein is expressed in the cells being studied, and the way James et al. interpreted their results versus standards. The language of these complaints, though, is openly exasperated, full of wording like “unfortunately”, “It seems unlikely”, “we can assume, at best” “(does) not permit rigorous conclusions to be drawn”, “might be erroneous”, “inappropriate and a misinterpretation”, “This could explain why”, “careful examination also (raises) some concerns”, and so on. After the bandilleros and picadors have done their work in the preceding paragraphs, the communication finishes up with another flash of the sword:
In summary, we agree with James and colleagues that type-2 experiments are useful and informative. . .Unfortunately, the experimental design proposed in James et al. to perform type-2 experiments seems incorrect and cannot be interpreted. . .”
James and Davis don’t take this with a smile, naturally. The journal gave them a space to reply to the criticisms, as is standard practice, and as they did for the earlier criticism. (At least the editors know that people are reading the papers they accept. . .) They take on many of the Salahpour/Masri points, claiming that their refutations were done under completely inappropriate conditions, among other things. And they finish up with a flourish, too:
"As we have emphasized, we were not the first to attempt quantitative analysis of BRET data. Previously, however, resonance energy transfer theory was misinterpreted (for example, ref. 4) or applied incorrectly (for example, ref. 5). (Note - reference 4 is to a paper by the first people to question their paper earlier this year, and reference 5 is to the work of Salahpour himself, a nice touch - DBL). The only truly novel aspect of our experiments is that we verified our particular implementation of the theory by analyzing a set of very well-characterized. . .control proteins. (Note - "as opposed to you people" - DBL). . . .In this context, the technical concerns of Salahpour and Masri do not seem relevant."
It's probably safe to say that the air has not yet been cleared. I'm not enough of a BRET hand to say who's right here, but it looks like we're all going to have some more chances to make up our minds (and to appreciate the invective along the way).
I had the opportunity the other day to take a look at the statistics for journal use from the library where I work. It’s the time of the year when they figure out which journals they need to subscribe to, as opposed to just paying per-document fees for individual papers.
That means that several factors go into the decision. The first is usage of the journal. If a lot of papers are downloaded from a given title, odds are that it’ll be cheaper to subscribe. Unless, of course, the subscription rate is completely exorbitant – but that’s certainly not unheard of in the academic publishing world, is it? So in those cases, you’d be better off paying per paper – unless the journal makes that so expensive that a subscription starts to look like a bargain. It’s a balancing act.
Several trends were apparent. The big-name big-impact journals are impossible to ignore, and if you’re a serious research site, they’re impossible not to subscribe to. You can’t have pretenses to keeping up with the latest results if you don’t have Nature, Science, Cell, and the like coming in. And you can’t ignore titles like the Journal of Biological Chemistry, either – sure, they publish eight zillion papers per year, but they get an awful lot of things that didn’t make it into those top-of-the-heap titles, and a lot of good stuff appears there.
In my particular field, the American Chemical Society journals come out pretty well. The subscriptions aren’t cheap, but they aren’t in the white-knuckle range of some of the more commercial publishers. And they get a lot of use – well, the main titles do, anyway. As for the other chemistry journals, Angewandte Chemie isn’t too cheap itself, but it’s also in the “unignorable” category. For a drug research shop, you can say the same thing about Bioorganic and Medicinal Chemistry Letters. There’s a lot of junk in there, but there’s also a lot of intelligence about what your competitors are up to – or were up to a while ago, anyway.
Who comes out looking bad? Well, I don’t know about other research sites, but our figures didn’t look good for either the “Expert Opinion” publications or the Bentham journals (“Current Whatever Whatever”). The latter had an especially large disconnect between the number of paper requests and the corresponding cost of a full subscription, which fits with my own experience. And yours?
We're in the final stages of moving into the newest version of Stately Lowe Manor. And that means, among many other more useful things, that I'll soon see my literature files for the first time in a few months. I've missed them.
Of course, paper files are slowly turning anachronistic, in the same way that paper libraries of the scientific literature are. (I've mused on their disappearance before). I now accumulate piles of PDF files and the like, scattered among folders on my hard drive(s). How to organize them?
That's what I'd like to ask people. I've come across reviews of various programs that are supposed to help with this kind of thing, but there are surely others that I don't know about. What I need is something that will cross-reference papers and graphics (in any format) as well as things like Excel files and such, and allow me to draw and return to connections between them. (At this point, I'm being paid for ideas as much as for anything else, and this is how I seem to generate them). There are several tools that can be made to do this work, with varying degrees of ease and efficiency, but I'm looking for something that's built specifically for the purpose.
One possibility is Yojimbo, which I haven't tried out yet. The ability to write down notes and ideas, with the relevant papers appended to them, is appealing. Does anyone out there have experience with this one, or with its competition? (And for that matter, if there are other ways that people find useful for generating interesting ideas, I'd be glad to hear about those, too. . .)
It might not always be obvious from the blog, but I've been rather literature-deprived lately. I'm not talking literature-literature, naturally - my favorites are all over here on the shelf to my left (For anyone who's interested, those would include Nabokov's wonderful Pale Fire, Brad Leithauser's underappreciated Hence, Martin Amis's best, which is Money, his father Kingsley's immortal Lucky Jim, Chesterton's The Man Who Was Thursday, and Evelyn Waugh's Decline and Fall). No, what I'm beginning to starve a bit for is full-text scientific literature, which is understandable, given that (a) I've been unemployed since the beginning of February, and (b) full-text subscriptions to all the major journals would cost me cash that I'd rather hose away on frivolities like my mortgage.
I do have some recourse to the local universities, where I've gone, uh, periodically to catch up on things. The same processes I saw at work in the Wonder Drug Factory's library are at work in these, too, of course: dwindling shelf space for paper journals and increasing numbers of flat screen displays in their place. It's a bit of a shame for a person like me, who grew up in the era of dead tree journals, because I still like to pick them up in my hands (and because the experience of reading them online still isn't as pleasant as it no doubt will be eventually).
What I've noticed is that the most widely read ones remain in paper as well as digital subscriptions. It's becoming a clear sign of respect for a journal's influence. That means Science, Nature and the like are always still to be had physically. Chemistry libraries always seem to have JACS, Angewandte Chemie and, interestingly, Organic Letters in hard copy, which is probably a good sign for the latter.
So I've been making due with the open-access journals, few of which have strong chemistry ties as yet, and with abstracts and individual free-access articles from the others. Which reminds me - does the ACS ever make anything from its journals open access? Fly-by-night rags like the NEJM or PNAS will open up the most-discussed papers in each of their issues to give them wider exposure, but I've never heard of that happening with, say, JACS. Maybe they figure that the most-discussed articles there are going to be read largely by people that they can get to pay for them, anyway, so why bother?
I had an e-mail the other day from a reader who pointed out that when anyone tells you something is simple or obvious in indole chemistry, that you should run for cover. He expressed similar worries about amino acids, and I take his point. There are some areas of organic chemistry that have huge encrustations of literature all over them, which can be a warning sign.
One way you get a literature jungle is for few general methods to work. That's always been my impression of indole chemistry - each reaction has its limited area of jurisdiction, but if you start messing around you find yourself in the lawless borderlands pretty quickly. I did my PhD work using carbohydrates for natural product synthesis, and I have to say that they've got the same problem. All kinds of interesting, funky things start happening when you do something so (apparently) innocent as using a galactose scaffold instead of glucose.
Of course, there's another way to pile up a lot of different reactions, which is for most all of them to work, more or less. I think that's been the situation for many years in the palladium-catalyzed reaction field. As I've said before, I believe that just about any Pd coupling can be made to work, if you're willing to devote enough of your life to tweaking the conditions. Can you imagine a comprehensive review of some of these reactions? You'd need a special nine-volume run of Organic Reactions to even have a hope, and where would you find the maniacs to write it all?
Readers are invited to submit other examples of literature swamps - another one I can think of, from the old days, are all those steroid transformations. But I know that there are others lurking out there. . .
I have boxes and boxes of files here at home these days, the contents of my filing cabinets in my former office. Obviously, I heaved the proprietary stuff before I left, but I still have plenty of folders full of papers from the literature. Some of those went into the dumper as well, though, as I pulled them from the cabinets and realized how old they were. The biology-based folders were the main candidates - stuff on Alzheimer's from the early 1990s, for example. Old NMR manuals and such got the heave, too.
The chemistry files have held up better, although some older reviews went into the shredder. I still have the first real journal articles that I ever copied off, from my undergraduate days back in 1981. These were a series of articles from the late 1970s by two guys named Burfield and Smithers on the best ways to dry common solvents. They're looking a bit tattered these days, but the information in them is still valid.
And there's another old folder that I'll never throw out. It has my continuation exam material in it from grad school, and it looks like something made by a caveman. The bonds are drawn with pen, using a plastic template, and the atoms are the good old rub-on letters. You used to be able to buy sheets of those things from Aldrich - standard rub-on sheets didn't have the letters biased toward common atoms and tended to get used up too quickly. When you messed something up with the template, you either did the whole thing again, or used some correction fluid. When you messed up with the letters, you scraped them off with a razor blade. The whole process was much, much, closer to making gouges in a tablet of pressed buffalo dung and leaving it to dry in the sun than it is to using Chemdraw.
And that's why I'm keeping them. When I get frustrated with some device or technology, I try to remind myself of the days when a page of structure drawings involved Scotch tape, ball-point pens, and razor blades. I just barely overlapped with that era, but it was more than enough.
If you're on the editorial staff of J. Med. Chem., and you've got one of those "Perspective" review articles to go over, and it's on the very important (and very complex) topic of the binding of kinase inhibitors, something that's going to catch the eyes of lots of people all over the place. . .wouldn't you (and your referees) want to make sure that the paper has the correct structures in it? Even down to the level of obscure drugs like, say, Gleevec? Kinase Pro is just asking. . .
So, what's this "PLoS One" thing I was talking about, anyway? PLoS is, as many will know, the acronym for the Public Library of Science, one of the beacons of the open-access science publishing movement. They had about half a dozen journals (almost entirely in the medical/biological fields) until recently, when they added PLoS One.
No, it's not the one with fewer calories. I'm not sure why the name was chosen, except perhaps as an attention-getting device. (Is there going to be a PLoS Two?) It is a fairly radical publishing move, establishing something that's part preprint server, part refereed journal, and part user-ranked content site. Papers can be submitted in just about any area of science, and will be checked to make sure that they're methodologically sound - that is, that their conclusions can reasonably be drawn from the evidence that they present. And that's it. Once past that point, everything gets in.
So how do you find out what's worth reading? Here's where the user-generated part comes in. You can leave comments on any aspect of any paper, and as long as they're presented reasonably, they're in to stay. The more comments/recommendations a paper gets, the more attention it will continue to draw. And (although they're not enabling this yet), there will be a ranking system, where readers can assign scores to each paper they're read, with visible aggregated ratings: Science meets Slashdot (or Digg, or Reddit, and yeah, I know that readers of each of these sites spend a fair amount of time making fun of each other). The PLoS people are bypassing most of the debate about how to referee papers, setting up what's essentially a garbage filter and letting the readers sort things out after that.
Papers can also be annotated, with comments attached to specific points in the manuscript. This (and the rating system) are the parts I'm most interested in seeing in action. As far as I can tell, they're going to have anonymity, although you'll have to register (confidentially) to use these features. The guidelines for commenting and annotating seem reasonable:
1. Language that is insulting, inflammatory, or obscene will not be tolerated.
2. Unsupported assertions or statements should be avoided. Comments must be evidence-based, not authority-based.
3. When previously published studies are cited, they must be accurately referenced and, where possible, a DOI and link to a publicly accessible version supplied.
4. Unpublished data should be provided with sufficient methodological detail for those data to be assessed. Alternatively, a permanent Web link to such information should be provided.
5. Arguments based on belief are to be avoided. For example the assertion, "I don't believe the results in Figure 2." must be supported.
6. Discussions should be confined to the demonstrable content of papers and should avoid speculation about the motivations or prejudices of authors.
I can see that they've devoted some thought to what might happen. I think that this will be a critical-mass phenomenon - if enough papers get annotated and ranked, it'll become the norm. And if not, these features might wither on the vine, which would be a shame. I've registered with the site as of this morning. Let the experiment begin!
Late last year, Nature made some noise with their experiment in open peer review. They invited authors, if they wished, to allow their submitted manuscripts to be opened for comments online. As the journal's summary of the process makes clear, though, very few were willing to try it. (My look at the situation back in September wasn't very encouraging, either).
Only 5% of the papers sent out for review during the trial were made available by their authors, and these 71 papers went on to attract very little notice on the peer-review site itself. The editors counted 92 comments total, despite decent amounts of traffic and numerous attempts to drum up interest. As an aside, the statistics would seem to indicate a power-law distribution, which should surprise no one, with a few papers getting a strong plurality of the comments and many getting none at all.
The experiment has ended, and Nature is through with the idea of open peer review for now. But you still have to wonder if there's merit there - and if so, what flaws led to this underwhelming performance. I'd like to suggest one gigantic, ground-shaking factor, which sat right down and made itself at home immediately: lack of anonymity. Does anyone doubt that one reason that traditional peer review works (to the extent that it does) is its anonymous format? Would people be willing to say the things that they do about papers that they're refereeing if they knew that the authors would have their e-mail address and phone number?
I sure wouldn't. In my career I've reviewed for several journals, and even though I've let more papers through than I've rejected, sometimes my assent has been less than enthusiastic - on the order of "Well, this isn't too exciting, but you publish lots of other things just like it, so why not?" And I'm pretty hard on abuses of literature citations - like not citing clearly anticipatory work from some other group, or results which challenge the paper's own conclusions. "Publish only if they fix this part" is a common report from me in those cases. When I do reject a paper, it's not because it's a borderline case, though, because I try to let those through (with suggestions for improvement).
I would feel very inhibited indeed if I knew that the authors of the papers I've criticized were able to contact me directly (or to file my name in their "To Have Dirt Done To" folder). Many more comments would have come in during Nature's experiment if people had the ability to use anonymous screen names (and had the assurance that their real identities would not be revealed, just as in standard peer review). The journal could have reserved to right to delete libelous comments, of course, which in Great Britain is a pretty wide mandate.
But how many authors would have willingly offered up their papers to such a process? Probably far fewer than the 5% who tried system as it was, I'd say. But think of how our current peer review system would go over if we didn't have it already. "You're telling me," says the distinguished first author, "that you're going to send my paper out and let various unknown people rip into it? And you'll never tell me who they are? And I don't get a say in any of it? And this is going to be a major factor in whether my paper even gets accepted?" Imagine dropping that on an unprepared scientific community. It'd never fly
Aaron Haspel over at God of the Machine took some good cuts at the English-Comp warhorse "The Elements of Style" the other day. He's good at invective, and the book deserves some, although arguably not as much as he had on hand. But then his post inspired another at a blog called Petrona, written by Maxine Clarke, a senior editor at Nature. Her take:
I don't have much, if any, problem with (Strunk and White). Scientists do well to follow the advice when writing up original research, because descriptions of technical concepts, methods and so on are vastly improved by brevity. In particular, the common habit among US authors of applying six or seven adjectives to a noun can be very hard to comprehend when many of the adjectives could equally well be nouns, and the whole consists of polysyllabic specialist terminology (oh, OK, then -- jargon).
Given her day job, I have to assume that she knows of what she speaks. That's especially true because Nature is one of the very few scientific journals to do real line editing - they'll take your manuscript and rework it after it's been accepted, as strange as that may seem to many people.
Scientific writing is notoriously poor. Some of the problem comes from younger scientists trying to emulate what they've already been exposed to. I remember a colleague of mine in the early years of my first job who couldn't have written a report on whether it was raining and make it in under ten pages. I remember talking with this person about their draft of an internal report, which spoke about how they'd systematically investigated the various steric and electronic factors involved by varying the substituents in the distal portion of the aromatic ring in an attempt to learn the effects of these variations on a number of parameters, including oral absorption, activity at the target, clearance, and selectivity, and. . .well, it went on like that, for quite a long time.
"What are you trying to say here", I asked. "Oh, I'm just saying that we did the SAR for the 4-position of the ring", was the reply. "Then say that" was my advice. Ruthless application would have trimmed things down by about 90%, but no, it wouldn't sound like a real report then, would it?
Some of the worst writing in the scientific journals, though, comes from people who are trying to turn out the best. I've seen several people who are overly impressed with their writing skills, and try to dress up their papers with knotty sentence structure, recondite vocabulary, and other cheap tricks. Unfortunately, many readers fall for it. If they can recognize the author's style, they figure, he must be some writer. A journal article doesn't give you much room for style, that's for sure. Having an individual voice for your publications is a real challenge, but here's the trouble: most of the ways you can do it are bad ideas. Better to have no style at all than a lousy one.
Back when I was a first-year graduate student, I had to do something that I'm not sure that folks today have to worry about: pass a German test. Mind you, it wasn't much of a test - you got a passage from a journal article, and could use a dictionary, and you had a couple of hours. Fast page-flipping would get you through it, which is basically how I did it, since I'd only had one semester of the language as an undergrad (and not much of it took). Little did I know that I'd have a year coming up when I'd have to speak the language in order to eat.
You couldn't substitute another language, either, because German is a uniquely important one in chemistry. A lot of the older physical and inorganic (and a huge amount of the early organic) work was done in Germany, which also produced huge reference works like Beilstein, Gmelin, and Houben-Weyl. But perhaps all the verbs in those sentences should be in the past tense, because both of those references are now appearing in English.
Beilstein switched over with the 5th printed supplement, which appeared only after massive delays which led many scientific libraries to give up on their subscriptions. At one point, the print edition was a good thirty years out of date. Organic grad students had regarded Beilstein with awe back in the 1950s and before, but by the 1980s many of them had never used it. The switch to electronic database searching, which was done in English right from the start, brought them back to relevence. Now libraries are having to remind people that the computer-based service used to be part of a printed handbook.
Houben-Weyl, for its part, switched to English in 1990 or so, but that doesn't seem to have raised its profile in the non-German-speaking world. I recall a Dylan Stiles post where he didn't seem to have heard of the work, for example. The publishers finally caught on to the fact that printed reference works are in trouble, and have moved into the electronic age.
So, here's a question for the grad-student readers: does anyone have to take a German exam any more? The importance of the language in chemistry has been in steady decline for decades, and (if anything) accelerated decline for the last fifteen years. And if you do have to take a test, does anyone at your department still know why?
It's taken a while, but a traditional science publisher is starting to make the leap to blog-style comments for scientific papers. Nature has begun offering authors the option of having their paper commented upon by the teeming masses of researchers while it's still in the review phase. (Tyler Cowen speculated on the pluses and minuses of this idea earlier this year). (Update: and does again here in response to an article on this very experiment).
Nature's been talking about peer review and its evolution for a while now - see this section and this blog at their site. They seem to have decided that debate is all very well, but that we're not going to know how well these ideas work until we put them into practice, thus their peer review trial site.
So, how's it working? Looking it over, I can see (as of today) ten current papers that are available for comments, posted roughly since the beginning of this month. None have attracted any comments at all, which is a situation that many bloggers will be all too familiar with. The site, though powered by Movable Type, doesn't seem to have date-driven archive pages as such, although it does have categories. Looking at the "Recent Comments" sidebar, though, will take you back to the last paper that attracted some, which was posted on August 29th. The navigation links at the top of its page will then take you back, paper by paper.
Digging through the stack in this manner, the only papers with substantial comments are found here, here, here, here, and especially here. That takes us back to early July, and the first papers seem to have appeared about a month before. The great majority of papers have attracted no comments at all - I wonder what sort of traffic the site is getting?
It's interesting to compare the behavior patterns there with those at a regular blog. There are a few "nice paper!" one-liners, which out here in the rest of the world are the sign of spam, but which appear to be sincere (if not very useful) communications on the Nature site. The comments are moderated, and I'd like to know just how many they've had to excise. I ask because there are still some off-topic oddities that make it through, like this, where a Chinese researcher makes a rambling complaint of harassment by his government. And there's a planetary science paper here, two of whose five comments are clearly by loons, which must please the original authors no end.
Where real comments appear, they're often done in the style of a peer reviewer, starting with the obligatory "The authors present interesting data on the. . ." type of sentence, and going on in rather stilted fashion. It seems clear that very few of the commentors have much exposure to the regular blog world, or if they do, they're taking great pains to not allow any of that experience to leak over into the exalted world of Nature.
I wish them luck with the experiment. What I'd like to see is an idea that's been proposed before, but never implemented in chemistry: comments on papers after they've published. Think of how interesting Organic Letters would be with comments after each paper in the table of contents - heck, I'd go all out and put a hit counter on each paper, so you could see what's getting attention and what isn't. Does the ACS have the nerve?
Today while walking through the library I passed the shelves where the Chemical Abstracts indices still sit. They don't appear to have been touched in some time - I certainly haven't disturbed them.
CA is the repository for abstracts of almost all the papers that bear on chemistry throughout the world's scientific literature, and for data on all the chemical compounds that are described in them. It's one of the largest compendia of reference data in the world, as you'd expect. For many years, it was available only as a print edition, because, well, everything was available only in a print edition.
Every five years a collective index would be issued, to great rejoicing. These included indices by author, by chemical formula, by compound name, and so on. I remember when the 10th index came out, which covered the 1977-1981 literature, so I guess I go back to the 9th edition, although I'd never had much need (or ability) to use it while it was current. The coverage of the 10th index, though, ran from a year when I knew little or no chemistry to a time when I'd finished my sophomore organic course and had some idea of what the papers it referenced were talking about.
The 11th came out while I was in graduate school, and man, was I glad to see it. The 10th had become rather ancient in the meantime, and my need for access to the literature had reached unheard-of levels as I wrestled with my PhD. Digging through the more recent volume-by-volume indices to catch up had become quite painful. But it in its turn began to show its age. The 12th collective index showed up after I'd been in industry for three years or so, and I was glad to see it as well. We had a library staff that would look things up for you, but I still found them no substitute for going down and digging around firsthand. I gave that edition a good workout, and I remember being quite distressed when the rows of softcover volumes disappeared for a while for binding.
But things changed. The 13th index, from 1992 to 1996, is the last one whose physical covers I ever opened, and I think that was only once or twice. The library here never even bothered to get it hardbound, and the big softcover volumes are slumped against each other on their shelves. I have never even seen a copy of the 14th index, and I wonder how many copies of next year's 15th will even be printed. Whether there will even be a hard-copy version of the 16th in 2012 is anyone's guess, but I'd be willing to bet against it.
What happened, naturally, was the machine that you're using to read this blog. Even while I was in graduate school, you could access Chemical Abstracts via a command-line interface through one of those rockin' 1200 baud modems, and I was the person in our research group designated as the high priest. There was one terminal in the library hooked up, with a special key to open the door. I still have copies of the manual pages somewhere in my files. I'm kind of surprised that I don't have the key as well, but they must have made sure that it was turned in.
The first time I used CAS Online, I felt as if I'd been given magical powers. Wildcards! Variable atoms and chain lengths! I talked everyone's ears off about the kind of searching that could be done, and as I recall, the big issue was convincing people that the online method differed not only in degree, but in kind. You could, of course, do things that were completely impossible to do with the print edition, a fact that is mostly appreciated now by people who are at least 40 years old. That is, people who have done it the old way. Fewer and fewer do, and good riddance to it.
If you'd parted the curtain of Time and shown me 2006 literature searching on a 2006 computer, I'd probably have fainted. Similarly, if you'd told me back in 1985 that I never would see the 14th CA Collective Index or any past it, I probably would have taken it as a prophecy of nuclear war. Something better happened instead.
I haven't thrown a question out to the readership in a while, and this one has been on my mind. Let's give it a try: what journal(s) would you just as soon see vanish from the earth?
A few ground rules: let's try stick to organic chemistry and associated fields, for now. If we start slamming the other scientific disciplines, we'll never get things under control. If this is a popular feature, we'll take on the others in due course. And I'd suggest these criteria for marking a journal for death:
1. You don't read it regularly.
2. You don't even read it occasionally.
3. You assume that any paper in it wasn't good enough to appear somewhere else.
4. You can't recall the last time you even got some reference data from it.
Those are pretty stringent criteria. For example, I never look at Synthetic Communications, but I do occasionally get a procedure from it (and they occasionally work). I picked up some analogous-compound data from a Heterocycles paper a year ago, so it doesn't quite make #4, but otherwise it's one that I'd mark for the trash heap.
Some people don't seem to believe it, but you definitely can publish too many papers. The problem is, the surest ways of publishing a lot of papers are to not do many things that are interesting or unusual, and to break everything down into tiny pieces. The first requirement is there because unusual results take time - time for you to believe them in the first place, time to replicate them, time to try to explain and extend them. And no matter what, eventually they aren't unusual results any more - they may still be interesting or useful, but the surprise has worn off, as it has to. So there's no one, really, who publishes a lot of fascinating unexpected stuff all the time, because it just can't be done.
The second requirement is the least-publishable-unit problem, which many people succumb to. After all, if you've got some red-hot results, why wait for the full paper? Bang out a communication before someone scoops you. Trouble is, people bang out communications all the time on things where being scooped is the last thing that could ever happen. As I put it once, many of these things could only get scooped in the sense of someone cleaning up after their dog. Short papers don't stick in the memory very well, either. If the stuff never gets condensed and systematized - that is, written up in some full papers - it's hard to get enough attention for it even if there is something worthwhile stretched across the publication list. It would have had a bigger effect were it not so diluted.
Even famous scientists have fallen into this trap, and I would like to adduce the late H. C. Brown as a shining example. Who, during the 1970s and 80s, did not groan on seeing yet another paper from Professor Brown? Variation after variation on his boron reagents poured forth, each with slightly different characteristics and reactivity, later superseded by other variations in the endless series. And the thing is, there are a number of real advances in there - the man didn't get the Nobel for nothing. But there's an awful lot of work that has, to put it kindly, not stood the test of time. Not everything he and his group did was worth being published.
So if a Nobel laureate can tarnish his reputation by acting like his own printing press, imagine what it does for less famous authors. Be quiet, you feel like whispering to them, be quiet until you've got something to say. . .
I had a promotional e-mail from the scientific publishing giant Elsevier the other day. The latest calculated impact factors for journals have been released, so it's time, naturally, for them to brag about how things are going.
The message goes on, in large type, about "Consistently Increasing Impact Factors!". I guess that it's nice to know, for example, that Bioorganic and Medicinal Chemistry Letters has moved up from 2.333 to 2.478. Hey, at that rate, they'll be over 3.0 by the 2010s! O, brave new world that has such journals in it! Imagine being able to unload your failed med-chem projects in a journal with such an impressive impact factor - I'd encourage you to start making plans on how you'll spend the bonus and promotion money.
Even by these (debased) standards, some of the hype seems a bit. . .forced. Bioorganic Chemistry, for example, is touted as moving up from 1.240 to 1.565 (translation: unimpressive to unimpressive, even if you believe in impac factors). Those numbers make me think that I still have several years of lead time before I'm strongly motivated to look at the journal.
But my favorite blurb is this one: "Heterocycles: WAS 1.064. NOW 1.070". Well, all right, then, spread the news! The impact factor for Heterocycles has moved up in the third decimal place! What, did three more people cite papers from it in 2005? Look, Elsevier knows the truth as well as anyone else: Heterocycles is just not a good journal. But then, it never has been. Back in the 1970s and 80s, it came directly from Japan on expensive glossy paper stock, which along with the sleek black cover made a distinctly weird impression on those infrequent occasions when you actually looked inside a copy. The paper completely outclassed that stuff that was printed on it. Most of the articles could have been titled "Not Particularly Surprising Rearrangements of Bicyclic Imidazo Compounds That No One Cares About". I have seen no evidence that makes me think that the situation has improved.
Actually, my favorite part of the e-mail is what it doesn't mention. You know, when you think about it, Elsevier publishes some other chemistry journals, too. . .where, for instance, is Tetrahedron Letters? You don't suppose they'd miss an opportunity to highlight that one, do you, assuming that there was anything to highlight?
There's an interesting scandal brewing in synthetic organic chemistry - well, actually, more than one, but I haven't covered the Sames matter at all. This is a new one. Back in February, Angewandte Chemie, one of the most prestigious outlets for organic synthesis we have, published online a paper by James J. La Clair on the total synthesis of a nasty molecule called hexacyclinol, originally isolated from a Siberian fungus.
The paper is remarkable in several ways, and not just because I'd never heard of La Clair. The synthesis is over 30 steps long, which is unfortunately not as uncommon as it should be. (I'm afraid that my bias against total synthesis is showing). But La Clair is the only author, which is highly unusual for such a large effort. And it must have been a large one, since the paper makes reference to starting on a molar scale and finished with over three grams of the penultimate intermediate. Experienced organic chemists will wonder if two or three decimal points have been misplaced there, but that's what it says.
Here's a paragraph for my fellow synthetic geeks - everyone else can skip ahead. When you read it closely, this synthesis has some pretty odd steps in it. One oxidation (aldehyde to acid in the presence of a dithiane) is accomplished through the slow addition of silver oxide in paraffin wax, of all things. If that's a reagent combination that's ever appeared in the literature, I've missed it. Silver oxide, sure - but not delivered by a cheese grater. There's a Mitsunobu inversion, via thiophenol, which occurs on a brutally hindered tertiary alcohol, which is certainly not something I'd expect to happen, or count on midway through a thirty-odd step route. A bit later, La Clair has a mesylation that's accomplished by adding methanesulfonyl chloride/triethylamine once an hour for five hours, which is sort of believable, as the kind of thing that you're driven to by frantic experimentation, but still a bit odd-sounding.
As mentioned, La Clair is the sole author, with an address given at the Xenobe Research Institute. The usual reaction to that statement is "The what?", as I've found empirically by wandering down my hallway at work. (Or, as Stiles puts it, "not to be confused with the Scientology outpost in low orbit around Mars") Xenobe's site is a bit odd, giving off the distinctive feel of a one-man operation. I particularly like what happens when you click the "Support" button and are informed that the Institute is not accepting donations at this time. Before Xenobe, La Clair was at Bionic Bros. GmbH, in Berlin, which sounds unavoidably like a firm from a William Gibson novel. This is where much of the synthesis was done, according to a footnote in which he acknowledges, glancingly, "the assistance of five technicians". (In his defense, that's very much the German style of chemistry, for better or worse).
Now we get to the brow-furrowing part. In the preprint section of the ACS journal Organic Letters, Scott Rychnovsky of Cal-Irvine unveils a computational technique for predicting the carbon-13 NMR spectra of complex structures. His test case is. . .hexacyclinol, La Clair's baby. But according to Rychnovsky, the published structure for the natural product has to be wrong. His method seems to work quite well on similar polycyclic terpenoid nightmare structures, but feeding the accepted hexacyclinol structure into it yields a terrible correlation.
So what's the correct structure? Rychnovsky points out that a related species of fungus has been shown to produce another natural product, panepophenanthrin. If that reacted with some methanol and a bit of acid, which might easily happen during the isolation procedure, it would produce a compound with the same molecular weight as hexacyclinol. . .and that structure, run through the NMR predictor, gives a fit that's right in line with the other known cases he used. Rychnovsky's quite sure that his proposed structure is the real structure of hexacyclinol.
But if it is, how on earth did La Clair get the data he has? His paper includes a proton NMR of the natural product and one of his synthetic material for comparison. They're identical. But if Rychnovsky's right, La Clair synthesized the wrong structure entirely. The spectra shouldn't match at all - that's one of the remaining reasons for total synthesis, to make the compound and see if the spectral data really fit. Now, Rychnovsky's argument hinges on the carbon spectrum, but that should be easy to obtain, given the monstrously huge scale that La Clair seems to have been working on. And given the discrepency between the two proposed structures, I can't see how the proton NMRs can possibly line up by chance.
The strangest part of La Clair's paper is its final footnote, added in proof. Here's how it starts; make of this what you will: "The 1H NMR spectra for this Communication were determined by contract services. The spectra provided in the Supporting Information were collected by N. Voss (Berlin, Germany). The operator added the peak for CDCl3 to the spectrum of synthetic hexacyclinol (1), however, this was done incorrectly at 7.5 ppm and against the request of the author." That doesn't make a whole lot of sense. The NMR operator "added the peak" for solvent to a spectrum? Why? And he put the peak in at 7.5 ppm (the wrong place, for non-chemists)? With what, Photoshop? No, this is very strange indeed.
One of these guys is wrong. And reading Rychnovsky's paper, it's clear that he's not in much doubt about who it is: "Recently, a provocative synthesis of hexacyclinol was reported (footnote to La Clair's paper), and interest in the paper triggered my reexamination of the original structural assignment." By the standards of organic chemistry, that's a gloved slap in the face in the public square. Someone at Angewandte Chemie should probably be feeling the sting, too.
Thanks to Dylan Stiles for calling this business to my attention - his post's comments, which are much more potentially libelous than things tend to get around here, are well worth a read for those interested. Update: La Clair has made an appearance in Dylan's comments, rather to everyone's surprise, I'd say. Still no word on a C-13 spectrum, though.
The Wall Street Journal ran an interesting article by David Armstrong the other day on the New England Journal of Medicine and the Merck/Vioxx affair. It's subscriber-only on the WSJ site, but the Pittsburgh Post-Gazette picked it up here. It brings up an angle that I hadn't completely considered:
While Merck has taken the brunt of criticism in the affair, the New England Journal's role in the Vioxx debacle has received little attention. The journal is the most-cited medical publication in the world, and its November 2000 article on Vioxx was a major marketing tool for Merck. . .Internal emails show the New England Journal's expression of concern was timed to divert attention from a deposition in which Executive Editor Gregory Curfman made potentially damaging admissions about the journal's handling of the Vioxx study. In the deposition, part of the Vioxx litigation, Dr. Curfman acknowledged that lax editing might have helped the authors make misleading claims in the article. He said the journal sold more than 900,000 reprints of the article, bringing in at least $697,000 in revenue. Merck says it bought most of the reprints.
The article goes on to detail the role of a public relations consultant in the release and timing of the "Expression of Concern", which I've expressed my own concerns about. The journal seems to have been worried about its own name, and seeking to put the focus back on Merck. And some of these efforts may have gone a bit over the line. Remember the infamous missing data?
Perhaps the most sensational allegation in the journal's expression of concern was that the authors of the November 2000 article deleted heart-related safety data from a draft just two days before submitting it to the journal for publication. The journal said it was able to detect this by examining a computer disk submitted with the manuscript.
The statement was ambiguous about what data the authors deleted, hinting that serious scientific misconduct was involved. "Taken together, these inaccuracies and deletions call into question the integrity of the data," the editors wrote.
In reality, the last-minute changes to the manuscript were less significant. One of the "deleted" items was a blank table that never had any data in it in article manuscripts. Also deleted was the number of heart attacks suffered by Vioxx users in the trial -- 17. However, in place of the number the authors inserted the percentage of patients who suffered heart attacks. Using that percentage (0.4 percent) and the total number of Vioxx users given in the article (4,047), any reader could roughly calculate the heart-attack number. . .
. . .Many news organizations, including The Wall Street Journal, misunderstood the ambiguous language and incorrectly reported that the deleted data were the extra three heart attacks -- which, if true, would have reflected badly on Merck. The New England Journal says it didn't attempt to have these mistakes corrected.
So, the matter of the missing heart attacks, which was the subject of a lot of heated language around here, appears to be closed. This sheds an interesting light on last December's "reaffirmation" of concern, where the NEJM made so much of the heart attack data and how it should have been included. Just about everyone who read that came away thinking that the whole fuss was about the deletion of the three MI events in the Vioxx treatment group. As you'll see from the comments to that post, many of us spent our time arguing about whether they should have been included or not, what the clinical cutoff date was, and so on.
We could have saved our breath. The heart attacks weren't deleted from the manuscript, and those who thought that they had been were responding to a well-thought-out public relations campaign. My opinion of the NEJM is not being enhanced by these revelations, let me tell you.
Problem is, my opinion of Merck isn't at its highest level these days, either. More on that tomorrow. . .
So, what's the problem if some of the yields in total synthesis papers are a bit. . .enhanced? If the reactions worked, why get worked up about it?
Well, aside from nagging thoughts of intellectual honesty, there's a practical implication. Even if no one ever tries most of these reactions again - and believe me, no one will - the problem with advertising our mighty chemical powers is that people who don't know any better will believe us.
You can start with other chemists. We've all heard stories about people who've tried to repeat reactions from the Famous Labs of one or another Famous Professors and been unable to get them to work. It's to the point that synthetic chemists with some experience automatically discount the yields that they see from some groups, because they know that they can't reproduce them. (See the comments to yesterday's post if you want some concrete examples). The ones who haven't heard, though, will continue to discover the nasty reality on their own, often wasting their time and effort in doing so.
But I think the serious trouble starts when we get outside of the field. I've worried for some time that synthetic organic chemistry has been in danger of making itself seem more powerful and efficient than it really is. This can be a noticeable problem in industry, where you deal with molecular biologists, toxicologists, and other people who have to take our word for things. And if what they hear about are mighty synthetic chemists banging out mighty molecules in mighty impressive yields, what kind of reception do we mortals get when we tell them that we're having trouble making their lead compounds in enough quantity?
We really don't want to make people think that we can make everything, you know, because we can't. Not in any real-world sense, we can't. Experienced synthetic chemists all know this, because we've all been humbled by comparatively simple molecules and supposedly easy reactions. These things are smarter and trickier than we are, and they'll remain so for a long time to come. It would go easier on us if people outside the field knew that we can't just magically deliver on whatever they'd like us to make. And who knows, perhaps if word got out that organic chemistry isn't a tapped-out all-tied-up field, we'd get some more good people to come help with it.
Thinking about the scientific literature, I got to wondering: which part of the chemistry literature is the most unreliable? (That is, fitting that "pessimist universe" model from my post the other day). There's been a recent bit of trouble in the organic chemistry world that I've been meaning to talk about, so I thought I'd lead up to it this way.
If you look at what organic chemistry papers have been retracted over the years, my strong impression is that most of them have been papers on new synthetic methods. That might make you think that this is the weak spot, but I'd disagree - I think that these stand out because people actually read them and try to reproduce them. An interesting new method is going to be tried out by others, and it'll be clear very quickly if there's a problem. I wrote about such a case on my old site, back in 2002:
". . .There's a paper out now (which for those of you in the field is Synthesis, 29, 2002), that belongs to the select group of articles whose sole purpose is to demolish another one. The now-discredited article was in the same journal over a year ago, presenting an interesting reaction that I thought we could make use of in my lab. (Keep in mind that the classic definition of "interesting" in the scientific literature is "interesting to me!") We actually tried the chemistry out. It flopped cleanly and completely, giving exactly the wrong product. I chalked it up to the weirdness of our current compounds, which is not to be underestimated. Some things work on them; some don't. We poured the reaction into the red waste can and did something else, which is one of the things I like most about medicinal chemistry.
The author of the latest article, though, had the same thing happen to him, and he didn't take it as quietly. Going back over the original examples, he shows that the published work won't, didn't, and can't go the way it was reported to. Some of this can be put in the "honest mistake" category, subheading "really sloppy honest mistakes," but I'm afraid, in the end, that some of it can't. . ."
Even things that don't look all that useful at the time will often get dragged out into the light by somebody eventually. No, my vote for unreliability is the papers on total synthesis of natural products. In almost all cases, no one will ever do those reactions again. The exceptions are the rare examples where the product is itself useful and the synthesis has industrial relevance (taxol!), or the times when more than one group is working on the molecule simultaneously and using similar approaches.
But how often do such things happen? Mostly, it's "Total Synthesis of Timewastotoxin", which either no one will ever see fit to make again, or if they do will have to use a completely different synthesis (because otherwise, what's the point?) The yield in step 48 could be inflated like a beach ball - who will ever know? Step 33 might be the only run that worked out of 64 tries. In fact, the pressures of such work make it rather likely that these very things have taken place.
I'm not saying that the majority of total syntheses in the literature are incorrect - since (in the end) the synthetic product can be compared against the natural one. Only outright fraud could explain papering over a problem that goes all the way to the last step. I don't think that that's very common at all, although I sure wouldn't bet on it never having happened. (When it has, it's most likely been due to a desperate grad student spiking a sample with the real product).
I supposed a better word than "unreliable" would be "indeterminate". Even though they're probably fine, no one knows if the advanced steps in a long synthesis are real or not. It's just not worth the trouble to find out. It's as if Schrodinger's cat stayed in a superposition of states because it was so hard to open the box, and because no one cared very much whether he was alive or dead anyway.
Keeping up with the literature? I'm clearly not, although I regularly (or semi-regularly) read nearly a dozen journals. There's too much stuff coming out, and has been for a long time. Even harder than just "keeping up" is integrating what you read into some sort of coherent whole. I'm pretty sure that only the greatest scientists have ever done a really good job on that front.
These thoughts are prompted by a recent paper in PNAS by some statisticians at Columbia and Yale. They reference the GeneWays project, an attempt at text-mining the biomedical literature for relevant information. To me, what the project looks like is an automated version of what I've done with papers at various points in my career - sit down and rewrite them into a condensed version that gets across their key points. Doing that forces you to read and comprehend the whole thing, in detail, and explain it to yourself in terms that you understand. It's labor-intensive, but worthwhile when you really have to absorb something you're unfamiliar with.
Here's a look at some of the information typically extracted out of a paper. As you can see, they're going for the basics - what interacts with what. This may seem like trivializing the papers involved, but it's a good first step, as any look into the molecular and cell biology literature will confirm. If you'd like to experience this for yourself, try running the term "NF-{kappa}B" through a search engine like PubMed, pretending as if you had rashly agreed to do a review article on its various biological activities. There are, as of this evening, nine hundred and thirteen pages of search results. Next week there will be more.
The PNAS article uses the GeneWays data set to look at how scientific information is dealt with over time. They searched out statements about the same pair of substances - things like "Kinase W phosphorylates (or does not phosphorylate) Protein X". Then they tracked how these cascaded through the literature. There are, of course, plenty of opportunities for honestly conflicting statements about the same actors, given the variables involved.
But there are probably even more reasons for such assertions to reinforce each other, as the authors point out. The statements could indeed be true, and relatively easy to verify, through experiments with a low error rate. Or some of the later statements could be unverified restatements of the earlier ones: "As is well known. . .". There's also the way that some substances have of interacting with so many things that statements about their involvement with some other molecule have better than even odds of being true just by chance - some kinases come to mind.
So they used a probabalistic model, with parameters for each step along the way. These include possibilities like discarding negative data, performing (or not performing) an independent experiment before publishing a statement, the probabilities of getting false positives or false negatives in such experiments, the probabilities that positive (or negative) data are actually published and the probabilities that other scientists actually read them before publishing their own work, etc.
They come up with a whole range of hypothetical chains through the literature. Turning the dials on the model in different ways can give you strongly conforming virtual scientists, who believe everything they read, strongly argumentative ones who are fond of reversing earlier data, scientists who go with the flow until someone brave publishes a contrary instance and then switch to follow that, and many others. Then there's the style they call "mild scepticism":
"In this hypothetical world, scientists do read their peer's articles and try to compare their own results to the published ones but tend to trust their own results more than the data published by their peers. Patterns that resemble the mild skepticism were prevalent in our real-world data set, but analysis revealed the presence of all five hypothetical patterns."
Putting some numbers on that, it appears from the real publication data that scientists tend to weight their own personal results about ten times more than those that they read in the literature. (I'd love to see this broken down by author, I can tell you). They also found, as I'd expect, that postive statements make up more than 95% of the whole data set. (The authors seem a bit baffled by this - negative results are famously difficult to publish, guys). They also found very high correlations within individual chains of statements - reversals are rare indeed.
What they found completely unnerving was that, given their assumptions, the real-world data are explained equally well by two possible research universes: one where false-positive and false-negative rates are low, and there's a huge perponderance of positive statements among the set of true ones. The other one is a world with very high error rates, in which a given positive statement is much more likely to be false than it is to be true:
"Another major question also remains open: In which of the two alternative universes discovered in our analysis are we living? Our results indicate that the optimistic and pessimistic realities are almost equally likely given currently available data."
My inbox at work has been receiving promotions for the new American Chemical Society journal ACS Chemical Biology. I've looked it over, and it's the sort of thing that I like to read. I should also note that they've started a wiki on the whole topic of just what chemical biology is, and it'll be interesting to see if they get contributions.
But there are other e-mails demanding my attention. The Nature Publishing Group has been sending me notices about new journal, Nature Chemical Biology. That one's not bad, either. Watching them fight with the ACS journal over the exact same papers should be fun.
Of course, you wouldn't expect a powerhouse like Elsevier to stay out of this, but they can point out that they were already there: Chemistry and Biology has been around for years. I don't read it as often as I probably should, but it's published some interesting articles.
And if Elsevier is there, can Wiley be far behind? That would be the Euro-flavored ChemBioChem, awkwardly named but a pretty good read. They have a bit more of a drug-discovery angle to them than the Elsevier journal, to my eye, and they've also been around a few years.
That angle is supposedly going to be dealt with more explicitly by Chemical Biology and Drug Design. This is the journal formerly known as the Journal of Peptide Research, but the times, they have changed. Would you be interested in BioMedCentral's BMC Chemical Biology? If you have some time after that, the Royal Society of Chemistry has synthesized their Chemical Biology Virtual Journal by amalgamating relevant papers from their other publications. How about Current Opinion in Chemical Biology, in case you're having trouble getting a handle on what's going on?
What's going on, as is probably clear by now, is a major publishing pileup. But there's something to be learned from it. If the various publishers didn't think that there was a market (for submissions and for subscriptions) they wouldn't be so eager to get in on the action. In some coming posts, I'll be taking a look at just what action that is, and whether it represents a takeover of biology by chemistry or the reverse.
The original "Expression of Concern" editorial over the VIGOR Vioxx trial in the New England Journal of Medicine was an odd enough document already. But today brought an "Expression of Concern Reaffirmed" in the journal, along with replies from the VIGOR authors.
It's going to take some doing to get these folks together, as you'll see. The NEJM's editors, in their "reaffirmation", add a few details to their December 8th expression. Their position is still that there were three heart attacks in the Vioxx treatment group that were not in the data submitted to the journal. And they're not buying the explanation that these took place after the end of the study, either:
"The authors state that these events did occur during the trial but did not qualify for inclusion in the article because they were reported after a "prespecified cutoff date" for the reporting of cardiovascular events. This date, which the sponsor selected shortly before the trial ended, was one month earlier than the cutoff date for the reporting of adverse gastrointestinal events. This untenable feature of trial design, which inevitably skewed the results, was not disclosed to the editors or the academic authors of the study."
Those academic authors (11 of them from seven different countries, led by Claire Bombardier of Toronto) have a reply to all this in the same issue. Regarding those three MI events, they say:
"The VIGOR study was a double-blind, randomized outcomes study of upper gastrointestinal clinical events. We, as members of the steering committee, approved the study termination date of February 10, 2000, and the cutoff date of March 9, 2000, for reporting of gastrointestinal events to be included in the final analysis. Comparison of cardiovascular events was not a prespecified analysis for the VIGOR study. . .the independent committee charged with overseeing any potential safety concerns recommended to Merck that a data analysis plan be developed for serious cardiovascular events. . .As a result, a cardiovascular data analysis plan was developed by Merck. Merck indicated that they chose the study termination date of February 10, 2000, as the cutoff date. . .to allow sufficient time to adjudicate these events. . . (The three events) were neither in the locked database used in the analysis for the VIGOR paper no known to us during the review process. However, changing the analysis post hoc and after unblinding would not have been appropriate."
The authors go on to say that including the three heart attacks does not, in their view, change the interpretation of the safety data. They also take issue with the journal's contention that the three events were deleted from the manuscript, saying that the table of cardiovascular events in the presubmission draft of the paper never included them in the first place.
The two Merck authors on the paper, in a separate letter, make the same point, and also mention that there was an additional stroke in the naproxen-treated group that didn't make the paper for the same reasons. They reiterate that including the three heart attacks wouldn't have changed anything:
". . .The article clearly disclosed that there was a significant different in the rates of myocardial infarction in the Vioxx and naproxen arms of the study and reported these rates as 0.4 and 0.1, respectively, with a relative risk reported as 0.2. The inclusion of the post-cutoff data myocardial infarctions changes the Vioxx rate to 0.5 but does not meaningfully change the relative risk or the conclusion that there was a significant difference between the two arms of the study. Indeed, with such a small number of events (which were not a primary end point of the study) - and with such wide confidence intervals around them - it is difficult to imagine that this small numerical change could affect the interpretation of the data."
Looking at everything together, I'm still coming down on the side of Merck and their academic collaborators in this part of the fight. The post-launch cardiovascular data on Vioxx and its advertising and promotion are worth debating separately, but as for the VIGOR study, I think the NEJM is overreaching. Still, from Merck's viewpoint, I think the damage has already been done. . .
Update: Y'know, it occurs to me that there are a few people who aren't as upset about all this editorial wrangling: the editors of JAMA and the other top-ranked medical journals. They'll be getting some manuscripts that otherwise would have gone to NEJM.
One of the glowing chunks of fallout from the South Korea stem cell scandal is the handling that Science gave to Hwang's two retracted papers. It's no secret that the top journals compete to see who can publish high-profile papers, and for research like this it's pretty much down to Science or Nature. As fate would have it, Nature got the one of Hwang's recent papers that actually doesn't have to be retracted - whether that's a coincidence or not, I don't know. But the scientific publishing world is no doubt speculating about that, and you can already see some uncharitable comments surfacing. For example:
"It sounds as though their processes were rather sloppy," said Dr. Benjamin Lewin, the founder and former editor of Cell, a biology journal known for its rigor. "At a minimum, Science should have been more careful and should never have reached the stage of publishing a paper with identical photos," he said, referring to the fact that some photos of cell colonies in Dr. Hwang's 2005 article were duplicates of one another.
Dr. Lewin said that a journal editor needed to develop an intimate knowledge of his reviewers' strengths and weaknesses, and that "Nature and Science don't have the reputation for rigorous review."
Meow! I have a lot of respect for Lewin, but this sort of thing isn't going to burnish his reputation. (Update: His deep admiration for the folks at Science turns out to go back a long way). The response of my fellow scientists has been to suggest that he try publishing in one of those non-rigorous journals under a pseudonym and see how far he gets. I guess it depends on how you define "rigorous", though, and it's true that there are other definitions besides rejecting 90% of all the papers submitted. The journal may well have rushed these hot papers through the process, since they were clearly the sort of thing that would be worth publishing. (See some thoughts on this here). The reviewers for both papers have no doubt been involved in some difficult e-mail exchanges in the last few weeks. . .
The danger with comments like this, though, is that every journal that publishes papers worth reading has published papers worth retracting. And that includes Call, naturally, although their most recent one was (weirdly) done without the lead author's consent. (Thanks to Ivan Oransky's blog at The Scientist for this). And in a fraud case that I missed last fall, Luk Van Parijs of MIT was fired after faking loads of data in several research papers. New Scientist found, while looking through his publications on their own, that one of his papers (in the journal Immunity) almost certainly contains faked flow cytometry data. The graphs shown are just too similar, and that's not a technique that churns out exact duplicates of anything, ever.
This is quite similar to the problem with Hwang's stem-cell illustrations, which Lewin is saying should have been caught. But Immunity, like several other single-noun-title journals is published by. . .Cell Press. That's the problem here. No one comes out of this business looking good, even if they try.
You know, I sure hope that there's a special issue of J. Med. Chem. coming up, all about computational chemistry and molecular modeling. No, I haven't lost my mind - yet. I was looking at the advance publications page on their web site this morning, which had 72 accepted papers on it slated for publication.
Fine, just fine. The problem is, at least twenty of them have little or no reality-based chemistry in them. I define that, understandably enough, as real compounds which were made and tested and put into a vial somewhere. True, the majority of the papers on the list are more traditional med-chem, and many of those have computational methods as a component. But to have over a quarter of the papers with only modeling, docking, simulations and scoring? There is, last I heard, a Journal of Computational Chemistry. And a Journal of Molecular Modeling. And a Journal of Molecular Graphics and Modeling. For that matter, the American Chemical Society itself, publisher of J. Med. Chem., publishes the Journal of Chemical Information and Modeling.
I know, I know, these aren't as prestigious, but where are all the medicinal chemistry papers going these days? (I mean the ones with experimental details, of course. The ones without any are going, as always, to Bioorganic and Medicinal Chemistry Letters.) There's some real irony in this situation, because one of those papers actually does a good job blowing up the reasons for some of the others to exist. More on this anon.
"In light of your recent series of postings about impact factors, I have my own questions about scientific publishing. It seems to me that in physics and chemistry the prestige of journals is more horizontal compared to biology, where there are a thousand specialty journals that are highly stratified with respect to their prestige. Now everyone knows that a paper in Nature or Science is supposed to be superb. In chemistry, it looks like even the best chemists send most of their articles to JACS, which, if my perception is correct, is not an especially prestigious journal. In biology, on the other hand, we have a strict hierarchy which is approximated by the impact factor. For molecular cell biology, I would rank them something like this, IM not-so HO:
One could spend endless hours cataloging this for each subdiscipline. . ."
Oh, yeah. It could be useful for outsiders, too. Would readers care to submit their lists for organic chemistry journals, in order of prestige? We'll take a survey and see how closely it matchs ISI's data.
I note that each of those lists stops while still well in the "good place to publish" category - the middle and lower parts of a complete list would be harder (and more painful) to rank-order. But I couldn't have reproduced either of those orders exactly. I don't think I've ever seen or read a paper from the Journal of Bacteriology, for example, and I wouldn't have known that it was reasonably prestigious, although I have the rest of those titles covered.
In the same way, although it may not seem that way from the outside, JACS is a pretty good place to publish for a chemist. If "even the best chemists" send their work there, which is more or less true, then it's de facto harder for others to get there work in, and therefore prestigious. Science, PNAS, and especially Nature publish so little chemistry that those journals aren't really on our lists, for the most part.
But it's Angewandte Chemie that has moved definitively into first place, after some years in a rough tie with JACS. And whatever we might think about impact factors, I think we can agree that the example given in the latest issue's editorial (by Peter Goelitz, Ang. Chem 44(35) 5538) is not the way to use them. You don't come across this exact alloy of nerve and cluelessness too often. Says Goelitz:
The hankering after the high impact factor has resulted in many authors giving up a healthy amount of self criticism and sending their manuscript first to a journal in which it has no chance of being accepted. This ideology is already producing fruit as can be seen in the following: An author had a manuscript rejected from Angewandte Chemie after the assessment of three scientists. Deputy Editor Neville Compton recommended publication in the European Journal of Inorganic Chemistry (the editor of which is Karen J. Hindson) and shortly afterwards received a letter containing (this passage):
"Our manuscript was evaluated by three referees. Referee 1 finds the work is important and recommends publication (in) Angewandte Chemie (IF = 9.16) after minor revision. Referee 2 finds the scope of the work is directed to a journal such as Crystal Growth and Design (IF = 2.74). Referee 3 finds the work of high quality and compliments the authors, and feels that it would be a pity not to publish in its actual form. He or she recommends publication in Inorganic Chemistry (IF = 3.38.)
We were pleased to read that you spoke with Dr. Karen Hindson and recommended our paper for publication. However, I would like to draw your attention that if you add up the values of the Ifs of the three journals cited above and then divide by three you will obtain a value of IF = 5.09. The closest journal to this value would be Chemistry: A European Journal (IF = 4.517.) I therefore gently ask you to accept our paper. . .since you are the Editor in Chief. . ."
Yesterday's post introduced journal Impact Factors to those who haven't had the honor of meeting them yet. Everyone whose livelihood depends on scientific publication, though, already knows them well, since anything that can be measured will be used at performance evaluation time. IFs are a particular obsession in academic research, since publishing papers is one of those things that an aspiring tenure-seeking associate professor is expected to do. (On the priority list, it comes right after hauling down the grant money.)
But that's not what we value in industry. We know about the pecking order of journals, but we just don't get a chance to publish in them as often as academics do. I'd much rather have a paper in Angewande Chemie than in Synthetic Communications (to pick the top and near-bottom of the reasonable organic chemistry journals), but it won't make or break my raise or promotion hopes. Now, having zero patents might do the trick, but that's because patents are a fairly good surrogate for the number of potentially lucrative drug projects you've worked on.
Nope, it's academia that has to live by these things, and there are complaints. On one level, people have pointed out that impact factors may not be measuring what they're supposed to. Here's a broadside in the British Medical Journal, pointing out (among other things) that the individual papers inside a given journal follow a power-law distribution, too. It's glossed over by the assignment of a single impact factor to each journal, but the most-cited 50% of the papers in a given journal can be cited ten times as much as the lesser 50%.
The less interesting papers are getting a free impact ride, while the better ones could have presumably been playing off in a super-impact league of their own, if such a journal existed. The authors also point out that journals covering new fields with a rapidly expanding literature - much of which is also ephemeral - have necessarily inflated IFs. Does it really indicate their quality? (Well now, say the pro-impact people, isn't this just the sort of carping you'd expect from the BMJ, who live in the shadow of the more-prestigious Lancet?)
But there's also the problem of self-citation. As ISI's own data make clear, lousy journals tend to have more of it. (The text of that article seems to spend most of its time trying to deny what its graphs are saying, as far as I can see.) So if you think that the Journal of Pellucidarian Materials Science has an unimpressive impact factor, wait until you see it corrected by stripping out all the citations from the other papers in J. Pelluc. Mat. Sci. If you accept what IFs are supposed to be measuring, you have to conclude that the huge majority of journals are simply not worth bothering with.
On a different level, there's plenty of room to hate the whole idea, regardless of how it's implemented. The number of citations, say such critics, is not necessarily the only (or best) measure of a paper's worth, or the worth of the journal it appears in. (As that link shows, the original papers from both Salk and Sabin on their polio vaccines are on no one's list of high citation rates.)
It is no coincidence, they go on to point out, that the promulgators of this idea make their living by selling journal citation counts. And by conducting interviews with the authors of highly cited papers and with the editors of journals whose impact factors are moving up, and God only knows what else. The whole thing starts to remind one of the Franklin Mint.
I had an email asking if scientists in industry care about journal impact factors. It's an interesting question, but it needs to be answered in parts. Unless you deal with academic publishing, the phrase probably doesn't mean much. "Impact factors" are an attempt to quantify what everyone knows empirically: some journals are more prestigious than others. You know how we science types love to quantify stuff.
The whole business comes from the folks at ISI (now owned by Thomson.) They had been publishing the Citation Index for years, which was (and is) a way to find out who had referenced a given paper in the scientific literature after it was published. This can be useful if you want to see if anyone's followed up or commented on an interesting paper (or if you just want to see if anyone's cited your own work.)
And as ISI realized early on, it could also furnish some interesting rankings of "most-cited" papers in a given field. (Here's a recent set of lists from Chemical Abstracts, the big dog in the chemical information world, who got into the citation-counting business themselves.) You could figure out who the most cited authors are, too, although that ISI link won't rank-order them for you. (You have to pay for that data! Or you can look here.) There are lists of the most highly cited institutions, too, naturally.
About ten years ago, they introduced the Impact Factor to do the same thing for scientific journals. That's the number of citations generated by a journal (usually over a multiyear period) divided by the number of papers it published in that time: the average number of cites per paper, in other words.
The publishing community - initially rather worried and sceptical, if my memory serves - has gone completely crazy over the whole idea. Now journals advertise themselves by their impact factors. "Publish here! We're a good journal, really! We have proof!" If you'd like to know what a particular journal's rating is, they'll probably shout it out if it's any good at all. A failure to mention the number, down to three decimal places, is an act that speaks for itself.
Want the whole list? It can be rather hard to come by, unless you're a paying customer of ISI's, but here's a place to start. Those aren't the latest figures, but they'll do. You'll notice that at the top are a bunch of review journals, who publish comparatively few papers but get cited out the wazoo. Among the original-research journals in the top ranks are big kahunas like Cell, Nature, the New England Journal of Medicine, Science, and such. But I find a perverse fascination in browsing the low end of the scale. The Ethiopian Medical Journal? Fertiliser Research? Bovine Practice? Annals of Saudi Medicine? Surely some of these trench-dwellers ceased publication and vanished from sight during the rating period. The list of "0.000" impact factors is particularly alarming, although most of the journals listed are there by some sort of statistical artifact or (I presume) don't really exist. But if no one reads them, how do we know if they're real or not?
In the next installment, we'll look at some problems with the whole idea - there are some - and I'll tell you if we industrial types give a hoot about it or not. . .
Physics prof Chad Orzel published a guide last week to surviving organic chemistry seminars. (I can understand that that's a particularly high priority for people who don't know any organic chemistry and have to sit through these things, actually.) He provides what is, for an organic chemist like me, an interesting man-from-Mars viewpoint:
"Stage Two: "Here's the stuff we start with." This will include a couple of diagrams showing different arrangements of hexagons. The jargon will get pretty thick, here, but almost all the strange words will be names of different parts and sub-parts of molecules. . .
Stage Three: "Here are the steps in the process." This will include at least one slide showing multiple diagrams of hexagons, with arrows between them. The jargon will again be pretty thick, but here, all the strange words will refer to methods of sticking pieces of molecules together. . ."
Ah, but are these the hexagons with little circles in the middle of them, or the plain ones? Very different, those guys are. Since he didn't mention dashed and wedged bonds, I'll assume it's the former. Rarely will those arrows point from one kind to the other, by the way. You can turn the circle-inside guys to the uncircled kind and vice versa, but it takes some real hammer-and-tongs reactions to do it. I haven't run one of those for over twenty years now, to give you the idea.
Keep in mind that we use pentagons and heptagons, too, but you won't see many circles inside them (although it's possible, especially if you're sitting through an organometallic chemistry talk.) Squares are rare, but triangles can represent a group that I've always liked, and that I have a reputation for introducing into my lab's molecules.
Once you've staggered through the talk, there's the problem of questions at the end, naturally. Orzel's advice:
"If you absolutely need to ask a question, remember that the crucial figure of merit for these talks is the "yield," which basically means "How much product do you get for a given volume of reactant?" If the speaker hasn't mentioned the yield specifically, you can't go wrong asking "What's the yield like?"
If they have stated the yield, ask "How does the yield stack up against other methods of producing this stuff?"
If they have stated the yield, and compared it to existing methods, and you still feel a need to ask a question, ask about the solvent peaks/ blobs/ pencil marks.
Questions of the form "Why are you trying to make this stuff in the first place?" are usually considered unsporting. . ."
Ay, that's for sure. I'm glad no one asked me that last one during my PhD defense, because by that time I'd begun to realize that most of my possible answers were pretty weak. I sure don't think I'd have been able to get them out with the tone of conviction I'd have had three years before. As I told people near the end - no, not my examining committee, what do you take me for - "The world doesn't need another synthesis of this molecule. But I do."
But I'd ask Chad to take heart - I don't hear many physics presentations, but I do have to sit through some pretty gritty molecular biology talks from time to time. I can deal with those, for the most part, except (as I've mentioned) for the sneaking suspicion that I've been seeing the same electrophoresis gel since 1987.
And, frankly, I'm not all that good at sitting through talks in my own field. My tolerance of uninteresting work has not improved over the years, and my definition of "uninteresting work" is gradually becoming more inclusive. It's to the point that some of my colleagues give me the raised eyebrow when I actually show up at a seminar. I feel, in other words, Chad's pain.
Instead of blogging tonight, I went outside with the telescope and froze my extremities. My wife loaned me a fashionable wool scarf, which saved my ears while I observed the winter sights. Every time I see Saturn (or Jupiter, which wasn't up yet), I wonder how much of those yellow and brown colors are the same things that stick to the top of my chromatography columns in the lab. I had a look at the yellow-orange dot of Titan, and thought about the methane rain that might have been pattering down on the Huygens probe at that very moment.
Not far away from Saturn's current position (in the constellation Gemini) is the "Eskimo Face" nebula. By the time I got around to that one, he didn't look much warmer than I felt, so I packed it in.
In earthbound exploration, tomorrow I'll be making a class of compounds that has never been made before, at least according to Chemical Abstracts. That's just what we need to plant the patent flag and claim the territory, and I was glad to see things show up that were fairly close, but not quite there. That way I can be fairly sure that the chemistry will work.
It's always worrisome when you get hundreds of hits from a literature search, because you know that you're going to have a hard time finding (and claiming) something new. But it's also troubling when you get zero hits across a broad class of related structures, because at this late date, there might well be a reason for that which you, too can rediscover. Of course, you can get literature hits that are all for uses like hair dyes, photoresist agents, corrosion inhibitors, and arthropodicides. Then while you figure you can probably make the compounds, you have to worry a bit about their status as something that you could ask someone to eventually put in their mouth. . .
I'll be occupied on and off in the next few months with writing several scientific papers (nothing wrong with bulking up the ol' resume, especially in this climate.) There's always the question of which journal to fire these cannonballs of wisdom towards. Two factors compete: where you'd ideally like to see the paper appear, and where you realistically think you can get it accepted. You aim for the intersection of those lines.
Sometimes the answer is clear - for example, if you've got a comprehensive report on a fairly new diabetes therapy, a good solid paper from discovery to clinic, you're probably going to send it to Diabetes. The situation is more complicated at the higher and lower ends of the scale. A startling head-turner of a paper has a number of venues to choose from, depending on its focus and who you might know in the various heirarchies - Science, Nature, Cell among others. I won't be sending any of this year's papers to those folks, sad to say.
You'd think that the premier journal in medicinal chemistry would be the Journal of Medicinal Chemistry. It may well still be, and I'm sure that the American Chemical Society (its owner) thinks so. I need to check this out, but it's my impression that the journal has had a shrinking percentage of industrial papers in it over the years. Some upstarts have siphoned off some of their raw material. A particular competitor is Bioorganic and Medicinal Chemistry Letters, which began life (and still spends a good part of its time) as a dumping ground, but has slowly changed into something more.
One big difference between the two is that J. Med. Chem. publishes both full papers and short communications, while BOMCL features only the latter. (That's the meaning of "Letters" in scientific publishing.) A full paper naturally includes a full experimental section, with preparative details of all the compounds and their physical characteristics. And there we come to the real split between the journals. For a full paper, J. Med. Chem. wants more details than I happen to have.
They want combustion analysis on important compounds, and I just flat out don't get that level of data on most of them. That's a primitive-sounding (but, in theory, very effective) method of checking a compound's purity. You burn a small sample of it and carefully measure the amount of carbon dioxide, water, etc. that come off. That gives you the percentage of the compound's weight that was made up of carbon, hydrogen and the other oxidizable elements, which is why we often call it a "CHN" analysis.
Then you see how well the experimental value matches up with your theoretical amounts. It comes out to a couple of decimal places, so you can distinguish pretty close matches, in theory. In practice, the compounds usually have to be thoroughly dried and handled carefully before this test, because many of them will soak up a bit of water from the air. Some of them actually crystallize with water molecules in their lattice, as part of the repeating crystalline pattern, and combustion analysis is a good way to see if your compound has done that. It also means that if you're willing to assume, say, one/third molecule's worth of water of crystallization, or some damn such, you can finagle the numbers to come out to most anything you need. (Mind you, a whole paper's worth of such fudge-factoring would get a frosty reception.)
But, for the most part, I don't care very much if my compounds combust well. During a drug discovery project, we don't have time (or material) to send samples away to be analyzed (it's a specialized job.) We rely on NMR (proton, some carbon) and the combination of HPLC and mass spectrometry. Those are enough to characterize a compound for a patent (well, except for some outlier countries like Taiwan), and they're enough to convince us that we've made the right thing and affect the expenditure of millions of dollars. But it's not enough for J. Med. Chem.
Well, not until recently. They've slowly been loosening the noose the last few years, offering high-resolution mass spectral data or data from two different HPLC systems as alternatives. Not that we usually have those, either, but it's a start. But I think I'll let the combustion lab do the work: if I'm going to be sending J. Med. Chem. anything this year, I'd better start getting ready for a Wonder Drug Barbecue.
Not much blogging time tonight - I spent all day sitting in front of my computer, anyway, working on a manuscript for one of the chemistry journals. I find that writing blog entries is (usually) no problem, and writing things like my Contract Pharma column come fairly easily, too. But not scientific papers, that's for sure.
I think it's because there's not very much actual writing involved. The text of my article isn't much to look at - just a few paragraphs, really. But getting the figures and tables right, looking up all the data and resolving all the discrepencies - it's all finicky detail work. All the problems you uncover are probably about three years old, too, since that's the (youngest) vintage of stuff we can publish, and you can imagine how that helps resolve things quickly. The construction project can be enjoyable, but most of the time it's like building something with more parts than you're in the mood for.
But I'm going to get plenty of it this year, which isn't a bad thing. For various reasons, I have several pieces of work that have reached the, um, mature stage of publication-worthiness. I'd rather that they were reaching a sick patient in a Phase III trial somewhere, or better yet, a pharmacy shelf, but I'll have to settle for Bioorganic and Medicinal Chemistry Letters.
Over at Uncertain Principles, Chad Orzel's commentators got into a discussion of how you list people in a large multi-author publication. My system is that the first author and the last author are the people who did most of the work and/or were in charge. It's worth amoment to think about the gap that can open up between those two descriptions. Between those bookend names, I've opted for alphabetical order when I've written or helped write papers, because the alternative is an institutional-sized easy-open pressurized Can O' Worms.
Assigning credit in a scientific project is an awful job. Multiple people will be sure that they thought of an idea first and that everyone else just borrowed it from them. Some people will be livid at how others on the team got by without seeming to ever contribute anything, while they carried the whole project on their backs. Meanwhile, some of that latter group will be furious at the first ones, who from their perspective got to do the easy stuff that generated all the cheap and flashy results while they labored in the salt mine.
Sometimes these things can be resolved by enough tedious effort, but most of the time they can't. And it's almost always not worth the effort - at least for a journal article. Now, for a patent, things are very different, as one of Chad's commentators rightly points out. Everyone listed on a patent has to be able to state clearly what their contribution to the invention was. If you can prove that a patent has people on it that did not contribute (or left out people who did), you can get the thing invalidated. That's not easy, but it has happened, and the mere threat is enough to make everyone take inventorship pretty seriously.
My quick-and-dirty test for inventorship has been to tell people to ignore the whole draft of the patent application except the claims. Go straight to those, and find something there that you thought of, and be ready to point to it. Ideally, you should check to make sure no one else is going to point to the same thing. Best are the things that you thought of first and were also the first to do. No one can take that away from you.
Next best are the things that you thought of first and handed off to someone else to accomplish - if they didn't add anything to your idea, you're probably an inventor and they certainly aren't. Being the first one to try someone else's idea in the lab doesn't mean much in inventorship terms, and quite rightly. Now, if the person you handed things off to added something meaningful, you may both be inventors, which is were things can become interesting. Sometimes the original idea has been mutated so thoroughly that the final claim is really the work of the second person, with nothing recognizable from the first one.
I tell people who work for me that if they want to be on the patents coming from our lab, they'd better have some ideas of their own to show when patent-writing time comes. Naturally, I try to fulfill my end of that deal by letting people work on their own stuff as much as possible. The only way we can end up in trouble is if we pick a total-loss part of the molecule to work on and end up with nothing worth including in the patents. You want to keep a sharp eye out for that situation, and be ready to steer yourself (and your lab) out of it.
I took the opportunity, while moving to a new project, to clean up my office and files. This time I dug in pretty deeply, and heaved out about 30 pounds of stuff, some of it a good fifteen years old. Looking at the folders, I realized that they were for kinds of chemistry that I hadn't done since then and had no plans to do again. And even if I did, I'd want more current references than these, so out they went.
Good reference papers are a lot less costly to get, in man-hours, than they used to be. Searching manually through Chemical Abstracts meant that when you found something, you held on to it. Who knew if you could find it again? Now, with SciFinder and the like, you can plow through the literature like a nuclear-powered icebreaker. When you find the citation, you hit another button and the PDF hoses right up on your screen. It's like having magic powers, and not many who missed out on the bound-volume days will ever understand how strange it will always feel.
Still, I'm keeping some golden oldies. While my old palladium coupling references are historical artifacts, the ancient papers are still good in some of the slower-moving fields, things like Friedel-Crafts reactions. I still have the first technical paper I ever copied off, from my undergraduate days in 1981, and that one's staying. And the references that are in my PhD thesis are still in their own folder, which is turning a worrisome yellow around the edges.
When I was an undergraduate, learning all the chemical reactions that you have to learn in undergraduate courses, I got a few wrong ideas into my head. Well, probably more than a few, but you know what I mean.
One of them came from Theodora Greene's book on protecting group chemistry, which was a new item in its first edition around the time I was doing my first summer research project. This was, and remains, the best single source for finding out how to attach and remove groups that will protect reactive parts of your molecule until they're ready to be unveiled. Organic chemists spend a fair amount of time using these, especially for things like amines and hydroxyl groups, which are liable to become personally involved in all sorts of reactions if they're not restrained.
In my first encounters with these, I got the (completely mistaken) impression that all protecting groups tended to have specific recipes, precisely targeted for their removal. Methoxymethyl ether group to take off? Reach for the zinc bromide! Pivaloyl ester? Methyllithium! And so on. There was page after page of this sort of thing.
And it's not like those are poor choices, to be sure. I've used that methyllithium one myself, although not since about 1988. But zinc bromide isn't called for because of its essential zinc bromidey properties, it's called for because it's a Lewis acid, and just about any Lewis acid will take off a MOM group. Some will be faster than others, some might do things to the rest of your molecule which others wouldn't, but they'll all get around to cleaving your methoxymethyl ether. As will good old protic acids - aqueous hydrochloric will take one off just like it was 1895 again, if the rest of your molecule can stand it.
Learning all this, which I did in my later undergraduate years, was a good way to start appreciating the mechanistic side of organic chemistry. There really aren't that many reactions, no matter what poor sophomores think while they memorize page after page of them. Organic chemistry isn't zoology. And it also hit me, after a while in graduate school, that the claim of a new, mild, wonderful way to remove a given protecting group was a pretty good method to get a paper published, which might - just might - have something to do with their proliferation.
Tonight, a few varied links from around the blogging world, which only serve to remind me that I need to reconstitute my shattered blogroll:
Via Chad Orzel I read this note from Preposterous Universe on publication of clinical trial data, and on the general problem of what to do with negative results. I know that there have been attempts to start journals in chemistry to allow an outlet for these, but I don't think that any of them have worked out.
Meanwhile, over at DB's MedRants, he has a good piece on drug reimportation from a practicing physician's point of view:
"The Canadian "solution" makes for good politics, but bad policy. This "solution" is destined to fail. We need higher level thinking to better understand pharmaceutical costs and our resultant expenditures. The "wonder drugs" are not created by spontaneous combustion. They result from expensive research.
Physicians need to understand newer drugs very well. We need to understand when an expensive drug is a better alternative, and when a cheaper generic works as well. We need the NIH (and associated Institutes) to sponsor important drug studies. Relying on the pharmaceutical industry to fund drug studies seems cost effective in the short run, but from a long term perspective, such studies are rarely designed to answer the important cost questions."
Readers will recall my attempts to warn Imclone shareholders of the risks of holding their stock at its current levels. Over at the Motley Fool, another attempt is made:
"But the thing that gets me about ImClone is that the company is valued as though Erbitux has already been a smashing success, as if it has already received approval for other therapies currently in trial such as earlier-stage colorectal cancers and head and neck cancers, extending its on-label marketing reach. At an enterprise value of $4.1 billion, ImClone has grossly improved both its operating and financial positions from the days of scorn and scandal when Sam Waksal ran the show. Even if the annual domestic sales for Erbitux hit the $1 billion mark -- blockbuster status, if you will -- ImClone garners only 39% of that amount, or $390 million. That's revenues, and yet the company's stock is priced 10 times that high today."
All true. But good luck getting those points across - it's like trying to reason with house cats. Turning from those to dogs, the world's most famous combination dog-breeding inorganic chemist has this to say about writing review articles covering the chemical literature:
"Why I do them I don't know. They're exhausting, I get no real feeling of satisfaction out of them, the remuneration doesn't begin to repay the effort involved and while the reader may appreciate the compliation of the state of the art, no credit accrues to the author. I suppose that since publications on original research are frowned on by my company, I do reviews to pad my resume. People can't read, but they can count."
True again. We have an easier time publishing original research, but only after the project is either ascending into the heavens of the market (which doesn't happen often enough, I can tell you) or is a complete dead letter. "The only thing we can do with that," we say, poking at the remains, "is to publish it."
Mentioning all the nanostructure papers in the journals brings up the topic of fashions in chemistry. We've got 'em, all right. Waves like this tend to wash over the literature every few years. (I can only speak for organic chemistry, but I assume that it's the same in the other disciplines.)
For example, only in the last couple of years have we begun to escape from the olefin metathesis craze. That's a useful reaction, and I can't make fun of it on its face. But a few years ago, it seemed that everyone with two alkenes in their lab was finding a way to get them in the same flask with some Grubbs catalyst or whatnot. People looked at the reaction and said "Hey, that's neat. I could do that", and they did.
Of course, people also look at these things and think "Hey, that's neat. I could publish that" or "Hey, that's neat. I could get funded for that", and those reasons carry a lot of weight. So journal editors and the reviewers at granting agencies carry some responsibility for these fads, but they're just as human as the rest of us. Most of the time.
These bursts of activity can serve a useful purpose. The state of the art advances rapidly when everyone's trying to improve it, as you'd figure. The best examples of waves of interest have a can-you-top-this quality in them; the worst have a bunch of chemists just doing the same thing because everyone else is doing it.
There are even fashionable molecules, ones that have been synthesized over and over again, because they come in handy to show that your new synthetic method is good for something. (That's not a line of argument that I find very compelling, but it's a common one.) An insect attractant called brevicomin is a good example. I'm sure that I've seen six or eight synthetic routes to that, and it's not like I've been collecting them, either. Twenty years or so ago, the jokes were about a molecule called cis-jasmone, which turned up over and over again when someone wanted to prove their synthetic moves.
But since we're talking about fashion, I should note that the whole field of natural product total synthesis isn't as fashionable as it used to be. That's a larger topic worth a post of its own, but it's something that's had its practioners a bit worried and defensive the last few years.
There's a type of paper that's showing up often in the major chemistry journals these days, and it's a type that didn't even exist a few years ago. I can't count the number of reports of nanometer-sized structures that have been described recently. Rods, filaments, sheets, cylinders, shells - you name it and someone's got it. That inorganic salt plugging up your filter? Turns out it's not just an annoyance, it's a publishable nanostructure!
On one level you can see why this happens, with all the publicity that nanotechnology has these days. But that's not what most of the papers are really about. No particular use or general principles are suggested, for the most part, just "We found these things, and they look like this." (You can spot these papers quickly in the abstracts at the front of the journals, because they're invariably illustrated with a photomicrograph of the new structure.)
There's a place for that kind of paper, naturally, but are there dozens of places? Some of these things may turn out to be useful, or at least point the way to something useful, but for now they're largely just being described as curiosities, and they're being published because - well, because they can be. Perhaps some of these groups are hoping that someone, someday, will make a breakthrough that makes their paper look ahead of its time.
The techniques to look for these structures have been around for some years, so it's not like we're just now able to see them. It's just that up until recently, no one has cared all that much. I have to wonder what would have happened if someone had submitted a paper to JACS fifteen years ago about, say, scandium salts that form nanoscale helices when precipitated out just so. Would the editors and reviewers have known what to make of it? Or would they have tossed it back, telling the authors to come back when they had more to say?
There's a lot of serious nanotech work being done in chemistry, but this stuff isn't it. I have to think that these papers are going to look a bit strange and dated in coming years, once this stamp-collecting phase passes. When will the editors at the likes of the Journal of the American Chemical Society,the Journal of Organic Chemistry, Organic Letters,and Angewandte Chemiecall a halt?
Now that ASCO's wrapped up (and the American Diabetes Association meeting as well) every attendee from the drug industry has gone back to report on the news: copies of poster presentations, handwritten notes from the talks, and (most importantly) information that was only given verbally. That would be in answers to questions after a talk, in conversation around a poster, or in small gatherings all over the place.
That sort of stuff is often the real gold from a large meeting. No one is going to spill anything vital, but you can often learn more than is strictly contained in the official presentations, and everything helps. The drug industry being what it is, we have plenty of things we'd like to know about what the competition is up to: Are they still interested in compound X? Have they moved on to another one? What's better about it? Is that side effect something that's showing up in the whole class of compounds? How did they ever dose that stuff at those levels, anyway? Are they in Phase II? In what sort of patients?
It's a well-known psychological effect in the business that we treat new information as something that's just happened. But you have to keep in mind that the information in these meeting presentations is at best several months old, and maybe older than that. (At least it's better than getting excited about a paper that's just shown up in the literature.) That's another reason for all the one-on-one questioning, naturally - we want to know, for once, what's happening right now.
I haven't been to any scientific conferences so far this year, and I have to admit that I in some ways I haven't felt the lack. There are a few meetings that I enjoy more than others (Gordon conferences and Keystone meetings come to mind), but there are others that I'd have to be paid extra to attend. Some of the really large ones have been out of control for years (Society for Neuroscience? I'm talking about you.)
You can pick up some good information at one of the better meetings, but even then it can be a strain. Scientific presentations can often be mistaken for a work of the devil: here, I have some important and interesting things to tell you. So I'm going to run it past you once, from fifty feet away, in the dark. Sound good? Or more like a technique to deliberately impair communication? Your best shot is a good poster session or a one-on-one talk, and the Gordon or Keystone type meetings I mentioned earlier are the best ones for that kind of contact.
It doesn't help that many scientists are such notorious speakers. I've had very bad times, there in the dark, watching someone who's clearly using his slides as mnenomic devices ("Next slide, please. . oh, yes, that's right, here's where we were trying to synthesize. . .") or someone who reads off every word on every slide, adding not a syllable of information along the way.
My patience for such things was never very well stocked, and I've run completely dry in the last few years. When I'm listening to a poorly delivered talk, or one on a subject that turns out not to interest me at all, I just sit there thinking of the time that's going to waste and what I could be doing.
At least I'm awake, though. I recall one seminar in graduate school where the visiting speaker pretty much put everyone into a vegetative state from about the second slide. The floor was thrown open to questions at the end, but an embarassing silence ensued. The faculty member who introduced the speaker caught on very quickly, and popped in with a question of his own: "Actually, one of your reactions reminds me of something one of my students is trying right now - right, Paul?" Dead air. "Paul? That addition to the acrylate?" Nothing. Elbows begin driving into Paul's ribs, waking him abruptly back in the next-to-the-last row: "Uh. . .what was the uh, question?"
I noticed this post over at A Scientist's Life on some recent instances of retracted papers and scientific fraud. Those two phenomena aren't linked in every case, but they're often seen in each other's company. People do tend to think they're a couple.
The papers were from Science and Cell, two of the really top-shelf journals (links are at the blog above.) I suppose that makes sense, because there's really no point in faking your way into the Transactions of the Ruritanian Academy or something. It would be like counterfeiting nickles. Lou, the biology post-doc blogger, rightly says (with reference to the recent Jan Hendrik Schon case at Bell Labs):
"I've never thought about falsifying data. That goes against my education and belief as a scientist. Naive as it may be, I thought the whole point of science was to look for answers in what is already there - how nature works. I was about to state that it must be biological sciences, but physicists do it too...hey, where are the chemists then?"
Well, clearly, it's because we're too, ahem, upstanding and - haaarrgh - intrinsically honest to do anything of th-aaaackh. . .sorry, couldn't make it all the way to the end of that one. I second that thought about never thinking about faking data, of course, but much as it pains me, I can give some examples of fraud in the chemistry world. To start with, I'll reprise a post from a couple of years ago on my old site, which pointed readers to an article in the journal Synthesis (p. 29, 2002). That paper belongs to the select group of those whose sole purpose is to demolish another one.
The original, now discredited paper was in the same journal over a year before, presenting an interesting reaction that I thought we could make use of in my lab. We actually tried the chemistry out, but it flopped cleanly and completely, giving exactly the wrong product. I chalked it up to the weirdness of our compounds, which was not to be underestimated. Some things worked on them, and some just didn't. We poured the reaction into the red waste can and did something else, which is almost always an option in medicinal chemistry.
But the author of that reference I cite had the same thing happen to him, and he didn't take it as quietly. Going back over the original examples, he showed that the first published work wouldn't, didn't, and couldn't go the way it was reported. Some of the discrepencies could have been put in the "honest mistake" category, subheading "really sloppy honest mistakes," but it seems to me, in the end, that some of it couldn't. The editors of Synthesis seem to have agreed, and to their credit the criticism found its way quickly into print. The (single) author of the non-reproducible work is still listed, though, as a faculty member at the institution he published his paper from - if there have been any consequences of this affair, I haven't heard of them.
I'll dredge up some other examples in future posts. Many of them involve self-deception, at least at first. But as time goes on, the deception becomes contagious, as the originator of the suspect work realizes just how far out from shore he is. "Stepp'd in so far that, should I wade no more, Returning were as tedious as go o'er. . .", as it's put.
I sent off a manuscript to a chemical journal not long ago. There's an initial flurry of e-mail activity when you do that - we've received your manuscript, we've sent your manuscript out to reviewers - and then a more or less prolonged period of silence. The next thing you hear is whether the paper's been accepted or not, along with the referee comments.
Mine were the usual mix of helpful suggestions and things that make you roll your eyes. One of the latter was a comment that immediately pegged the reviewer as someone from academia. They noticed that the data from our primary assay, against a human enzyme, didn't always match up well with the secondary assay, which was against a rodent cell line, and wanted some more explanation for why some groups of compounds weren't active.
To which I could only reply "You and me both!" That's a constant problem in medicinal chemistry. A majority of projects are set up in that format, with a cell-free assay as the first filter, then cells expressing the target as the next hurdle. And it's just about inevitable that there will be whole groups of compounds that work fine in the first assay, but just wipe out in the second one.
Why should that be? As far as we know, there are two general ways that compounds can get into cells: passive transport and active transport. The passive route is just diffusion across the cell membrane: "Wonder drug? You're soaking in it!" It's affected by broad trends in molecular size, polarity, and so on. The second route is when your compound hitches a ride on some transport protein.
There are hundreds of these things involved in opening up channels into and out of the cell. Some of the famous ones move ions (calcium, potassium and the like), which makes sense. Those are small and electrically charged, so they're not going to just wander across the membrane on their own, and the cellular machinery depends on keeping such membrane potentials tightly controlled. Then there are transporters for large proteins, which are too huge to diffuse by themselves, and for essential classes of small molecules like fatty acids.
No one's sure how many of these things exist. Just in the last few years, there's been a whole new class discovered, the aquaporins, which (as the name implies) move water itself across the cell membrane. You wouldn't think that you'd need an active transport system for that (at least a lot of people didn't think so) but the things turn out to be ubiquitous. If there's a transporter for water, there can be one for anything.
The efflux pumps I spoke of the other day in antibiotic resistance are active transport proteins, too, naturally. Those complicate things by taking compounds that diffuse perfectly nicely into cells and making them look like they're bouncing off a layer of armor plate instead. You'll also get that effect when your standard project compounds ride in on some transport system, then you make some small structural change which causes your drug to lose its train ticket.
It's a lot of work to figure out what's going on, and often you can't get a handle on it, anyway. Many of these transport systems don't have specific inhibitors, so it's not like you can switch them off one by one to see which one is the problem. If you have a good way to monitor your compound on a cellular level (like a fluorescent probe), you can actually see the things going in and being pumped back out sometimes, or you can see if the transport system can be saturated as you load up on drug. But there's no way you can do this for hundreds of drug candidates on every project.
So, it's just one of those things. I'm on a project right now that has the same thing going on. We make tiny changes to our molecules, and the cell activity suddenly gets a hundred times better, or a thousand times worse. But are these trends going to translate to the cells inside a real animal? And if they do, will they be relevant to the active transport systems in humans? Bite your tongue.
Want to feel old? Then let it hit you that there are entire scientific journals you used to read that don't exist any more. I was looking up an old paper the other day in a German chemical journal, when it struck me that there aren't as many of those as there used to be. The grand old Chemische Berichte, for example, ceased to exist a few years ago (along with a whole group of other publications) and was subsumed in the rather blandly titled Chemistry: A European Journal. Even the journals that (theoretically) publish papers in German don't have as many, like Synthesis or Helvetica Chimica Acta. When's the last time anyone saw a German-language paper in Tetrahedron Letters? Do they even accept them any more?
They used to publish them, years back, and French as well. The French chemistry literature is in even worse shape than the German, because it didn't start from such a position of strength. I used to pull the occasional paper out of Comptes Rendus in a literature search, but that journal spent decades sinking slowly into obscurity. Another outlet for French was one of my old favorites, the Benelux journal with the flowery name of Recueil de Traveaux Chimie du Pays-Bas. The older generation can remember when it formed from the separate Belgian and Dutch society journals (surely there wasn't a Luxembourg one?), but not me. It, too, disappeared into the aforementioned Chemistry.
No, for a long time now publishing chemical work in anything other than English has been the sign of second-rate material. That goes for Japanese and Chinese chemistry, too. There are a number of "internal" journals in each country, but the good stuff appears in English. My literature searches in SciFinder or Beilstein still turn up journals I've hardly heard of, but the papers are almost always in English, or an approximation thereof.
And at this point, it takes a lot to come up with one that I've hardly heard of, I can tell you. Journal of the Siberian Oil Chemist's Association? I've seen that one, along with most of the others from the old parallel-universe world of Soviet science publishing. The Royal Thai Chemical Society's journal? Check. That old weekly Swiss medical bulletin? Yep. The trickiest one I've had in recent years was the house research organ of a New Zealand medical school, but an interlibrary search turned it up.
Mind you, there are a few journals we still have with us that could stand to disappear. I hate to say it, but when was the last time anyone paid any attention to anything in Accounts of Chemical Research? Why is it that I can count on one hand the number of papers I've ever needed from Biochemistry? The continued existence of Synthetic Communications has long been a mystery to me as well. Perhaps some of these will provide nostagia for the next generation.
Little time for blogging the last day or two. I've been finishing up a paper to send to Bioorganic and Medicinal Chemistry Letters, the first one I've written in a while, and I'm checking over another paper that I'm a coauthor on. We're deciding on where to send that one - the lead author suggested Tetrahedron Letters as a possibility, and I thought "Hmm. I haven't had a paper in Tet. Lett. for quite a while." Which was true - a moment later, I realized that the last time was twenty years ago this year! I'm just glad that I usually don't feel as old as that makes me sound. (Talk to me twenty years from now.)
Matthew Holt has a long article on his site that's well worth reading. I'm going to write in response to it this week, because I think a few of its assumptions are incorrect, but it's a good piece nonetheless. It's yet another in the saga of drug prices and research costs, whichs bids fair to be an inexhaustible topic. I am not, though, an inexhaustible blogger. After this round, I'm going to take some time off the topic to recharge my argumentative batteries.
I have a number of other topics backed up in my queue. And I'm going to start off a new occasional feature, a complement to my "How Not to Do It" series of lab stories. This one will be "Things I Won't Touch", and will feature a different reagent each time that I refuse to ever work with. It's a fairly lengthy list, and I'm only a moderately cautious guy. (If anyone else out there has made fluorosulfonic acid from scratch, starting with concentrated hydrofluoric acid and KOH pellets, I'd be glad to hear from you. We can start a club. Admittedly, I was young and foolish at the time, but I made it through without destroying any property. Mostly.)
And one more thing tonight: I'd like to thank everyone for making February by far my highest-traffic month ever. There were about 25,000 page views, which is a roundoff error for the likes of Glenn Reynolds, but thoroughly broke my old record. Much of that was due to my broadside against Gregg Easterbrook, which he certainly seemed to recover from nicely.
I don't want to give the impression that there are hundreds of gems buried among the papers that no one references. Sometimes no one references them because they're not worth very much, or because no one can get ahold of the actual article. I had a old reference turn up the other day from the local "Proceedings" journal of an obscure Egyptian university - I should have threatened our library staff with a photocopy request. You probably couldn't find it short of London; I seriously doubt that any reference library on this continent has a copy. Heck, you'd probably have trouble tracking it down in Egypt. No one will ever know if it's any good.
But most journal articles in chemistry just disappear from view, because they say what they have to say and get off the stage: "We made compound Z for the first time," or "This palladium catalyst is great when you have exactly the sort of starting material that we have," or "Sometimes this reaction works well, and sometimes, darn it all, it doesn't."
These aren't groundbreaking classics, but they're still valid work. And thanks to modern literature-searching tools, they'll be found whenever someone might really need them (if ever.) Some paper that sits composting quietly for years can suddenly turn out to be vital for another researcher who wasn't even born when it first appeared (I've been that researcher a couple of times myself.) At least twice in my career I've gone to copy a paper out of a bound volume of an old journal and realized that a few years before I'd copied the paper right next to it, for a completely different research project. Last time that happened, I looked at the next paper after that one, wondering if I'd need to come copy it a few years from now. (On closer inspection, I hoped not.)
I've always enjoyed being back in the wilderness of the bound journals in a large library. Of course, as time goes on, I can't help but notice that some of these journals that I can remember seeing seeing as new issues are now in the back storage room. Hmmm. You mean to say they've filled out this entire shelving unit with Journal of Organic Chemistry since I was reading it my senior year of college? Let's see, at this rate, it'll be out to. . .here by the time I retire. Hey, J. Alfred Prufrock measured out his life with coffee spoons; things could be worse.
Spent some quality time in the library at work today, digging into another aspect of a project that I'm working on. As you get deeper into the literature on a given scientific subject, some things happen over and over. There will be articles that everyone refers to, the standards that are like showing a form of ID: "Yes, you can take me seriously, because I'm referring to the big papers that everyone in this field should know about."
And there will be papers that somehow got lost in the shuffle, things that are a lot more important than they look, that no one paid enough attention to. You really have to know the field well to recognize these when you see them. And there's always a nagging doubt: "Why doesn't anyone talk about this? What's wrong with it, anyway? If it were good, people would have referenced it. . .right?" The literature-searching tools we have now are gradually giving these papers a better chance to be noticed, but if they're published in an out-of-the-way journal, they still won't get read the way they should be.
Sometimes these papers are lost more in time than in space. More than once I've found that a topic I thought was the latest rage had been anticipated years before. Sometimes the nomenclature has changed enough so that people don't realize that the earlier work is relevant, and sometimes people just don't bother looking at the old literature. A recent project of mine turns out to have relevant papers from thirty years ago, which is remarkable since the same underlying idea is still of interest. Very few modern papers reference these at all; you have to look closely.
What strikes me every time I learn more about a field, though, is how the details start resolving as I get closer. From a distance, when you don't know much about an area, the main points of it look large and chunky: Enzyme X is involved in doing Reaction Y, and it's found in tissue Z. Then as you start to get into the primary literature, all these start breaking into pieces. . .turns out there are several subtypes of Enzyme X - at least, some people say there are, but this other group says that they can't verify that. . .and it does Reaction Y, all right, but it can also do three others, one of them both in forward and reverse - seems to vary depending on the species you look at. . .and here's a paper saying that it's in tissue Z, sure, but it's a lot more important in this other organ where you can barely find it. . .and so on. All these solid rocks of knowledge start turning into mica, exfoliating into piles of complicated details.
The same thing happens when science as a whole approaches a new area. Broad ideas are all we can see at first, but further inspection is rewarded by puzzling anomalies. We may come to a point where we understand things, but the complexity just keeps on increasing the closer we look. It's like a fractal image - just when you think you can see the outline of the whole thing, you realize that those curves have tiny curves on them, which really look as if they themselves have. . .and so it goes. It's just how the world is put together. And I have to say, it would be a lot less interesting if it were easier to understand.
My earlier gloss of the Journal of Medicinal Chemistry as "Jay Med Chem" prompts me to provide a complete guide to faking your way into sounding like a professional organic chemist. Not that that's the road to fame and fortune, but you never know when it'll come in handy.
The first lesson is the lingo of the literature. Journals are often referred to in shorthand: The Journal of the American Chemical Society is known as "Jay-Ay-Cee-Ess," or, more commonly "Jacks." Listening to chemists talking, you'd think some guy named Jack ran a prestigious journal. The Journal of Organic Chemistry is, similarly, "Jay Oh Cee," but never "Jock" and certainly not "Joke." There are journals that deserve that last nickname, but not JOC.
Other "Journal" contractions are made the same way: "Jay Het Chem," "Jay Fizz Chem," and the like. That latter one doesn't come up in organic chemistry conversations much, so be advised. The Royal Society journals would be a mouthful if they weren't known by their nicknames - imagine having to say "Journal of the Chemical Society Perkin Transactions I" instead of "Perkin One."
Another common name for journals is "Bulletin. . .etc.," but that doesn't lend itself to much shortening. I have to confess, though, that every time I look up a paper in the BCSJ that I think "More bull from the Chemical Society of Japan. . . " Another Japanese journal, Chemical and Pharmaceutical Bulletin(their version, roughly, of J. Med. Chem.) comes in for similar mental abuse, not fully deserved.
Tetrahedron Lettersis "Tet Lett" (but its longer stablemate Tetrahedronis never "Tet.") Organic Lettersis pretty new, but I've already heard as "Org Letters" (but not "Org Lett," for some reason.) Synthetic Communications,in those rare times it comes up in conversation, is "Syn Com." and Chemical Communications is "Chem Com." Noting all this, the folks over at Synthesiscut to the chase a few years ago when they named their new short-communications journal
Some of the titles are hard to deal with. I've never heard Tetrahedron Asymmetryreferred to as "Tet Asym," but that's because I've hardly ever heard the journal referred to at all. gets shorted to "med chem" in the middle, but that's still a mouthful. Sometimes you hear it referred to as "Bioorganic" only, but that invites confusion with its longer partner journal, which is just plain At least it would, if anyone ever read the long one.
If you're going to fake your way through a medicinal chemistry conversation, be sure to drop some more biology-oriented journals into your mix. Many of the better ones have one-word names that don't have to be contracted. Science, Nature,and Cell speak for themselves, for example. But a quick nod to "Pee Enn Ay Ess" for the Proceedings of the National Acadamy of Sciences or "Jay Bee Cee" for the Journal of Biological Chemistrywill establish your credentials. Note that that last one, despite the name, is a torrent of densely packed biology from start to finish, with not much chemistry in sight.
That should do the trick. At some later date, I'll get everyone outside the profession up to speed on the acronymic jargon of the lab itself, which led to the following conversation one day at my lunch table: "I used DDQ in THF to try to take off my PMB, but the THP keeps coming off, too." (From the far end of the table) "BFD."
There's an interesting post over at A Dog's Life about the comments that the reviewers of scientific papers make. These comments are (ahem) supposed to be confidential, but good ones do get around.
I don't have enough of those to contribute much to Gregory Hlatky's list, but I do have one that shouldn't be left out (although it's one of those jokes that's largely only funny to chemists.) A famous prize-winning chemist, unnamed by me, was being visited by the second-hand-source of my story. Although getting old, the famous one was still publishing plenty of papers - perhaps more papers than some people wanted to read.
In fact, he'd just had a paper rejected by the Journal of the American Chemical Society, where he was accustomed to publish. "Look at these referee comments!" he said, showing the reviewer's report to my source. "Can you believe they can write things like this?" This fellow's biggest problem wasn't believing it; it was keeping a straight face as he read: "This paper should be substantially reduced. Either that, or it should be completely oxidized."
of such, very interesting for what it shows you about the ways that human reason can go off the rails.
?
. (If you can't, you're not as well-informed as you think you are). And there is indeed a lot of good science reporting in newspapers and magazines, although we can't ignore the fact that there's an awful lot of 
.
. Many votes also were cast for Camille Wermuth's 
. For getting up to speed, several readers recommend Graham Patrick's 
. And an older text that has some fans is Richard Silverman's 
.
by Neal Anderson and 
by Stan Lee (no, not that Stan Lee) and Graham Robinson.
by Rick Ng and also in Walter Sneader's 
.
.
and 
by Nogrady and Weaver.
by Kerns and Di. For getting up to speed in this area, there's 
by Donald Birkett.
. Since all of us make a fair number of poisons (as we eventually discover), it's worth a look.
The perfectly reasonable-looking diazabicyclo compound shown here, for example, has no references in SciFinder, but can be purchased on a multigram scale. (There are about fifty derivatives of that bare scaffold known in the literature, which makes it pretty much uncleared ground compared to the absolutely pulverized IP landscape around, say, piperazine). Next time you're searching for such things, refine your answer set to give only those compounds with no references, and take a look at how many of them are commercially available anyway. . .
There’s a rather embarrassing note 
