About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Does That Answer Your Question? Not Quite, No. | Main | MedChemica: When One Compound Collection Isn't Enough »

July 16, 2013

Touching Up the Spectra

Email This Entry

Posted by Derek

Organic chemists have been taking NMR spectra for quite a while now. Routine use came on in the 1960s, and higher-field instruments went from exotic big-ticket items in the 1970s to ordinary equipment in the 1980s. But NMR can tell you more about your sample than you wanted to know (good analytical techniques are annoying that way). So what to do when you have those little peaks showing up where no peaks should be?

The correct answer is "Live with 'em or clean up your sample", but wouldn't it be so much easier and faster to just clean up the spectrum? After all, that's all that most people are ever going to see - right? This little line of thought has occurred to countless chemists over the years. Back In The Day, the technology needed to remove solvent peaks, evidence of isomers, and other pesky impurities was little more than a bottle of white-out and a pen (to redraw the lovely flat baseline once the extra peaks were daubed away). Making a photocopy of the altered spectrum gave you publication-ready purity in one easy step.

NMR spectra are probably the most-doctored of the bunch, but LC/MS and HPLC traces are very capable of showing you peaks you didn't want to see, either. These days there are all sorts of digital means to accomplish this deception, although I've no doubt that the white-out bottle is still deployed. In case anyone had any doubt about that, last month Amos Smith, well-known synthetic organic chemist and editor of Organic Letters, had this to say in a special editorial comment in the journal:

Recently, with the addition of a Data Analyst to our staff, Organic Letters has begun checking the submitted Supporting Information more closely. As a result of this increased scrutiny, we have discovered several instances where reported spectra had been edited to remove evidence of impurities.

Such acts of data manipulation are unacceptable. Even if the experimental yields and conclusions of a study are not affected, ANY manipulation of research data casts doubts on the overall integrity and validity of the work reported.

That it does. He went on to serve notice on authors that the journal will be checking, and will be enforcing and penalizing. And you can tell that Smith and the Org Lett staff have followed up on some of these already, because they've already had a chance to hear the default excuse:

In some of the cases that we have investigated further, the Corresponding Author asserted that a student had edited the spectra without the Corresponding Author’s knowledge. This is not an acceptable excuse! The Corresponding Author (who is typically also the research supervisor of the work performed) is ultimately responsible for warranting the integrity of the content of the submitted manuscript. . .

As the editorial goes on the say, and quite rightly, if a student did indeed alter the spectrum before showing it to the boss, it's very likely because the boss was running a group whose unspoken rule was that only perfection was acceptable. And that's an invitation to fraud, large and small. I'm glad to see statements like Smith's - the only ways to keep down this sort of data manipulation are to make the rewards for it small, increase the chances of it being found out, and make the consequences for it real.

As for those, the editorial speaks only of "significant penalties". But I have some ideas for those that might help people think twice about the data clean-up process. How about a special correction in the journal, showing the altered spectra, with red circles around the parts that had been flattened out? And a copy of the same to the relevant granting agencies and department heads? That might help get the message out, you think?

As an aside, I wanted to mention that I have seen someone stand right up and take responsibility for extra peaks in an NMR. Sort of. I saw a person once presenting what was supposed to be the final product's spectrum, only there were several other singlet peaks scattered around. "What are those?" came the inevitable question. "Water" was the answer. "Umm. . .how many water peaks, exactly?" "Oh, this one is water in solution. And this one is water complexed with the compound. And this one is water adsorbed to the inside of the NMR tube. And this one is water adsorbed to the outside of the. . ." It took a little while for order to be restored at that point. . .

Comments (38) + TrackBacks (0) | Category: Analytical Chemistry | The Dark Side | The Scientific Literature


1. Redlich-Kwong on July 16, 2013 8:12 AM writes...

Finally, someone has the temerity and moral compass set beyond the avaricious pursuit of their "next grant." With any luck it will engender a bit of accountability.

Permalink to Comment

2. A Nonny Mouse on July 16, 2013 8:20 AM writes...

The water story isn't too far fetched if it is a highly chelating type molecule of the type that I have recently been working on (highly substituted phenanthroline with 4 sulfonic acids).Some of these have been shown by X-ray to have quite a few waters/methanol associated with the molecule (yes, I know it's in solution....but see below).

I recently made a pyridine analogue which contained residual ethanol even after drying at 70 under high vacuum. NMR showed that there were 2 sets of peaks for the ethanol- one was hydrogen bonded to the pyridine in the cavity while the rest was unbound.

Permalink to Comment

3. Toad on July 16, 2013 8:27 AM writes...


You need to touch up the post title with an "r". Don't worry, the Google data analyst will keep the original cached forever.

What a blast from the past. I remember the water on the NMR tube walls, and still use it as a teachable moment example of how trying to bs your way through a presentation can adversely affect your scientific reputation within your peer group for a long time.

Permalink to Comment

4. Curious Wavefunction on July 16, 2013 8:38 AM writes...

That last anecdote has the makings of a publication in it. It's actually really interesting to distinguish all those different kinds of water molecules.

Permalink to Comment

5. eugene on July 16, 2013 8:45 AM writes...

The sad thing is, is that having all those minor impurities in the NMR doesn't matter. I've always submitted NMR spectra with new compounds and yeah, sometimes they had minor impurities that were obvious. But I just wrote under the spectrum the shifts in ppm and said 'impurities'. Also, sometimes the impurity is identifiable. I've never had an issue with reviewers bothering me for it. Some of those air sensitive compounds are very hard to purify anyways.

If the elemental analysis is acceptable and the impurity is less than 10% of the signals (obviously in case of a catalyst that should be a lot less), I think it's fine. No need to waste time with the white out either. Just send it out for review.

Permalink to Comment

6. eugene on July 16, 2013 8:54 AM writes...

"That last anecdote has the makings of a publication in it."

No, it doesn't. It's bs. I know you're a theoretical person, but it's clear to us NMR jockeys that the person saying that stuff is making a very bad joke at best.

If you take plenty of NMR, there is no excuse for having water adsorbed on the inside and outside of the tube. Unless you just finished rinsing it and didn't dry it after you put the sample inside. Even then, you're likely to get only one giant blob peak. He can only get away with one water peak, and if it was sharp, there won't be any water adsorbed on the walls (plus, there is a way to prevent it if you really care about it). I'm not even willing to believe their molecule could complex water to give a legitimate second peak, after hearing the rest of the excuses. And that's without even knowing what molecule it was and just reading Derek's anecdote.

Permalink to Comment

7. Curious Wavefunction on July 16, 2013 9:13 AM writes...

#6: So is there no NMR experiment that can distinguish complexed water from bulk? My curiosity is piqued now. The anecdote itself might be a bad joke, but I think it would be really cool if you could actually design an experiment to distinguish between different kinds of water molecules.

Permalink to Comment

8. CMCguy on July 16, 2013 9:31 AM writes...

Perhaps the person who was able to detect these various forms of water then went on to work for Nativis or other homeopathic type cures? Alternatively they could have been reading too much Kurt Vonigate.

Permalink to Comment

9. sciencemonkey on July 16, 2013 9:36 AM writes...

@7, You could run a DOSY (diffusion) experiment to measure complexed vs free water if whatever the water was complexed to had a diffusion coefficient at least ~3 times that of the free water. There is also STD-NMR, but I don't have hands on experience with that method:

Of course you can't hexanes or DCU peaks, but I'm comfortable with ignoring small unidentifiable impurities if they don't matter. However, it can be fun to try and understand these small impurities so you can avoid them the next time around. Some notable examples are some phthalates (how the hell did that plasticizer get in there?), or when I rotovapped from THF a number of times and subsequently collected a nontrivial amount of BHT in my sample.

Permalink to Comment

10. petros on July 16, 2013 9:42 AM writes...

Probably an essential development

Re Eugene, does anyone still do elemental analysis? When we stopped doing it, it ruled J Med Chem out of bounds for many years until they moved with the times.

Permalink to Comment

11. Hap on July 16, 2013 9:43 AM writes...

Unless the singlets were really broad, there's no way they should have been water, right? I thought that because of exchange water peaks were generally broad, and so even if the water wasn't exchanging between bound states, its peaks should still be broad. They would also seem likely to exchange on the NMR timescale, which would broaden them further.

Maybe if you ran experiments on small scale and at low T you could distinguish bulk from complexed water, where bulk and complexed waters aren't exchanging fast and where the ratio of the two doesn't favor the bulk water so much as to overwhelm any complexed water peaks. Depending on the material of the tube, you might have to worry about tube material or its water absorption (polywater?).

Permalink to Comment

12. eugene on July 16, 2013 10:07 AM writes...

"So is there no NMR experiment that can distinguish complexed water from bulk?"

No, of course there are. A normal spectrum can find a complexed water molecule. The hydrogens will exchange fast with bulk water impurity if it is present in the sample, but I guess it depends on the type of complex (maybe some weird cicurbituril might bind it so well that it doesn't readily exchange, but you would be able to measure the exchange rate). You can still see a different signal in my compound, from the water impurity and the complexed water. This doesn't only happen with water. Complexed ammonia will give a different signal that you can differentiate from free ammonia.

It's just that person was giving bs excuses with their adsorbed water on the inside and outside of the tube.


Also it's tough to run experiments on water heavy samples at low T, as it tends to freeze. And faster timescales are no good if the exchange rate is faster than the collection time. You are measuring a timescale before the Fourier transform and it's slow for a lot of things. And then the relaxation rate of the nucleus is going to kill you on multiple scans if the delay between pulses is too small...


I actually agree with you and feel very strongly about the uselessness of the EA and the ridiculous 0.4% difference requirements in journals when the instrumental error is bigger than this. I won't outline my reasons here, but JACS still requires it and a good number of other ACS journals do as well. Obviously, for an organic compound that goes through an extraction, and then a column, and is shown to have a clean NMR... okay, I promised I wouldn't get worked up about it.

Permalink to Comment

13. good lab tech on July 16, 2013 10:40 AM writes...

What happens if all of my NMRs are beautiful and without impurity. Should I "Photoshop-in" some ethyl acetate peaks before submitting to Organic Letters?

Permalink to Comment

14. PFE geek on July 16, 2013 10:57 AM writes...


Permalink to Comment

15. Anonymous on July 16, 2013 11:04 AM writes...

Couldn't they ask for the submitter to send the accompanying .fid file with any submitted spectrum? Even if the .fid isn't compared to the spectrum in review it may scare people from manipulation. Or can .fid's be manipulated as well?

Permalink to Comment

16. Prog Chem on July 16, 2013 11:30 AM writes...


A .fid is simply a data file that the NMR control software outputs and the NMR processing software can read. It can be manipulated just like any other data file, although the author would have to have a really good knowledge of the math behind Fourier transforms and would need a whole lot of time on their hands to either do it manually or to program a system to do it for them.

Which is time that they could have spent purifying their compound.

Permalink to Comment

17. Paul on July 16, 2013 11:33 AM writes...

Can I throw in some extra things to worry about here? As an analyst I am often asked to assess how pure someting is from the NMR spectrum, and the chemists are usually not impressed when I say 'it depends'. Of course they usually want me to ignore the residual solvent peaks, water, ethyl acetete DCM etc. How about that residual DMSO though - can we ignore it if the assay is to be run in DMSO, is is there enough however to significantly change the sample concentration.

Thats before we get to those troublesome baseline peaks, if they are from a known material we can easily quantify the molarity and mass contribution - but unknowns? How many species are present, we can get an idea if we have a spread of integrals to group; but often we are too close to the noise to get a perfect answer. The alternative is to just add up the integrals from all of the minor peaks and use this to give an estimate.

So at the end of all, if you don't make up the sample in a truly quantitative fashion, the answer is still 'it depends'!

Permalink to Comment

18. stop on July 16, 2013 11:48 AM writes...

Food for thought: baseline and phase correcting are commonly accepted techniques in "working up" an NMR spectrum. Am I wrong to assume most people use them? These techniques seem to me to be a lesser form of touching up a spectrum, although, during my duration as a chemist I've seen some impurities nearly "disappear" with a simple baseline correction.

Which brings up two questions:
1. What is the purpose of spectra in publications?
(my answer: to prove you made the stuff - so a few little blips that don't integrate shouldn't be cause for concern; although on second thought, they now make you question the yield)

2. When will .fid files be submitted with the supporting information?
(my answer: I dunno)

Permalink to Comment

19. stop on July 16, 2013 11:50 AM writes...

@16 - I guess I never really thought about manipulation of the .fid file. Today I learned something...

Permalink to Comment

20. MattF on July 16, 2013 12:07 PM writes...

Hmm. Just curious... what do you guys think about signal processing, e.g.,

Considering that you've apparently already got the Fourier transform sitting in a file, I'm a little puzzled that anyone would bother with White-Out.

Permalink to Comment

21. Xero on July 16, 2013 12:26 PM writes...

@2: ...and my boss said to me that during PG deprotection in ethanolic HCl, the compound has formed ethanolate as NMR showed ethanol peaks. I had to start-over the whole route just to change the PG.

Permalink to Comment

22. Sili on July 16, 2013 1:26 PM writes...

Of course 'original' outputs can be manipulated, but it still should be standard to submit all data as supplementary information. There's really no excuse for not doing it in this day and age.

Permalink to Comment

23. Anonymous on July 16, 2013 2:06 PM writes...

@12 JACS doesn't require EA

Permalink to Comment

24. Toad on July 16, 2013 3:40 PM writes...

@21 Xero:

FYI, years ago we had a clinical compound that we med chemists prepared as the HCl salt in EtOH as the last step, and it formed an 1:1 EtOH solvate which was resistant to further removal under a variety of conditions.

The process chemists developed something incredibly simple - they just took up the whole lot of final solid solvate in water and refluxed it overnight. The EtOH solvate was exchanged to a water solvate, which formed nicely crystalline drug substance.

Permalink to Comment

25. li on July 16, 2013 9:45 PM writes...

Hypothetically, how much do you think I could get for a .fid editing program? $200? $500? The math is quite straightforward, and wave-form editors a dime a dozen. Do you think I could ask to be put on paper as co-author, or is that an over-reach?

Permalink to Comment

26. Tamborine on July 16, 2013 10:34 PM writes...

I agree with Eugene in that people (mostly grad students who learn the behavior from older grad students and so on...) are too caught up with having perfectly pure samples. The purpose of characterization data is to provide evidence of a compound's identity and purity -- that's it. It is often misconstrued that the purpose of the characterization data is to establish proof of purity >9x%, but this isn't right. You don't need to show that the compound is above a purity threshold, you simply need to be able to measure and report the observed purity. For example, Procedure X provides compound Y with purity Z. That's it. It tells the complete story of the experiment and of the outcome.

The absolute purity is only important if:

1. a more highly purified sample is required in order to be able to establish compound identity. For instance if the impurities interfere with interpretation of the NMR spectrum (with current 2D techniques, this is becoming less relevant). Or if you need to get an elemental analysis to help prove identity. In the olden days elemental analysis was a key method to establish identity and so compound purity was more relevant.

2. the purity is crucial for the interpretation of some other data or experiment using that compound -- for instance cases where the presence of an impurity can impact further experiments, e.g. if you are claiming biological activity or catalytic activity etc. for a given compound. This is almost never that case for intermediates along a synthetic route and so time spent carefully characterizing intermediates (beyond what is necessary to confirm identity and measure actual purity) is a complete waste of time.

Permalink to Comment

27. Hudlicky Fan on July 17, 2013 1:57 AM writes...

The purpose of a paper is not always just to show that you made something. Purity is extremely important for reporting accurate yields. If a methodology paper is reporting 95% yields, but the NMRs show 90% purity, this is just bad science and, as Amos mentioned in his editorial, reflects poorly on the integrity of the entire work. How many people reverse calculate their yields by doing the algebra to subtract out the solvent peaks? And this method doesn't allow for the determination of inorganic impurities or deuterated solvents in your sample. My PhD supervisor told us to just rotovap our samples down in CDCl3 a few times to get characterization quality spectra.

On a side note, we recently received 1 gram of an expensive ($1400) substance that was labeled 93% pure on the basis of an HPLC trace where many tiny little impurities (1% or less) were integrated and added up. NMR spectra were also included as an indication of purity. Our own NMR spectrum of the material looked quite good. Turns out the sample was quite wet (at least 5% H2O by mass) and, upon trying to dissolve the entire batch for recrystallization, we filtered off a good 200 mg of silica gel.

Permalink to Comment

28. Anonymous on July 17, 2013 2:39 AM writes...

3. You want the yield to have any meaning whatsoever.

Permalink to Comment

29. Anonymous on July 17, 2013 3:04 AM writes...

Weight-weight NMR.

Permalink to Comment

30. Paul on July 17, 2013 3:51 AM writes...

Whilst we are on this topic, how many of you filter your NMR samples? This is common practice, if not compulsory in many academic environments.I have always challenged my chemists to ensure that they use a solvent where everything is in solution! The question is what have you filtered off - and what impact does that have on your reported yield / purity? (possibly that 200mg of silica Hudlicky Fan)

As an aside, for those looking to use accurate NMR quantitation; I have somewhere a paper which calculates that to get a quality weight measurement from a four figure balance you should be weiging out at least 15mg.

Permalink to Comment

31. Hudlicky Fan on July 17, 2013 5:09 AM writes...

After chromatography and concentration, before the final transfer to a small vial for accurate weighing, I always filter through a pipette with a small plug of cotton or Kimwipe. Amazing how much silica gel and dust you remove this way. And then, if you have a relatively pure substance with reasonable solubility, you don't need to filter your NMR sample.

Permalink to Comment

32. rockhopper on July 17, 2013 5:56 AM writes...

O-17 NMR works quite well to distinguish bulk from bound water. The proton exchange on water is usually quite fast on the NMR time scale, so in protic solvents you won't be able to see distinct signal for free and bound water by 1H NMR. You only get an averaged signal. The oxygen exchange is usually much slower so you can see distinct signals in 17O NMR. I worked on metal complexes in aqueous solutions and we could see the free and bound water, and depending on the geometry of the complex distinguish between water molecules in different positions: in a complex like [ML(H2O)5] see the H2O in trans position to L and the ones in cis position. Signal integrals would be in the ratio 1 to 4.

Permalink to Comment

33. Paul on July 17, 2013 8:47 AM writes...

On the unusual water topic, it is easy to demonstrate by NMR that there is a significant quantity of unfrozen water in typical frozen foods at -18C (the freezing process tends to concentrate salts and lipids within the cells to leave compartments where water will not freeze). This leads to faster degradation than would be expected than if you had assumed all of the water content was frozen at this temperature.
I know it's slightly off topic but I found it interesting!

Permalink to Comment

34. Joe Q. on July 17, 2013 1:55 PM writes...

Seems to me that if you want to make a point about compound purity, GC-MS or LC-MS would be better approaches than pursuing some fetishized perfect NMR spectrum.

Permalink to Comment

35. startup on July 17, 2013 2:57 PM writes...

What to do? Simlpe. Find the lousiest, oldest 300 MHz machine in the building and run your samples on it.

Permalink to Comment

36. Canageek on July 17, 2013 4:02 PM writes...

Prog Chem: Sure, but that data will be in some binary format, right? That makes it really hard to edit, and while still possible, would probably cut down on 95% of that type of editing. You could then have some automated system at the journal automatically FT the fid file, and match it to your reported peaks, and flag it for manual review if it doesn't match (due to the fact you'd probably have a fair number of false positives).

Heck, you could probably also do some sort of hashing and timestamping to make it even more challanging to forge. I'm sure someone dedicated could do it, but we've seen that truly dedicated frauds will get though no matter what you do, short of having someone duplicate every result submitted to the paper.

Permalink to Comment

37. mittimithai on July 17, 2013 8:30 PM writes...

A little late to the discussion, but the editor highlights what is actually a computer science problem. There is no fundamental reason why data acquisition and processing software shouldn't be able to keep an audit trail on data (provenance); it is relatively cheap in terms of computing time and storage space.

Doing this in an interoperable way that doesn't burden application developers is the tricky part; but there are solutions:

Journals, scientists and instrumentation developers should be able to form a working group to implement useful standards and tools for provenance for experimental data collection and processing.

Permalink to Comment

38. eugene on July 23, 2013 2:57 PM writes...

@12 JACS doesn't require EA

Sorry, couldn't comment earlier. My comment wouldn't go through. In my field, it is an unstated requirement actually for JACS. The reviewers would demand it for sure. Definitely the specialty ACS journals insist on it.

Hudlicky fan and Paul:

Usually after a column with a sand and glass wool plug, I find that my samples don't have any dirt or silica in them. But if I see something in a fraction (rare, means I made the column badly), then I filter it before rotovapping it down.

"The purpose of a paper is not always just to show that you made something. Purity is extremely important for reporting accurate yields. If a methodology paper is reporting 95% yields, but the NMRs show 90% purity, this is just bad science and, as Amos mentioned in his editorial, reflects poorly on the integrity of the entire work."

Yeah, you report crude yield, and NMR is there for purity? I almost never get yields above 90%, usually in the 70s or 80s if lucky. The purpose of my work is to make something new and show a new reaction, not to do the job of industry in optimizing it to death. I optimize stuff to a reasonable amount. And NMR purity of 95% is fine in my book. Better than a lot of Aldrich starting materials that I get.

Permalink to Comment


Remember Me?


Email this entry to:

Your email address:

Message (optional):

The Last Post
The GSK Layoffs Continue, By Proxy
The Move is Nigh
Another Alzheimer's IPO
Cutbacks at C&E News
Sanofi Pays to Get Back Into Oncology
An Irresponsible Statement About Curing Cancer
Oliver Sacks on Turning Back to Chemistry