About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« More Binding Site Weirdness | Main | Data, Raw and Otherwise »

December 1, 2009

Climategate and Scientific Conduct

Email This Entry

Posted by Derek

Everyone has heard about the "Climategate" scandal by now. Someone leaked hundreds of megabytes of information from the University of East Anglia's Climatic Research Unit, and the material (which appears to be authentic) is most interesting. I'm not actually going to comment on the climate-change aspect of all this, though. I have my own opinions, and God knows everyone else has one, too, but what I feel needs to be looked at is the scientific conduct. I'm no climatologist, but I am an experienced working scientist - so, is there a problem here?

I'll give you the short answer: yes. I have to say that there appears to be several, as shown by many troubling features in the documents that have come out. The first one is the apparent attempts to evade the UK's Freedom of Information Act. I don't see how these messages can be interpreted in any other way as an attempt to break the law, and I don't see how they can be defended:

Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise. He's not in at the moment - minor family crisis. Can you also email Gene and get him to do the same? I don't have his new email address. We will be getting Caspar to do likewise.

A second issue is a concerted effort to shape what sorts of papers get into the scientific literature. Again, this does not seem to be a matter of interpretation; such messages as this, this, and this spell out exactly what's going on. You have talk of getting journal editors fired:

This is truly awful. GRL has gone downhill rapidly in recent years.
I think the decline began before Saiers. I have had some unhelpful dealings with him recently with regard to a paper Sarah and I have on glaciers -- it was well received by the referees, and so is in the publication pipeline. However, I got the impression that Saiers was trying to keep it from being published.

Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted. Even this would be difficult.

And of trying to get papers blocked from being referenced:

I can't see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow - even if we have to redefine what the peer-review literature is !

Two questions arise: is this defensible, and does such behavior take place in other scientific disciplines? Personally, I find this sort of thing repugnant. Readers of this site will know that I tend to err on the side of "Publish and be damned", preferring to let the scientific literature sort itself out as ideas are evaluated and experiments are reproduced. I support the idea of peer review, and I don't think that every single crazy idea should be thrown out to waste everyone's time. But I set the "crazy idea" barrier pretty low, myself, remembering that a lot of really big ideas have seemed crazy at first. If a proposal has some connection with reality, and can be tested, I say put it out there, and the more important the consequences, the lower the barrier should be. (The flip side, of course, is that when some oddball idea has been tried and found wanting, its proponents should go away, to return only when they have something sturdier. That part definitely doesn't work as well as it should.)

So this "I won't send my work to a journal that publishes papers that disagree with me" business is, in my view, wrong. The East Anglia people went even farther, though, working to get journal editors and editorial boards changed so that they would be more to their liking, and I think that that's even more wrong. But does this sort of thing go on elsewhere?

It wouldn't surprise me. I hate to say that, and I have to add up front that I've never witnessed anything like this personally, but it still wouldn't surprise me. Scientists often have very easily inflamed egos, and divide into warring camps all too easily. But while it may have happened somewhere else, that does not make it normal (and especially not desirable) scientific behavior. This is not a standard technique by which our sausage is made over here.

What I've seen in organic chemistry are various attempts to steer papers to particular reviewers (or evade other ones). And I've seen people fire off angry letters to journal editors about why some particular paper was published (and why the letter writer's manuscript in response had not been accepted in turn, likely as not). The biggest brawl of them all was still going early in my career (having started before I was born): the fight over the nonclassical norbornyl cation, the very mention of which is still enough to make some older chemists put their hands over their ears and start to hum loudly. That one involved (among many others) two future Nobel Prize winners (H. C. Brown and George Olah), and got very heated indeed - but I still don't recall either one of them trying to get journal editors fired after publishing rival manuscripts. You don't do that sort of thing.

And that brings up an additional problem with all this journal curating: the CRU people have replied to their critics in the past by saying that more of their own studies have been published in the peer-reviewed literature. This is disingenuous when you're working at the same time to shape the peer-reviewed literature into what you think it should look like.

A third issue I want to comment on are the problems with the data and its analysis. I have deep sympathy for the fellow who tried to reconcile the various poorly documented and conflicting data sets and buggy, unannotated code that the CRU has apparently depended on. And I can easily see how this happens. I've been on long-running projects, especially some years ago, where people start to lose track of which numbers came from where (and when), where the underlying raw data are stored, and the history of various assumptions and corrections that were made along the way. That much is normal human behavior. But this goes beyond that.

Those of us who work in the drug industry know that we have to keep track of such things, because we're making decisions that could eventually run into the tens and hundreds of millions of dollars of our own money. And eventually we're going to be reviewed by regulatory agencies that are not staffed with our friends, and who are perfectly capable of telling us that they don't like our numbers and want us to go spend another couple of years (and another fifty or hundred million dollars) generating better ones for them. The regulatory-level lab and manufacturing protocols (GLP and GMP) generate a blizzard of paperwork for just these reasons.

But the stakes for climate research are even higher. The economic decisions involved make drug research programs look like roundoff errors. The data involved have to be very damned good and convincing, given the potential impact on the world economy, through both the possible effects of global warming itself and the effects of trying to ameliorate it. Looking inside the CRU does not make me confident that their data come anywhere close to that standard:

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!

I do not want the future of the world economy riding on this. And what's more, it appears that the CRU no longer has much of their original raw data. It appears to have been tossed over twenty years ago. What we have left, as far as I can see, is a large data set of partially unknown origin, which has been adjusted by various people over the years in undocumented ways. If this is not the case, I would very much like the CRU to explain why not, and in great detail. And I do not wish to hear from people who wish to pretend that everything's just fine.

The commentator closest to my views is Clive Crook at The Atlantic, whose dismay at all this is unhidden. I'm not hiding mine, either. No matter what you think about climate change, if you respect the scientific endeavor, this is very bad news. Respect has to be earned. And it can be lost.

Comments (171) + TrackBacks (0) | Category: Current Events | General Scientific News | The Dark Side | The Scientific Literature


1. Andrew30 on December 1, 2009 9:30 AM writes...

It is understandable that many people have latched on to the emails, but in their defense the people at CRU indicate that the emails are ‘without context’ or somehow ‘normal banter’ in a scientific institution.

The program code however is different.

It is the actual program code, the modeling code that contains the most damaging evidence. I am not talking about the 'comments' in the code but rather the actual computer program source code itself.

Unlike comments and emails the computer code can only be interpreted in one way. Unlike the comments and the emails the computer code is whole unto it self and requires no external context.

So now everyone has the code.

However now CRU have somehow ‘lost’ the world’s raw climate data that they used in their modeling.

It may have been necessary for them to have lost the raw temperature data. If the raw temperature data was available then they might be asked to reproduce Exactly The Same Results, in front of skeptical witnesses, as they had used in their peer-reviewed publications that were distributed to the world. This might have been impossible without using some infected modeling code, which an investigating scientist might discover.

If the results can not be reproduced the paper that used the results should be withdrawn. Then every paper that cited that paper, and so on until the whole web of pseudo-science that can be traced back to the original fabrication has been purged from the libraries

It is not scientific unless an independent body can reproduce the results.

For information on the possible code infection see:

Anthropogenic Global Warming Virus Alert.

Permalink to Comment

2. wei on December 1, 2009 9:37 AM writes...

they should never put it down in writing

Permalink to Comment

3. Evorich on December 1, 2009 9:51 AM writes...

The climate change issue has become the world's most powerful religeon. Science has nothing to do with it anymore and hasn't done for quite some time.

Permalink to Comment

4. Edward on December 1, 2009 9:53 AM writes...

If raw data were destroyed by a scientist in any other field there would be calls for their head to roll, funding be cut, etc. but as long as you are "saving the world" apparently the rules do not apply to you.

Permalink to Comment

5. RB Woodweird on December 1, 2009 10:07 AM writes...

I really haven't been following this story, mostly because I used my psychic ability to see where it was going. No reason to pay attention. It was going to be a big long "oh noes, some scientists are cheating douchebags, therefore there is no global warming", this mostly rolling off the drums of shills working for industries guarding their bottom lines.

How much of the poor state of the data - missing, incomplete - can be traced back to refusal to fund such work in the first place, stonewalled by the usual crowd?

If our lunchroom banter and hallway small talk was collected and leaked, how big of a brick would the FDA, the EPA, and maybe the ATF pass?

This data that was lost. Was it all the climate data ever collected? Or was it just some kept by this University of East Anglia place.

And where the hell is the University of East Anglia anyway?

Permalink to Comment

6. alig on December 1, 2009 10:14 AM writes...

Unfortunately, the Univ. Of E Anglia CRU data was considered to be the best data. Now it has come to light that the raw data has been missing for 20 years? WTF.

Permalink to Comment

7. Derek Lowe on December 1, 2009 10:21 AM writes...

RB, as I mentioned in the post, I'm not going to get into the is-global-warming-real aspect of things at all. Even the firmest believer should be shaken up by the CRU story (as witness George Monbiot).

That's because the data from this center is a key piece of the UN's IPCC recommendations. You may not have heard of the University of East Anglia, but its data set is actually very important indeed.

And I haven't even linked to the "lunchtime banter" stuff, because I agree that it's trivial. What's left, though, isn't trivial at all.

Permalink to Comment

8. Jonathan on December 1, 2009 10:22 AM writes...

Your points might have more validity if the CRU was the only place in the world doing climate research or the only place in the world that's showing rising temperatures etc. Yet everyone else is seeing the same trends and coming to the same conclusions. Sure, the tone in the emails is impolitic but I'd bet you'd find the same if you hacked into the accounts of people working in, say, the autism field discussing the lunatics who believe that vaccinations are the cause. Just a guess, but I imagine you have less sympathy for the vaccine-autism movement getting their papers published and you'd be more charitable if these leaked emails were discussing getting a crank off an editorial board who thought vaccines caused autism.

And that's without bringing up the PNAS scandal.

I would refer you to John Timmer's rather excellent post on the matter over at Ars Technica:

One of the problems caused by the e-mails is that the scientists involved aren't discussing their data and its analysis using scientific terminology; instead, things come across more as what you might hear in an office environment. In short, the scientists sound like regular human beings (more on that below). When faced with two different data sets that provide different answers, the e-mails don't phrase things in terms of "what scientifically valid adjustment can be made to bring these two data sets into agreement?" Instead, the authors consider the problem in terms of how they can make the discrepancy go away.

Similarly, it's apparently widely recognized that, although tree ring data nicely tracks the temperature record for roughly a century, it diverges after 1960, when the modern rise in temperatures started. So, in a variety of papers, researchers have presented the instrument record, either superimposed or instead of the tree ring data, for periods where it's available (and clearly labelled the graphs accordingly). In the e-mails, this is described as a "trick" to "hide" the problem.

All of this is more pronounced when the data is preliminary, and researchers may not yet know how to interpret it or fit it into the larger body of existing data. That will get smoothed out by the time the data eventually gets published, but preparing data for publication is generally a small portion of an entire research project, and the e-mails largely reflect the longer period when confusion and frustration dominate.

As a result, the e-mails sound awful. But, the unfortunate truth is that this is the way scientists talk. "Lab-speak is full of shortcuts," said physics researcher and Ars contributor Chris Lee. "The way I discuss things internally is not the same way as I present them to the rest of the scientific community." And my experience from biology is that if I heard a coworker mention they had a trick to get better data from mouse embryos, I'd assume they were talking about a microscopy technique, not scientific fraud.

Permalink to Comment

9. Wavefunction on December 1, 2009 10:24 AM writes...

Here's my take on this. The climate change scientists are in a real dilemma. The climate is a complex system and it won't be surprising if they keep on finding major and minor errors and disagreements in their work. In fact that's what you would expect when you are scientifically studying such a complex system. Nothing wrong with that at all.

But the real problem is that even if they find these problems, they are going to fear going public with them because of the political climate. There is no dearth of right-wing extremists who are just poised to pounce on any such discrepancies and declare the entire enterprise futile. There is also no dearth of left-wing extremists who will disown the scientists and declare them to be paid corporate shills. All this is forcing the scientists to stay silent in my opinion even when they know better.

Of course that's bad science, since science only proceeds by way of disagreements and by making all of your data public. But the economic stakes with climate change are so high that good politics is often trumping good science.

Call it whatever else you will, but science is the real casualty here. And as scientists we are part of the casualty too. That's what's sad.

Permalink to Comment

10. MDW on December 1, 2009 10:30 AM writes...

One thing often overlooked regarding this mess is the selection effect involved in driving the makeup of the entire AGW field. That is, the very idea of 'climate science' is relatively new. There are of course communities of scientists who study weather, various physical aspects of the earth's history, etc., but specifically studying the history of the Earth's temperature and weather patterns is a recent phenomenon. The field of climatology itself has been rather politicized from the start ('global cooling' decades ago, the ozone hole) and has been associated with the green movement from early on. So, what sort of young person would tend to go into this field? If you are thinking about studying something in this general area, wouldn't your politics and general beliefs influence you to move toward or away from climate studies? As the field became more high-profile and overtly political, wouldn't that effect deepen? Over time, what sort of person would you expect the field to be comprised of? Do you think it would be pretty heterogeneous, or not? Ask yourself a similar question: what sort of person becomes a poststructuralist?

Ah, you say. But these people are scientists. That will cleanse the field of misapprehensions. This thought misses the point. Scientists are people, and all the bad aspects of people go along with that. It is only the scientific method and specifically the rigor of repeated and repeatable experimentation that allows us to muddle in a direction we can safely refer to as ‘forward.’ Without that, you wind up with something like particle physics, which without the benefit of experimentation has for decades now wandered off into a playland of mathematics where symmetry, beauty, and indefinable feelings of correctness are valued, and some of the very smartest people in the world have been reduced to having a quasi-religious stance toward their, and their fellow scientists’ work. Look at economics, where the arcane of mathematical models has led the field to a point where the field as a whole can agree on very few propositions, and where ideas such as MMT, which is a very simple collection of obvious statements, can be summarily ignored. Why? How? Because it is hard to check theory with experiment, and given the choice, people will continue to believe what they want to believe. People with careers spent defending Propositions X and Y will naturally have a hard time coming around to the view that actually, Propositions A and B seem to have much more evidence behind them. In fact, they will tend to fight that change vigorously. There is a constant tension between what people want and need to believe and what people are forced to accept. It is so, so much easier to be wrong than to be (tentatively) right. Given the chance, people will almost always lead themselves astray. It is just too difficult to understand something new. A community of people can be persuaded of just about anything, as history has shown – need we list some examples? Without being forced – I think that is an appropriate word – to constantly check our ideas vs what we charmingly call ‘reality,’ any group of ‘scientists’ will eventually become nothing more than a group of people with the same hobby.

And so we come to climate science. You have a fairly small group of people strongly predisposed to believe in a particular idea; a group most of whom are invested from the start in such an idea. You have an external environment which copiously rewards you for holding such ideas. You have limited data, much of which has proved to be of questionable enough quality that even the people in charge constantly employ fudge factors for it. You rely on models which are clearly incapable of robust predictions, and which again can and must be constantly tweaked with ‘improved’ assumptions. You have, in short, a giant socially constructed epicycle-building machine.

What do you think will happen?

Is it somehow wrong to argue in this way? Perhaps we should let the data decide. Evidencing reasons are far, far superior to motivating reasons. I agree. But much of the evidence looks far from evidentiary, much of it we can’t look at at all, and the rest we have to take people’s word for, and/or assume some model is correct.

Ultimately, evidence will decide things. But we have very, very good reason to think that evidence may not be what has decided things to this point. And it is entirely reasonable to argue THAT point based on motivating reasons. Because science is hard, and people are weak.

Permalink to Comment

11. Anonymous on December 1, 2009 10:31 AM writes...

book smart vs. common sense

Permalink to Comment

12. road on December 1, 2009 10:44 AM writes...

In my own field, I can read someone else's paper and decide if their assumptions and data-manipulations are bullshit or not. I can't do that in other fields because I'm not intimately familiar with the experimental techniques. I think most people probably feel the same -- that it's hard to be objective and critical of research in other fields. I have NO IDEA if the climatology data really support the claims of global warming. And frankly, neither does anyone else, unless they've spent a lifetime studying it. HOWEVER, for some reason, just about everyone who's spent a lifetime studying climatology data appears to come to similar conclusions and to me that's very convincing. You can have all the data-scandals you want, but I'm betting that there's a reason that climate-scientists believe in anthropogenic climate change.

It's the same issue with the evolution-deniers: Anyone that spends their life looking at DNA sequences and orthologies couldn't even imagine an alternative to the theory that evolution proceeds randomly through mutation. However, the public, which doesn't look at data, can argue about it for decades. On that issue, it's clear to me that familiarity with the raw data engenders obvious conclusions and I assume climate-change is the same.

And also, Derek, it's really not fair to claim that the nonbornyl cation controversy was even remotely similar to something like this. I'm not defending these particular scientists, but a lifetime of having CNN report-on and and mis-interpret every single paper you published would probably push even the most level-headed scientists to focus on persuasive data presentation.

Permalink to Comment

13. mrr on December 1, 2009 10:46 AM writes...

Thank you Derek for this, your finest post.

Permalink to Comment

14. Hap on December 1, 2009 10:54 AM writes...

I can see reasons for not wanting to publish papers with questionable hypotheses but some data - in politically charged fields, the presence of a paper (regardless of its quality) is likely to be taken as evidence of the correctness of its underlying theory, and used as such among people who are unlikely to know its context and evaluate its quality. The alternative (excluding papers with theories that disagree with the dominant one) seems like a much worse idea.

UEA seems to be behaving in precisely the way that people who disagree with AGW have been claiming. Since lots of people can't evaluate the evidence for/against AGW (and lots of those are the people determining what to do in response to it), trust and integrity are really their main assets. If you lose those, you don't have anything - people can't trust your data, and they can't trust what you generate from either your data or that of others. It gives your opponents a shining example of behavior to use against anyone who thinks as you do. As such, it seems patently stupid.

Science is supposed to be a way of thinking and testing our thoughts that makes it hard to fool ourselves (as others have put it), and marrying our theories to achieve the political ends we believe are needed because of them makes it harder for us to test our theories and ourselves. In politics, certainty is valued, often over truth, but that's a dangerous value order for science (and for politics, as well), and the people at UEA ought to have understood that. (Voters ought to understand that, at some point, but uncertainty isn't easy for most people to handle.)

Permalink to Comment

15. CB on December 1, 2009 11:03 AM writes...

It seems clear that these idiots should be run out of town, just based on the email exchange. I agree with Edward, heads should roll.

In the long run the best scientific models will win out!

Unfortunately, the climate skeptics arguments are very similar to creationists, or spritualists arguments against the various aspects of science they take a dislike. They have no testable models, they make no predictions and they generate no empirical data.
I have seen no serious efforts to generate more predictive climate models that are not reliant on greenhouse gases.

This is surprising since 1) copious research funding from the fossil fuel industry would certainly be available 2) the surest path to fame and fortune in science is in slaying myths. 3) predictive climate models provide extremely valuable information that can be used by agriculture, finance, insurance, govt. etc.

Permalink to Comment

16. BACE on December 1, 2009 11:07 AM writes...

I know this post is not about global warming per se but I would recommend an objective and well-written book that describes the history of the science that led to the global warming consensus- Spencer Weart's "The Discovery of Global Warming"

Permalink to Comment

17. rob on December 1, 2009 11:10 AM writes...

Derek says the results of climate science have to be very good and very convincing if we're going to believe them.

I agree.

So where are the safety data showing that pumping gigatons of CO2 into the air is an ok thing to do? Where are the coal companies' data and models? The oil companies'?

And can we please plow through the last N years of internal emails from these companies as well? Or those from the lobbies they fund?

Why don't you hold polluting companies to a standard that even remotely approaches the one you hold climate scientists to? Or even the one that-- you say-- pharma companies routinely meet. After all, as you rightly say, the stakes are enormous.

How do you justify using different standards for different institutions?

Permalink to Comment

18. Anonymous on December 1, 2009 11:18 AM writes...

My question is how many other global temperature records are there? Basically to get at what percentage of climate scientists rely on the CRU global values wich now seem to be backed only by "lost" data. I realize that this data is considered "very important" but how much? If only 20% of the literature relied on this data then with the remaining 80% one can still pull together some analysis. But if say 70% or 80% of the literature studies relied on this data analysis of the collective body becomes much more difficult.

Permalink to Comment

19. Robert Bruce Thompson on December 1, 2009 11:19 AM writes...

At this point, I'm not even sure that "climate science" has made the jump from a pseudoscience to a protoscience, but the Hadley/CRU mess is a strong argument that it's still in the former category. Given that Hadley/CRU was one of the two major sources of historical climate data and that both of those sources have now contaminated each other to the point where none of their data are trustworthy, it seems that the question is no longer whether global warming is anthropogenic, but whether it is even occurring at all.

The real cost of this mess can't be calculated, because all scientists will be tarred with the same brush. Already, the creationists and IDiots are using the Hadley/CRU mess to attack evolution on the basis that scientists are proven liars.

Permalink to Comment

20. David P on December 1, 2009 11:24 AM writes...

I hate the fact that the way of doing science is undermined by this scandal.

I had (have still even) hopes that climate change would be the thing that got developed countries to be less wasteful, which seems to me to be a good thing regardless of the effect on the planet.

Oh, and since no one else answered, University of East