About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
Not Voodoo

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
Realizations in Biostatistics
ChemSpider Blog
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Eye on FDA
Chemical Forums
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa

Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
Gene Expression (I)
Gene Expression (II)
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net

Medical Blogs
DB's Medical Rants
Science-Based Medicine
Respectful Insolence
Diabetes Mine

Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem

Politics / Current Events
Virginia Postrel
Belmont Club
Mickey Kaus

Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« More Binding Site Weirdness | Main | Data, Raw and Otherwise »

December 1, 2009

Climategate and Scientific Conduct

Email This Entry

Posted by Derek

Everyone has heard about the "Climategate" scandal by now. Someone leaked hundreds of megabytes of information from the University of East Anglia's Climatic Research Unit, and the material (which appears to be authentic) is most interesting. I'm not actually going to comment on the climate-change aspect of all this, though. I have my own opinions, and God knows everyone else has one, too, but what I feel needs to be looked at is the scientific conduct. I'm no climatologist, but I am an experienced working scientist - so, is there a problem here?

I'll give you the short answer: yes. I have to say that there appears to be several, as shown by many troubling features in the documents that have come out. The first one is the apparent attempts to evade the UK's Freedom of Information Act. I don't see how these messages can be interpreted in any other way as an attempt to break the law, and I don't see how they can be defended:

Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise. He's not in at the moment - minor family crisis. Can you also email Gene and get him to do the same? I don't have his new email address. We will be getting Caspar to do likewise.

A second issue is a concerted effort to shape what sorts of papers get into the scientific literature. Again, this does not seem to be a matter of interpretation; such messages as this, this, and this spell out exactly what's going on. You have talk of getting journal editors fired:

This is truly awful. GRL has gone downhill rapidly in recent years.
I think the decline began before Saiers. I have had some unhelpful dealings with him recently with regard to a paper Sarah and I have on glaciers -- it was well received by the referees, and so is in the publication pipeline. However, I got the impression that Saiers was trying to keep it from being published.

Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted. Even this would be difficult.

And of trying to get papers blocked from being referenced:

I can't see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow - even if we have to redefine what the peer-review literature is !

Two questions arise: is this defensible, and does such behavior take place in other scientific disciplines? Personally, I find this sort of thing repugnant. Readers of this site will know that I tend to err on the side of "Publish and be damned", preferring to let the scientific literature sort itself out as ideas are evaluated and experiments are reproduced. I support the idea of peer review, and I don't think that every single crazy idea should be thrown out to waste everyone's time. But I set the "crazy idea" barrier pretty low, myself, remembering that a lot of really big ideas have seemed crazy at first. If a proposal has some connection with reality, and can be tested, I say put it out there, and the more important the consequences, the lower the barrier should be. (The flip side, of course, is that when some oddball idea has been tried and found wanting, its proponents should go away, to return only when they have something sturdier. That part definitely doesn't work as well as it should.)

So this "I won't send my work to a journal that publishes papers that disagree with me" business is, in my view, wrong. The East Anglia people went even farther, though, working to get journal editors and editorial boards changed so that they would be more to their liking, and I think that that's even more wrong. But does this sort of thing go on elsewhere?

It wouldn't surprise me. I hate to say that, and I have to add up front that I've never witnessed anything like this personally, but it still wouldn't surprise me. Scientists often have very easily inflamed egos, and divide into warring camps all too easily. But while it may have happened somewhere else, that does not make it normal (and especially not desirable) scientific behavior. This is not a standard technique by which our sausage is made over here.

What I've seen in organic chemistry are various attempts to steer papers to particular reviewers (or evade other ones). And I've seen people fire off angry letters to journal editors about why some particular paper was published (and why the letter writer's manuscript in response had not been accepted in turn, likely as not). The biggest brawl of them all was still going early in my career (having started before I was born): the fight over the nonclassical norbornyl cation, the very mention of which is still enough to make some older chemists put their hands over their ears and start to hum loudly. That one involved (among many others) two future Nobel Prize winners (H. C. Brown and George Olah), and got very heated indeed - but I still don't recall either one of them trying to get journal editors fired after publishing rival manuscripts. You don't do that sort of thing.

And that brings up an additional problem with all this journal curating: the CRU people have replied to their critics in the past by saying that more of their own studies have been published in the peer-reviewed literature. This is disingenuous when you're working at the same time to shape the peer-reviewed literature into what you think it should look like.

A third issue I want to comment on are the problems with the data and its analysis. I have deep sympathy for the fellow who tried to reconcile the various poorly documented and conflicting data sets and buggy, unannotated code that the CRU has apparently depended on. And I can easily see how this happens. I've been on long-running projects, especially some years ago, where people start to lose track of which numbers came from where (and when), where the underlying raw data are stored, and the history of various assumptions and corrections that were made along the way. That much is normal human behavior. But this goes beyond that.

Those of us who work in the drug industry know that we have to keep track of such things, because we're making decisions that could eventually run into the tens and hundreds of millions of dollars of our own money. And eventually we're going to be reviewed by regulatory agencies that are not staffed with our friends, and who are perfectly capable of telling us that they don't like our numbers and want us to go spend another couple of years (and another fifty or hundred million dollars) generating better ones for them. The regulatory-level lab and manufacturing protocols (GLP and GMP) generate a blizzard of paperwork for just these reasons.

But the stakes for climate research are even higher. The economic decisions involved make drug research programs look like roundoff errors. The data involved have to be very damned good and convincing, given the potential impact on the world economy, through both the possible effects of global warming itself and the effects of trying to ameliorate it. Looking inside the CRU does not make me confident that their data come anywhere close to that standard:

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!

I do not want the future of the world economy riding on this. And what's more, it appears that the CRU no longer has much of their original raw data. It appears to have been tossed over twenty years ago. What we have left, as far as I can see, is a large data set of partially unknown origin, which has been adjusted by various people over the years in undocumented ways. If this is not the case, I would very much like the CRU to explain why not, and in great detail. And I do not wish to hear from people who wish to pretend that everything's just fine.

The commentator closest to my views is Clive Crook at The Atlantic, whose dismay at all this is unhidden. I'm not hiding mine, either. No matter what you think about climate change, if you respect the scientific endeavor, this is very bad news. Respect has to be earned. And it can be lost.

Comments (171) + TrackBacks (0) | Category: Current Events | General Scientific News | The Dark Side | The Scientific Literature


1. Andrew30 on December 1, 2009 9:30 AM writes...

It is understandable that many people have latched on to the emails, but in their defense the people at CRU indicate that the emails are ‘without context’ or somehow ‘normal banter’ in a scientific institution.

The program code however is different.

It is the actual program code, the modeling code that contains the most damaging evidence. I am not talking about the 'comments' in the code but rather the actual computer program source code itself.

Unlike comments and emails the computer code can only be interpreted in one way. Unlike the comments and the emails the computer code is whole unto it self and requires no external context.

So now everyone has the code.

However now CRU have somehow ‘lost’ the world’s raw climate data that they used in their modeling.

It may have been necessary for them to have lost the raw temperature data. If the raw temperature data was available then they might be asked to reproduce Exactly The Same Results, in front of skeptical witnesses, as they had used in their peer-reviewed publications that were distributed to the world. This might have been impossible without using some infected modeling code, which an investigating scientist might discover.

If the results can not be reproduced the paper that used the results should be withdrawn. Then every paper that cited that paper, and so on until the whole web of pseudo-science that can be traced back to the original fabrication has been purged from the libraries

It is not scientific unless an independent body can reproduce the results.

For information on the possible code infection see:

Anthropogenic Global Warming Virus Alert.

Permalink to Comment

2. wei on December 1, 2009 9:37 AM writes...

they should never put it down in writing

Permalink to Comment

3. Evorich on December 1, 2009 9:51 AM writes...

The climate change issue has become the world's most powerful religeon. Science has nothing to do with it anymore and hasn't done for quite some time.

Permalink to Comment

4. Edward on December 1, 2009 9:53 AM writes...

If raw data were destroyed by a scientist in any other field there would be calls for their head to roll, funding be cut, etc. but as long as you are "saving the world" apparently the rules do not apply to you.

Permalink to Comment

5. RB Woodweird on December 1, 2009 10:07 AM writes...

I really haven't been following this story, mostly because I used my psychic ability to see where it was going. No reason to pay attention. It was going to be a big long "oh noes, some scientists are cheating douchebags, therefore there is no global warming", this mostly rolling off the drums of shills working for industries guarding their bottom lines.

How much of the poor state of the data - missing, incomplete - can be traced back to refusal to fund such work in the first place, stonewalled by the usual crowd?

If our lunchroom banter and hallway small talk was collected and leaked, how big of a brick would the FDA, the EPA, and maybe the ATF pass?

This data that was lost. Was it all the climate data ever collected? Or was it just some kept by this University of East Anglia place.

And where the hell is the University of East Anglia anyway?

Permalink to Comment

6. alig on December 1, 2009 10:14 AM writes...

Unfortunately, the Univ. Of E Anglia CRU data was considered to be the best data. Now it has come to light that the raw data has been missing for 20 years? WTF.

Permalink to Comment

7. Derek Lowe on December 1, 2009 10:21 AM writes...

RB, as I mentioned in the post, I'm not going to get into the is-global-warming-real aspect of things at all. Even the firmest believer should be shaken up by the CRU story (as witness George Monbiot).

That's because the data from this center is a key piece of the UN's IPCC recommendations. You may not have heard of the University of East Anglia, but its data set is actually very important indeed.

And I haven't even linked to the "lunchtime banter" stuff, because I agree that it's trivial. What's left, though, isn't trivial at all.

Permalink to Comment

8. Jonathan on December 1, 2009 10:22 AM writes...

Your points might have more validity if the CRU was the only place in the world doing climate research or the only place in the world that's showing rising temperatures etc. Yet everyone else is seeing the same trends and coming to the same conclusions. Sure, the tone in the emails is impolitic but I'd bet you'd find the same if you hacked into the accounts of people working in, say, the autism field discussing the lunatics who believe that vaccinations are the cause. Just a guess, but I imagine you have less sympathy for the vaccine-autism movement getting their papers published and you'd be more charitable if these leaked emails were discussing getting a crank off an editorial board who thought vaccines caused autism.

And that's without bringing up the PNAS scandal.

I would refer you to John Timmer's rather excellent post on the matter over at Ars Technica:

One of the problems caused by the e-mails is that the scientists involved aren't discussing their data and its analysis using scientific terminology; instead, things come across more as what you might hear in an office environment. In short, the scientists sound like regular human beings (more on that below). When faced with two different data sets that provide different answers, the e-mails don't phrase things in terms of "what scientifically valid adjustment can be made to bring these two data sets into agreement?" Instead, the authors consider the problem in terms of how they can make the discrepancy go away.

Similarly, it's apparently widely recognized that, although tree ring data nicely tracks the temperature record for roughly a century, it diverges after 1960, when the modern rise in temperatures started. So, in a variety of papers, researchers have presented the instrument record, either superimposed or instead of the tree ring data, for periods where it's available (and clearly labelled the graphs accordingly). In the e-mails, this is described as a "trick" to "hide" the problem.

All of this is more pronounced when the data is preliminary, and researchers may not yet know how to interpret it or fit it into the larger body of existing data. That will get smoothed out by the time the data eventually gets published, but preparing data for publication is generally a small portion of an entire research project, and the e-mails largely reflect the longer period when confusion and frustration dominate.

As a result, the e-mails sound awful. But, the unfortunate truth is that this is the way scientists talk. "Lab-speak is full of shortcuts," said physics researcher and Ars contributor Chris Lee. "The way I discuss things internally is not the same way as I present them to the rest of the scientific community." And my experience from biology is that if I heard a coworker mention they had a trick to get better data from mouse embryos, I'd assume they were talking about a microscopy technique, not scientific fraud.

Permalink to Comment

9. Wavefunction on December 1, 2009 10:24 AM writes...

Here's my take on this. The climate change scientists are in a real dilemma. The climate is a complex system and it won't be surprising if they keep on finding major and minor errors and disagreements in their work. In fact that's what you would expect when you are scientifically studying such a complex system. Nothing wrong with that at all.

But the real problem is that even if they find these problems, they are going to fear going public with them because of the political climate. There is no dearth of right-wing extremists who are just poised to pounce on any such discrepancies and declare the entire enterprise futile. There is also no dearth of left-wing extremists who will disown the scientists and declare them to be paid corporate shills. All this is forcing the scientists to stay silent in my opinion even when they know better.

Of course that's bad science, since science only proceeds by way of disagreements and by making all of your data public. But the economic stakes with climate change are so high that good politics is often trumping good science.

Call it whatever else you will, but science is the real casualty here. And as scientists we are part of the casualty too. That's what's sad.

Permalink to Comment

10. MDW on December 1, 2009 10:30 AM writes...

One thing often overlooked regarding this mess is the selection effect involved in driving the makeup of the entire AGW field. That is, the very idea of 'climate science' is relatively new. There are of course communities of scientists who study weather, various physical aspects of the earth's history, etc., but specifically studying the history of the Earth's temperature and weather patterns is a recent phenomenon. The field of climatology itself has been rather politicized from the start ('global cooling' decades ago, the ozone hole) and has been associated with the green movement from early on. So, what sort of young person would tend to go into this field? If you are thinking about studying something in this general area, wouldn't your politics and general beliefs influence you to move toward or away from climate studies? As the field became more high-profile and overtly political, wouldn't that effect deepen? Over time, what sort of person would you expect the field to be comprised of? Do you think it would be pretty heterogeneous, or not? Ask yourself a similar question: what sort of person becomes a poststructuralist?

Ah, you say. But these people are scientists. That will cleanse the field of misapprehensions. This thought misses the point. Scientists are people, and all the bad aspects of people go along with that. It is only the scientific method and specifically the rigor of repeated and repeatable experimentation that allows us to muddle in a direction we can safely refer to as ‘forward.’ Without that, you wind up with something like particle physics, which without the benefit of experimentation has for decades now wandered off into a playland of mathematics where symmetry, beauty, and indefinable feelings of correctness are valued, and some of the very smartest people in the world have been reduced to having a quasi-religious stance toward their, and their fellow scientists’ work. Look at economics, where the arcane of mathematical models has led the field to a point where the field as a whole can agree on very few propositions, and where ideas such as MMT, which is a very simple collection of obvious statements, can be summarily ignored. Why? How? Because it is hard to check theory with experiment, and given the choice, people will continue to believe what they want to believe. People with careers spent defending Propositions X and Y will naturally have a hard time coming around to the view that actually, Propositions A and B seem to have much more evidence behind them. In fact, they will tend to fight that change vigorously. There is a constant tension between what people want and need to believe and what people are forced to accept. It is so, so much easier to be wrong than to be (tentatively) right. Given the chance, people will almost always lead themselves astray. It is just too difficult to understand something new. A community of people can be persuaded of just about anything, as history has shown – need we list some examples? Without being forced – I think that is an appropriate word – to constantly check our ideas vs what we charmingly call ‘reality,’ any group of ‘scientists’ will eventually become nothing more than a group of people with the same hobby.

And so we come to climate science. You have a fairly small group of people strongly predisposed to believe in a particular idea; a group most of whom are invested from the start in such an idea. You have an external environment which copiously rewards you for holding such ideas. You have limited data, much of which has proved to be of questionable enough quality that even the people in charge constantly employ fudge factors for it. You rely on models which are clearly incapable of robust predictions, and which again can and must be constantly tweaked with ‘improved’ assumptions. You have, in short, a giant socially constructed epicycle-building machine.

What do you think will happen?

Is it somehow wrong to argue in this way? Perhaps we should let the data decide. Evidencing reasons are far, far superior to motivating reasons. I agree. But much of the evidence looks far from evidentiary, much of it we can’t look at at all, and the rest we have to take people’s word for, and/or assume some model is correct.

Ultimately, evidence will decide things. But we have very, very good reason to think that evidence may not be what has decided things to this point. And it is entirely reasonable to argue THAT point based on motivating reasons. Because science is hard, and people are weak.

Permalink to Comment

11. Anonymous on December 1, 2009 10:31 AM writes...

book smart vs. common sense

Permalink to Comment

12. road on December 1, 2009 10:44 AM writes...

In my own field, I can read someone else's paper and decide if their assumptions and data-manipulations are bullshit or not. I can't do that in other fields because I'm not intimately familiar with the experimental techniques. I think most people probably feel the same -- that it's hard to be objective and critical of research in other fields. I have NO IDEA if the climatology data really support the claims of global warming. And frankly, neither does anyone else, unless they've spent a lifetime studying it. HOWEVER, for some reason, just about everyone who's spent a lifetime studying climatology data appears to come to similar conclusions and to me that's very convincing. You can have all the data-scandals you want, but I'm betting that there's a reason that climate-scientists believe in anthropogenic climate change.

It's the same issue with the evolution-deniers: Anyone that spends their life looking at DNA sequences and orthologies couldn't even imagine an alternative to the theory that evolution proceeds randomly through mutation. However, the public, which doesn't look at data, can argue about it for decades. On that issue, it's clear to me that familiarity with the raw data engenders obvious conclusions and I assume climate-change is the same.

And also, Derek, it's really not fair to claim that the nonbornyl cation controversy was even remotely similar to something like this. I'm not defending these particular scientists, but a lifetime of having CNN report-on and and mis-interpret every single paper you published would probably push even the most level-headed scientists to focus on persuasive data presentation.

Permalink to Comment

13. mrr on December 1, 2009 10:46 AM writes...

Thank you Derek for this, your finest post.

Permalink to Comment

14. Hap on December 1, 2009 10:54 AM writes...

I can see reasons for not wanting to publish papers with questionable hypotheses but some data - in politically charged fields, the presence of a paper (regardless of its quality) is likely to be taken as evidence of the correctness of its underlying theory, and used as such among people who are unlikely to know its context and evaluate its quality. The alternative (excluding papers with theories that disagree with the dominant one) seems like a much worse idea.

UEA seems to be behaving in precisely the way that people who disagree with AGW have been claiming. Since lots of people can't evaluate the evidence for/against AGW (and lots of those are the people determining what to do in response to it), trust and integrity are really their main assets. If you lose those, you don't have anything - people can't trust your data, and they can't trust what you generate from either your data or that of others. It gives your opponents a shining example of behavior to use against anyone who thinks as you do. As such, it seems patently stupid.

Science is supposed to be a way of thinking and testing our thoughts that makes it hard to fool ourselves (as others have put it), and marrying our theories to achieve the political ends we believe are needed because of them makes it harder for us to test our theories and ourselves. In politics, certainty is valued, often over truth, but that's a dangerous value order for science (and for politics, as well), and the people at UEA ought to have understood that. (Voters ought to understand that, at some point, but uncertainty isn't easy for most people to handle.)

Permalink to Comment

15. CB on December 1, 2009 11:03 AM writes...

It seems clear that these idiots should be run out of town, just based on the email exchange. I agree with Edward, heads should roll.

In the long run the best scientific models will win out!

Unfortunately, the climate skeptics arguments are very similar to creationists, or spritualists arguments against the various aspects of science they take a dislike. They have no testable models, they make no predictions and they generate no empirical data.
I have seen no serious efforts to generate more predictive climate models that are not reliant on greenhouse gases.

This is surprising since 1) copious research funding from the fossil fuel industry would certainly be available 2) the surest path to fame and fortune in science is in slaying myths. 3) predictive climate models provide extremely valuable information that can be used by agriculture, finance, insurance, govt. etc.

Permalink to Comment

16. BACE on December 1, 2009 11:07 AM writes...

I know this post is not about global warming per se but I would recommend an objective and well-written book that describes the history of the science that led to the global warming consensus- Spencer Weart's "The Discovery of Global Warming"

Permalink to Comment

17. rob on December 1, 2009 11:10 AM writes...

Derek says the results of climate science have to be very good and very convincing if we're going to believe them.

I agree.

So where are the safety data showing that pumping gigatons of CO2 into the air is an ok thing to do? Where are the coal companies' data and models? The oil companies'?

And can we please plow through the last N years of internal emails from these companies as well? Or those from the lobbies they fund?

Why don't you hold polluting companies to a standard that even remotely approaches the one you hold climate scientists to? Or even the one that-- you say-- pharma companies routinely meet. After all, as you rightly say, the stakes are enormous.

How do you justify using different standards for different institutions?

Permalink to Comment

18. Anonymous on December 1, 2009 11:18 AM writes...

My question is how many other global temperature records are there? Basically to get at what percentage of climate scientists rely on the CRU global values wich now seem to be backed only by "lost" data. I realize that this data is considered "very important" but how much? If only 20% of the literature relied on this data then with the remaining 80% one can still pull together some analysis. But if say 70% or 80% of the literature studies relied on this data analysis of the collective body becomes much more difficult.

Permalink to Comment

19. Robert Bruce Thompson on December 1, 2009 11:19 AM writes...

At this point, I'm not even sure that "climate science" has made the jump from a pseudoscience to a protoscience, but the Hadley/CRU mess is a strong argument that it's still in the former category. Given that Hadley/CRU was one of the two major sources of historical climate data and that both of those sources have now contaminated each other to the point where none of their data are trustworthy, it seems that the question is no longer whether global warming is anthropogenic, but whether it is even occurring at all.

The real cost of this mess can't be calculated, because all scientists will be tarred with the same brush. Already, the creationists and IDiots are using the Hadley/CRU mess to attack evolution on the basis that scientists are proven liars.

Permalink to Comment

20. David P on December 1, 2009 11:24 AM writes...

I hate the fact that the way of doing science is undermined by this scandal.

I had (have still even) hopes that climate change would be the thing that got developed countries to be less wasteful, which seems to me to be a good thing regardless of the effect on the planet.

Oh, and since no one else answered, University of East Anglia is on the east coast of England, In Norwich. That is the flat sort of round bit on the right, above London.

Permalink to Comment

21. pi* on December 1, 2009 11:25 AM writes...

Is e-mailing your friends trying to figure out who is reviewing a Science paper OK?

Permalink to Comment

22. Derek Lowe on December 1, 2009 11:30 AM writes...

Rob, I tend to think that pumping excess CO2 into the air is, in fact, probably not a good idea. The question is, just how bad will the consequences be, how long will they take to develop, and what should we then do about them? All of these are very much open to argument, and the answers are very important. So if someone comes forward and strongly advocates a course of action based on their data and models, scrutiny is the only appropriate response.

Your question about the safety data from the oil and coal people is rhetorical, of course - for many years, no one even thought about such things, and the carbon dioxide emissions were the least of anyone's worries (compared to lead, sulfur and nitrogen oxides, etc.) No one ever ran environmental impact studies on the Industrial Revolution or its sequels.

I don't see this as a competition between one group that says that excess CO2 is a disaster and one group that says it's peachy.

Permalink to Comment

23. bearing on December 1, 2009 11:33 AM writes...

@Road 10:44:

"In my own field, I can read someone else's paper and decide if their assumptions and data-manipulations are bullshit or not. I can't do that in other fields because I'm not intimately familiar with the experimental techniques. I think most people probably feel the same -- that it's hard to be objective and critical of research in other fields. I have NO IDEA if the climatology data really support the claims of global warming. And frankly, neither does anyone else, unless they've spent a lifetime studying it. HOWEVER, for some reason, just about everyone who's spent a lifetime studying climatology data appears to come to similar conclusions and to me that's very convincing. You can have all the data-scandals you want, but I'm betting that there's a reason that climate-scientists believe in anthropogenic climate change."

Yes, I am where you are. There is a reason that climate scientists believe this. But what isn't clear to me is this: How *much* of that reason is directly or indirectly based on the integrity of the missing data sets? Because now the integrity of those data sets is un-testable: they are removed from the reach of scientific research. Perhaps a historian could make something of them. A scientist cannot.

Suppose we were to remove from consideration, because suspect, all the results of all the models that took as inputs the missing data set. And then, went forward and removed all the papers that relied heavily on the results of those models to bolster their arguments or as inputs for future models and calculations. And then kept moving forward and deleting all results that rest implicitly on the integrity of the specific data that we now know was destroyed and cannot be examined. What will be left then? Perhaps what's left will still be enough to convince the world's honest climatologists that the AGW situation is still so unimpeachably grave. But we do not know until we see how far the missing data set has propagated through the literature.

Permalink to Comment

24. gyges on December 1, 2009 11:45 AM writes...

Interesting links from the Activist teacher,

told you so,


Global Warming Truth or Dare. A critique from Feb 2007.

Permalink to Comment

25. Nick K on December 1, 2009 11:46 AM writes...

This represents an appalling betrayal of trust, especially in a domain where integrity and openness are so important. How can anyone take Jones' statements on climate seriously now he has revealed just how partisan he is?

Permalink to Comment

26. road on December 1, 2009 11:51 AM writes...


That's a good question. Again, all I know is my field. And in that field nobody believes anything that isn't supported by lots and lots of separate data that comes from all over. Each isolated piece of evidence, in and of itself, is not wholly convincing, but over years of seeing supporting data over and over again one becomes overwhelmingly convinced. I could be wrong, but I assume that these people (climatolagists) are not basing something so important on one dataset. The public seems to think that scientists do one experiment and then a thing is proved or disproved, and I presume that we as scientists all know that this is 99% preposterous, no?
Again, I have no idea how important this particular dataset is to their claims, but if the entire climatology community is basing a majority of their conclusions on a single dataset then it really is psuedoscience. And from what I've seen serious governmental agencies like NSF and DOE don't fund a lot of pseudoscience. I think it's probably a credible field of study, perhaps excepting the CRU lot.

I'm a biologist and the way I look at it is that there isn't a single lab or result that one could discredit and profoundly change our understanding of biology (excepting really recent stuff that people still argue over). I assume climatology is similar (and that's why I doubt that this CRU debacle really changes much, scientifically) but what do I know?

I totally agree. Why does everyone have to be 1000% convinced that something is bad before acting? If there was some compelling evidence that you were developing clogged arteries and might get a heart attack, you'd probably act without proof. Shouldn't the burden of proof be on those that want to continue polluting?

Permalink to Comment

27. Don'taskme on December 1, 2009 11:57 AM writes...

I can't wait to hear what C and E News Editor Rudy Baum is going to say about this.

He has been on a tirade about how anyone who questions the science of global warming is a knuckle-dragging moron, and can't be considered a scientist.

I'm sure he'd rather pass a cinder block than write about it. But in order to have any integrity, he will have to.

Permalink to Comment

28. Rob on December 1, 2009 12:11 PM writes...


Thanks for your response. Actually, my question about safety data was NOT rhetorical. Yes, its true that people once thought there was nothing wrong with dumping CO2 into the air, but why is that relevant for today?

Are you seriously arguing that "well, because we once had no idea that this was problematic, we'll therefore continue to assume-- or just pretend-- that it isn't?"

You write: "So if someone comes forward and strongly advocates a course of action based on their data and models, scrutiny is the only appropriate response."

Well, the oil and coal people are strongly advocating a course of action. They are proposing (and getting) massive subsidies for their products, and advocating that society continue to use their products. And the lives of billions of people around the world (not just their users) depends on their safety claims being correct.

Where is the safety data to back up their claims?

Where are the studies they have done?

Where were they submitted for peer review? How did the process work? Did the oil and coal companies ever attempt to influence the peer review process or the political process?

Again, these aren't rhetorical questions. I'm honestly quite curious how you justify refusing to hold those who advocate the use of fossil fuels to standards even remotely similar to those you hold climate scientists (or even pharmaceutical scientists) to.

Permalink to Comment

29. bearing on December 1, 2009 12:12 PM writes...

@road #25,

I could be wrong, but I assume that these people (climatolagists) are not basing something so important on one dataset.

Yes, that is the assumption that I would like to check now.

I also assume that they are not basing *everything* on one data set, but I assume that they are basing *some fraction* of their total, shall we say, "picture" on that data set. I want to know how big is that fraction, and when you take the data set and its sequelae away, how much does the uncertainty increase.

Permalink to Comment

30. Tom Myers on December 1, 2009 12:17 PM writes...

In reading the emails, I think you have to remember that they're being written by people who mostly believe that the "skeptics" are mostly or entirely non-skeptical; they think that they are scientists faced with an ideological and commercial opposition, not a scientific opposition. See, e.g., "The Manufactured Doubt industry and the hacked email controversy." They may be right, but the belief itself induces circling-the-wagons behavior, effectively forming counter-conspiracies of the sort you describe.

I don't think that most of the email fragments amount to much, in context, nor even the throwing away of their copies of other people's data, back when storage was a big deal. (Yes, it was bad, but it doesn't amount to much.) I'm not at all impressed by the "ARTIFICIAL CORRECTION" code comment. I do think that if the overall curve were fake, we would need to be looking for a really gigantic multi-country conspiracy with a cast of thousands; I don't find that plausible, and I do find the warming scenarios plausible in light of, e.g., ice loss. (I did make fun of climate models in the 80s, but not lately.) But I would personally be delighted by a rule that whenever any paper is published with Federal funding, the data and source code and commentaries must be made available in the form of a bootable disk image which can be downloaded and which can then run whatever modeling etc. software is required. Even "open source" is not good enough because open source code, e.g. my own, depends on specific versions of underlying software so it's often a major hassle to set things up even when you completely trust the software author. Self-contained bootable; that should do it.

Thoughts? :-)

Permalink to Comment

31. road on December 1, 2009 12:18 PM writes...


agreed. if you figure anything out, please post here.

Permalink to Comment

32. dearieme on December 1, 2009 12:24 PM writes...

"I can't do that in other fields because I'm not intimately familiar with the experimental techniques." There are no controlled experiments of the sort most of us are familiar with. There are mathematical models of climate, there are reported temperature observations (but few before 1880) with hard-to-estimate biasing because of Urban Heat Island effects, poor instrument location and so on, there are attempts to coax historical temperatures out of proxies (but rather a short run of temperatures to calibrate them against) and, for about 30 years or so, there are satellite measurements (but I gather that they are subjected to some calibration against the likes of CRU data). When you have people tweaking the models, fudging the proxies, and adjusting the observations in ways that they intend to keep secret, it adds up to a shoddy evidence base for a decision to spend trillions of dollars.

Permalink to Comment

33. Chris on December 1, 2009 12:30 PM writes...

Why do journals allow publications without the appropriate data being available as supplementary information?

Permalink to Comment

34. Straw Man on December 1, 2009 12:32 PM writes...

Rob, your logic is impeccable.

Permalink to Comment

35. milkshake on December 1, 2009 1:13 PM writes...

I have problem with salesmanship and policy agendas in science. Man-made global warming has became a huge international project. Now it seems the data points were massaged to fit the desirable curve to support the pre-determined conclusions.
Enviro-activists became victims of their own success (banning Freons, PCBs and chlorinated insecticides and commercial whaling) and there is a whole political pressure machinery and gravy train in predicting the impending doomsday. Neomalthusians predicted world-wide famine to start around 1975, and silent spring all about the same time. Banning DDT and opposing GMOs has been a great disservice to the third world.

One of the reason why Freeman Dyson is lukewarm is that as a JASON he got involved with simulations of consequences all-out thermonuclear war, and he was close to people who were running these models, and he knew they started back-tracking from the original "Nuclear winter" scenario more towards "nuclear dimming" when the better data did not support their initial catastrophic conclusions - and finally they decided to keep mum about the whole discrepancy.
They privately admitted that the climatology science behind the "nuclear winter" scenario was atrocious but they would not want to retract it - because they did not want to lend support to some generals starting a nuclear holocaust!

Permalink to Comment

36. bearing on December 1, 2009 1:18 PM writes...

There's nothing illogical in what Rob has suggested, it just betrays a difference in the relative emphasis on two different values (and I'm not suggesting with this comment that one is inherently better than the other). One is the value of the good we call "economic freedom" and the other is the good we call "public safety."

If you value economic freedom very highly -- and it is true that society derives benefits from economic freedom, benefits that will be diminished if laws and treaties restrict economic freedom -- you will require a high degree of certainty that a given economic behavior endangers the public before you will permit governments to act to restrict that behavior.

If you do not value economic freedom very highly compared to public safety, you will be willing to restrict behavior based on quite scant evidence that the behavior might endanger the public.

This is why two people could hold the two "sides" of the debate to different standards and still be logically consistent.

Permalink to Comment

37. oxsg on December 1, 2009 1:18 PM writes...

You use computer/internet, you sit in a warm cozy roo, eat a foof from refrigerator, etc.
ALL of this because we pump "gigatons of CO2 into the air". Do you want to stop your privileged lifestyle ? Fine, but don' expect many others to follow. Alternatives to "gigatons of CO2 into the air" ? Do you have pocket money to finance trillion dollar restructuring ? To commit to it, public must have a firm confidence in the perils of current way. You/we must prove it, and the burden of prove lies on descredited CRU scientists and their collegues. Gas/oil industry has nothing to prove, just ask you to swith on your computer.

Permalink to Comment

38. rhodium on December 1, 2009 1:29 PM writes...

Your take on point #1 is one I agree with (and you will always have a few people acting dumb). However, the other points can be refuted, I think. Basically, some researchers are tired of deniers misrepresenting their work and in some cases gaming the system. But arguments over these emails can be seen throughout the web.
What we should recognize is that chemistry has already gone through a similar controversy, with a different outcome: ozone depletion. DuPont and the other chemical companies could have dug in their heels, hired lobbyists and PR firms and done everything to block the Montreal Protocol. Arguments that the models were flawed, that measurements only went back a few years so that the observed changes were within natural limits, that there were no economical freon substitutes, that developing countries would cheat, that the scientists were socialists (many from the People's Republic of Massachusetts) and that less ozone is good for you anyway (after all it destroys rubber), etc. could have delayed action for decades. It seems obvious that had the ozone problem been discovered in today's political climate, it would be facing the same attacks by the same groups climate change is facing. Facts are irrelevant, otherwise everyone would accept evolution.

Permalink to Comment

39. rob on December 1, 2009 1:48 PM writes...

oxsg: It seems we agree that there are different standards of proof required for claims that suggest we can continue our privileged lifestyle, and claims that suggest we can not.

Bearing makes much the same point.

I can't stop you from having such a double standard, but you (and Derek) should admit this explicitly up front.

And the next time you file an IND with the FDA, add an introduction saying something like "I bet my career (or company) on this compound working. If it fails, this will be awful for my lifestyle. That's why we didn't bother to get any rat PK data."

And should you ever talk to an Indian farmer whose ability to grow food depends on rapidly diminishing glacial runoff, or a citizen of Bangladesh who farms in a delta of what is soon to be brackish water due to rising seas, just tell them this:

"We don't know whether our current lifestyle will destroy your ability to feed your family. We don't care enough to find out. We also don't know whether we can change our lifestyle in ways that will keep it enjoyable without harming you. Again, we've never bothered to find out.

"But as a scientist, I will promise you this: I will spend countless hours scrutinizing any claim that threatens my lifestyle, and hold to the highest possible standards anyone whose views potentially threaten my lifestyle or make me feel uncomfortable.

"Good luck with the whole starvation thing."

Permalink to Comment

40. Peej on December 1, 2009 2:03 PM writes...


If you look at the tone of some of the emails, its clear that the frustration is due to the politicization of this topic.

You dont have the same aspect of this in the med chem world, but imagine if there was a really, really well funded homeopathy concern who was trying to publish all kinds of data about water memory in Chemistry journals, and after dismissing it and proving it was false, they keot persisting and actually got some things published...

I think thats what the East Anglia people were up against.

Permalink to Comment

41. milkshake on December 1, 2009 2:09 PM writes...

#37: The Poor Indian farmer whose ability to grow food depends on rapidly-diminishing glacial runoff...

From the most recent study: The best available data do not support the alarmist statements about melting Himalayan glacier.

Geologists R K Ganjoo and M N Koul of Jammu University's Regional Centre for Field Operations and Research of Himalayan Glaciology visited the Siachen glacier to record changes in its snout last summer.
"The field studies from other glaciers in India also corroborate the fact that inter and intra-annual variations in weather parameters have more impact on the glaciers of northwest Himalayas, rather than any impact due to global warming," they said.

"To our surprise, the Siachen glacier valley does not preserve evidences of glaciation older than mid-Holocene, suggesting that the glacier must have advanced and retreated simultaneously several times in the geological past, resulting in complete obliteration and modification of older evidences," they said reporting their findings in 'Current Science'.

Ganjoo and Koul dubbed as "hype" some earlier studies which suggested that the Himalayan glaciers were melting fast and caused serious damage to the Himalayan ecosystem.

Permalink to Comment

42. startup on December 1, 2009 2:17 PM writes...

Derek, you are wrong in approaching this issue as scientific, it is political through and through. And frankly, I think that from practical point of view this scandal does not matter at all. You either believe in global warming or you don't, and nothing can change that.
But, I would love for this to have happened here, just to see how ORI handles it.

Permalink to Comment

43. Zippy on December 1, 2009 2:32 PM writes...

Appalling and sad. An example of what can happen when a small branch of science encounters politics, money and attention. #30 is on target concerning the problems with the “science” of climate change raised by these emails.

A distinction can be made between lowering the bar to allow publication of unorthodox ideas and permitting poor science. Avoiding rigorous evaluation of unorthodox (or accepted) theories is unquestionably poor science. The contents of some of these emails seem to imply a lack of enthusiasm to pressure test the models.

Having spent a few years building and testing complex models, I have felt uneasy with the confidence expressed in model projections by climate change debaters. One lesson from study of nonlinear systems is that future behavior can only be predicted for a limited time, even with vast amounts of nearly perfect data supplied to an exact model, none of which are true in climate models. The recent revelations about data substitutions and normalizations amplify these concerns.

Permalink to Comment

44. rob on December 1, 2009 2:38 PM writes...

Milkshake: Your claim about the glacier is a classic example of the double-standard.

You provide no link for your claim. When I googled it, I found it was published in Current Science. Current Science appears to be a third (fourth?) tier journal, which may or may not be peer reviewed (I can't tell). And yet, somehow this is the "latest science."

There is, however, overwhelming evidence that we are melting glaciers. Around 95% of glaciers worldwide are retreating (so, yes, you can find counterexamples), and their rate of retreat appears to be accelerating. I don't have time to provide links for you, but they're really not that hard to find.

But, apparently, a single paper about a single glacier in a 3rd tier journal outweighs the hundreds of papers in much more respected journals. That's a classic double standard.

As for "the latest science", IIRC the current issue (it may have been last week's) of Science shows a graph with a nice hockey stick curve of how Greenland melting has skyrocketed within the last few years. So, actually, the current science says we are melting glaciers all over the world.

I'm sorry things have worked out this way, since previous posters are quite correct that I rather do like my current lifestyle. But Nature doesn't care about what I like.

Permalink to Comment

45. Person of Choler on December 1, 2009 2:52 PM writes...

road: "HOWEVER, for some reason, just about everyone who's spent a lifetime studying climatology data appears to come to similar conclusions and to me that's very convincing. You can have all the data-scandals you want, but I'm betting that there's a reason that climate-scientists believe in anthropogenic climate change."

The Roman Curia have spent their lifetimes studying religion, but I don't follow their recommendations for conducting my life because I find the evidence for their beliefs unpersuasive. Same for AGW.

Permalink to Comment

46. EconRob on December 1, 2009 2:54 PM writes...

Two observations: New PhDs spend most of their brain power trying to discover new things. Old PhDs put their energy into defending previous work - right or wrong.

It seems these AGW "scientists" had a hunch that MM CO2 emissions could cause the climate change. They reduce this hunch to models and gathered data. However backtesting the models and data against observations did not pan out, so they started to tinker with both. Eventually they got a successful backtest that when run forwards confirmed the theory. However, similar to most computer driven trading (remember LTCM) the results do not track the forecasts. At this point you either throw out the model, admit the errors and ask for help from your critics, or go into attack mode.

The AGW types now need tuck in their tails and reach out to the biggest skeptics (big oil or no big oil).

Permalink to Comment

47. Link on December 1, 2009 2:58 PM writes...

"Lost data" sounds sexy, but it's missing the elephant in the room.

Everything Michael Mann & Co have done is based on their 1,000 year temperature data bank. If this data is bad, their science is bad. It's that simple. This data is mostly based on tree rings.

By 1998, many of these guys had invested years and even decades of work based on tree ring data. It's in 1998 that the "divergence problem" was first recognized: Starting in 1960, tree ring data "diverted" from actual instrument readings. That means thermometers gave one reading, tree rings a different reading.

This is actual scientific proof that tree ring data is suspect. If tree rings don't work for the last 40 years, why do we think they work for the last 1,000 years. If tree ring-based data is suspect, everything that Mann & Co have done is suspect. QED There are other issues with their methodologies, but this is fundamental. This is the elephant in the room.

They say that the "trick" to "hide the decline" was over how they spliced the old tree ring data with the new data from instruments. So they're not denying the elephant in the room. They just hope we don't see it.

Permalink to Comment

48. Straw Man on December 1, 2009 2:58 PM writes...


That's the beauty of this particular logical fallacy. You don't have to defend the actions of the CRU, since now we're talking about melting glaciers, the Himalayas, and holding Oil Companies to a scientific burden of proof. Well done!

Permalink to Comment

49. pi* on December 1, 2009 3:00 PM writes...

re comment to Milkshake.

If there is one point this post should raise it is the possibility that perfectly valid science can't get published somewhere better than 'current science' because a select few people will keep everything that disagrees with them out of the literature.

did you happen to notice who were the authors of last weeks 'Science' paper were?

Permalink to Comment

50. Biker Trash on December 1, 2009 3:01 PM writes...

Consider the outrage if these e-mails had been found in the offices of those who certify, (1) the accuracy of the delivery of gasoline pumps, (2) the safety of elevators and escalators, (3) the composition and proper installations of building materials, (4) the airworthiness of commercial aircraft, (4) the safety of medicines and medical devices, (5) the safety of power plants, (6) the crash-worthiness of automobiles, (7) that Environmental Impact Statements are correct and true, and on and on and on.

There would be no rationalizations, none whatsoever, presented to insist that the e-mails simply reflect Business as Usual. Independent investigations would be started immediately.

Permalink to Comment

51. bad wolf on December 1, 2009 3:02 PM writes...

Rob is a great example of why answering questions on the internet is pointless. And also how to have the best of both worlds--enjoy a priviledged lifestyle but offset it with lots of complaints and hand-wringing.

Permalink to Comment

52. K T Cat on December 1, 2009 3:07 PM writes...

As a mathematician (or a recovering one, at least), the whole thing spooks me. If these guys were faking stuff and turning the peer review process into a KGB firing squad operation, then who knows what else has been going on.

There was an earlier comment that bemoaned the lack of funding for research and using that as an excuse for the loss of data. You've got to be kidding. These guys have gorged themselves on S&T funding, sometimes at the expense of other fields of research.

It seems to me that when the data started going the wrong way, they decided to cheat, either to keep the flow of cash coming in or to pursue what equates to a religious crusade. If the latter is true, then East Anglia is closer to a cult than a university.

As for global warming, it's a problem of time scales. 30 years of data in a 4 billion year planetary history isn't much more than noise. If core samples and tree rings are any evidence, it's been hotter before in the absence of man-made conditions. That should give anyone pause before imposing what amounts to fascist economics* upon the rest of the world.

* - Fascism is where you get to keep your private property, but only so long as you do what the government tells you to do with it.

Permalink to Comment

53. A.W. on December 1, 2009 3:07 PM writes...

For me, the other disturbing thing is this: the sheer banality of it.

They talk about screwing with data, destroying data, blackballing "deniers" like it is nothing, like it has no more significance than ordering a cup of coffee. It says to me that they didn't even find this to be at all weird. And that suggests that they were not an outlier, but part of a larger culture that would be accepting of it.

although given the allegations that the email theft was an inside job, maybe not all of their culture accepts it.

So contra Biker Trash comments, yes, this seems like business as usual among climate change scientists, which is not a defense but an indictment.

Permalink to Comment

54. CosmicConservative on December 1, 2009 3:09 PM writes...

As many of these comments demonstrate, AGW is a cult at worst and a religion at best. The lack of outrage and demands for prosecution of the offenders and recreation of their results is deafening. People are so wholly invested in their belief that human beings are destroying the planet that even ironclad proof of corruption, conclusion bias, felonious intent and manipulation of data and computer code doesn't even budge them an inch from their position on the subject.

This is an excellent post. Whether global warming is happening or not, whether human beings contribute greatly, slightly or not at all, the behavior of these people is an egregious affront to all that is meant by the word "science." Any person who does not immediately realize that is not worth listening to on any subject whatsoever, much less a subject that is determining the disposition of TRILLIONS of dollars and is making decisions about what sort of LIGHT BULB I can put in my own bedroom lamps.

There truly are DENIALISTS in the AGW debate, but the ones in denial are the ones in SUPPORT of AGW, not the skeptics.

Permalink to Comment

55. silvermine on December 1, 2009 3:11 PM writes...

When I was in college, I worked in an analytical lab that did contract work for drug companies.

If we did half of what those CRU researchers did, the feds would have thrown us in jail.

Permalink to Comment

56. RB Woodweird on December 1, 2009 3:13 PM writes...

EconRob sez: "Two observations: New PhDs spend most of their brain power trying to discover new things. Old PhDs put their energy into defending previous work - right or wrong."

Your power of observation needs new batteries. I suggest changing them twice a year. Try doing it when you reset all the clocks in your house.

Maybe, just maybe in academia, and maybe at third-tier junior colleges. But in industry, new PhDs spend a lot of brain power learning where the restroom is. The older PhDs are busy using their hard-won knowledge and experience to solve problems.

Permalink to Comment

57. ThomasD on December 1, 2009 3:14 PM writes...

Derek's article is a critique on the shoddy CRU effort being passed off as 'science.'

Rob, for all his earnestness, blithely displays that for some this was never a matter of, nor concern for science, but instead has always been a high stakes political endeavor utilizing science as a rhetorical fig leaf.

Permalink to Comment

58. John Skookum on December 1, 2009 3:16 PM writes...

@rob: "As for "the latest science", IIRC the current issue (it may have been last week's) of Science shows a graph with a nice hockey stick curve of how Greenland melting has skyrocketed within the last few years. So, actually, the current science says we are melting glaciers all over the world."

WE? Who's WE, kemo sabe? Got a mouse in your pocket?

The presence or absence of global warming is a separate question from whether such warming (if it exists) is man-made. Glaciers will melt whether warming is due to anthropogenic CO2 or sunspots or orbital oscillations. Indeed, twenty or thirty thousand years ago there may have been ice a mile deep over where you now sit.

Glaciers melt and form all the time, but the burden of proof is on the alarmists to show that THIS time it's our fault. I found the evidence questionable even before the CRU document release.

Permalink to Comment

59. milkshake on December 1, 2009 3:16 PM writes...

The reason why I did not provide a link