Climate Audit

Raven

Scenario 1: Papers are critiqued by doing a different analysis and claim it is better.

Alarmist Response: Insist that the different analysis is not correct for some reason.

Scenario 2: Exactly replicate the analysis and demostrate why it is flawed.

Alarmist Response: Refuse to provide the information required replicate the analysis.

One can get very dizzy trying to keep up….

jae

I know I’m not allowed to vent, but &*(%@(##*!

Craig Loehle

Santer is annoyed that you “appointed” yourself a climate auditor. He has forgotten that anyone in the world is allowed to comment on or check the work of any scientist. And I mean anyone–I have never had a journal check if I have a Ph.D. nor is it required. It is not a closed club, as much as these people would like it to be.

• PaddikJ
  Posted Dec 24, 2008 at 1:39 AM | Permalink

  Re: Craig Loehle (#7):”It is not a closed club, as much as these people would like it to be.”

  I believe you have put your finger on it: Climatology meets one of the sociological definitions of Ghetto – an insular ethnic enclave which is resistant, and at times hostile, to outside ideas or influence.

  Re: Brian (#19):”Considering the tone and maturity of this comment thread … I’m neither defending nor criticizing anybody for any specific actions/inactions … just an observation … ”

  Of course you’re not; that would require some, um, specific examples, and the only ones I’ve seen are in Santer’s reply. Steve’s characterization of it as discourteous was charitable; juvenilistically petulant would have been closer to the mark.

  ———————

  Thank you for your mail. I have asked the publishers to respond to me regarding your question about data policy. Hopefully I should have a response over the next few days.

  The Editor of a major refereed journal doesn’t know the data disclosure policies of his journal? That inspires no end of confidence.

  ———————-

  Re: Eric (#77): “. . . properly documenting my procedures.” Isn’t that exactly what Steve’s asking for?

Steve McIntyre

There’s a certain irony here – Santer et al 2008 was itself an “audit” of Douglass et al 2007. Did any third parties “appoint” Santer to carry out this “audit”?

And for the record, I haven’t granted myself any grandiose titles or appointments.

• Craig Loehle
  Posted Nov 10, 2008 at 4:41 PM | Permalink

  Re: Steve McIntyre (#8), It is further ironic that the long long discussions of both the original Douglass paper and the new Santer paper covered many points of view stats-wise and showed many analyses and simulations. I would not even say that it was agreed on CA that Douglass et al did it right.

BarryW

You seem to have the same reputation in the Climate Science world as an “Internal Affairs” division has in a police department!

kuhnkat

There is so much irony in climate science the corrosion will eventually cause a fire and melt it!!

Sorry Rosie, you are sooo wrong also.

Urederra

Naive newbie question: Is this sentence true?

I note that both the gridded model and observational datasets used in our IJoC paper are freely available to researchers.

Or is there a thread I fail to see?

• Ignatus
  Posted Nov 10, 2008 at 11:23 PM | Permalink

  Re: Urederra (#13),

  Yes, all the data are already available, therefore I don’t see where is the problem: you just have to download the files, and compute some averages etc. Nothing too difficult.

Roger Pielke Jr.

Some comments on this:

http://sciencepolicy.colorado.edu/prometheus/im-a-mac-and-im-a-pc-climate-science-edition-4711

henry

So how did you wind up on Santer’s “naughty list’ this year?

I’ll bet you won’t even get a lump of coal in your stocking, either…

• Craig Loehle
  Posted Nov 10, 2008 at 4:22 PM | Permalink

  Re: henry (#16), Not even coal, it emits greenhouse gases….

Ross McKitrick

Looks like “egregious” is the new favourite team adjective. All our methods are rigorous and conservative, all your errors are egregious.

Steve McIntyre

#22. I drew no such conclusion. However, I replicated Santer’s Table III and showed that this calculation used the standard error of the mean of the models, notwithstanding various fulminations by beaker and Gavin Schmidt against this procedure. I also was able to show that Santer’s Table III test fails for the H2 hypothesis when 2006, 2007 or 2008 data is included – a result that Santer either omitted to calculate or failed to report.

Ross McKitrick

#19: That’s your excuse? That the mere anticipation of criticism for concealing one’s data fully justifies concealing one’s data? Hmmm, let’s apply that excuse more broadly. Your honor, I knew the busybody police officer was going to hassle me for driving drunk, so it’s no wonder I got plastered before driving home.

This thread is a response to Santer’s decision, not vice versa. In any case, it matters not a whit whether the comments on a blog seem a little peremptory to Santer. Does he want people to accept his results or not? If yes, he should release his data, period. If he wants to keep his data to hiimself he should keep his results to himself too.

Mark T.

I’d be willing to bet that 99.9% of the public has no idea, nor cares, about these issues (egregious, IMO). Terms like “very public,” it seems, really are code words for “CA audience.” If, just once, someone like NBC (yeah right), ABC, CBS, CNN, the AP, or Reuters would actually run a story on something like this, then we might see some backlash. Till then, we are alone.

Mark

Paul Penrose

It’s real simple: If you publish your results then you must provide all your data a and computer codes upon request, including intermediate results. This goes double if you are a public employee. If you can’t accept that responsibility then don’t publish. Surely Santer understands this?

Fred

Well, Santer is afraid of his work being scrutinized because he knows it’s barely good enough for government work.

Why else hide your work Santer?

Steve McIntyre

Santer is employed by Lawrence Livermore (not NOAA) and, as far as I know, Livermore is not subject to FOI. Four of his coauthors are employed by NOAA which is subject to FOI. It’s quite possible that none of his coauthors ever saw Santer’s data or discussed it with him – this too would be interesting.

Update – This impression may not be right. It looks like Livermore is a DOE agency and may well subject to FOI.

Soronel Haetir

Steve, Craig,

Okay, I thought the various labs like LL, Sandia, INEEL etc were subject to archiving and FOIA. Sad to find out I was mistaken.

• jae
  Posted Nov 10, 2008 at 10:38 PM | Permalink

  Re: Soronel Haetir (#40),

  Okay, I thought the various labs like LL, Sandia, INEEL etc were subject to archiving and FOIA. Sad to find out I was mistaken.

  More travesty, since I’m paying for all of this. They are run by “private corporations,” which are 100% funded by public money, so they can get away with this crapola. I would like to mention the political implications of this, but it’s a sure snip. (probably a snip, anyway).

J.Hansford.

So Steve. That’s the Santer and the clause……. Eh? 😉

kim

It must be awfully frustrating; in the private sector this kind of bloodhounding would be putting people in jail; here, in the public sector, the dog gets beaten.
================================================

Alan Wilkinson

This is really funny. It also becomes funnier every year the temperature refuses to obey the IPCC climate model predictions.

And every year this continues further discredits the Santer paper, not merely its methodology but more importantly its conclusion.

For your next official information request, you should ask for the Santer data to be updated to all available latest measurements since you wish to publish an updated analysis and version.

No doubt that was why he doesn’t want to talk to you anymore!

Garrett H

It’s obvious that this isn’t merely a coincidence. If I wanted to, what would I need to do in order to statistically prove that such a coincidence is impossible?

Bishop Hill

Oops. Santer who was accused…

Link

Bickers

With the passing of Michael Crichton, who besides being a famous author was a Harvard medical graduate and academic, I suggest those who genuinely believe in the scientific method read his Cal Tech lecture from 2003. It expresses most chillingly the sorry mess that science has become, corrupted as it is by politics and advocacy groups:

http://www.michaelcrichton.net/speech-alienscauseglobalwarming.html

Steve McIntyre

Phil Jones copied me on the following reply to Santer’s email:

Ben,
Your response is already up on the Climate Audit site, and when I looked there were over 50 comments. Don’t feel picked on – we in CRU had another FOI request related to tree-ring data yesterday as well. It is in a similar vein. We put up all the individual tree-ring series (widths, densities) – i.e. what we consider the raw data. He already had the chronologies. He now wants to know why some individual series were excluded from the chronologies and why some chronologies were excluded in subsequent analyses. This time they have asked for manuals, computer code and correspondence explaining the exclusions! It seems neverending. If they just did some paleo fieldwork with trees, corals, sediment cores they might understand why some samples are excluded.

I would urge the 4 NOAA people on the paper to make a joint response to the FOI request when it filters through that the raw data for our paper are all publically available. I know it’s not in their (skeptic) make up, but the sooner they get their hands dirty with the sorts of analyses we/you’ve done for this and many other papers the better. They seem only to want to come in at the interpretational end, particularly on the statistical side.

Cheers
Phil

• craig loehle
  Posted Nov 11, 2008 at 8:35 AM | Permalink

  Re: Steve McIntyre (#59), This is disengenuous and they know it. The raw data is not sufficient to reproduce their work. It is only sufficient to start from scratch and do your own analysis,which is what Phil is suggesting. No concept here on their part that REPLICATION is a crucial and accepted part of the scientific process.

  • akm
    Posted Feb 25, 2010 at 9:45 PM | Permalink

    Replication is starting from scratch and doing your own analysis.
    It should be on your own data.

Michael Jennings

To use very improper English,

it just keeps getting worser and worser

! I see nothing snarky in Steves request and don’t understand the combative tone of Santer in his response. If the information is so readily available as he claims, then just provide the url for the location and poof, it’s done. As for Phil Jones, he is just touchy because his work doesn’t hold up when subjected to scientific scrutiny.

Eric

I don’t get it. Of what value to Steve is Santer’s processed data? If you want to replicate the work and verify the result you need to start with the monthly-mean model output and replicate the analysis. That’s exactly what Santer suggested. Go the PCMDI site, download the model runs and do the analysis. If Santer were to give Steve all of the processed data and/or his codes, Steve is guaranteed to simply get the same result. Then what has he learned or verified – that he can generate the same plot from the same processed data? I do not see the value in that. Santer’s response was obviously rude and inappropriate in tone, but the essence of his suggestion is perfectly appropriate. Get the data and try to replicate the analysis. Only if there is insufficient documentation in the paper to do so, or an attempt to replicate the analysis yields a different result, is there something for Santer et al. to answer for.

Steve: As I said in my request, my interest is in the statistical analysis of the 47 time series, assuming that Santer has collated the information correctly. That’s what Santer et al 2008 is about. I’ve already determined that his Table III doesn’t hold up for his H2 hypothesis; now I want to look at his H1 hypothesis. The collation is irrelevant to that exercise. Re-collating the data contains a risk of introducing an irrelevant discrepancy.

• RomanM
  Posted Nov 11, 2008 at 9:45 AM | Permalink

  Re: Eric (#64),

  I don’t get it. Of what value to Steve is Santer’s processed data?

  Before you can decide if something was done correctly, you need to determine exactly what the authors did. Having the intermediate results can greatly reduce the work and can help find deviations from the descriptions given. If you have the raw data and the final results, it can be very time consuming and sometimes virtually impossible to decode the intervening steps. Look at the Mann 2008 reconstruction. Try to go from the raw proxies to the final reconstructions. Follow the paper and the SI. Oops, some steps were described incompletely, some steps weren’t followed as described and some were simply omitted. However, with the intermediate results, you have signposts which can help you find those places where such inconsistencies occurred. Someone who is confident that their scientific work is properly done would not object to such scrutiny by creating impediments through their lack of cooperation. After all, if a team of 17 authors can’t get it right, who can? 😉

• Kenneth Fritsch
  Posted Nov 11, 2008 at 10:19 AM | Permalink

  Re: Eric (#64),

  I don’t get it. Of what value to Steve is Santer’s processed data? If you want to replicate the work and verify the result you need to start with the monthly-mean model output and replicate the analysis. That’s exactly what Santer suggested. Go the PCMDI site, download the model runs and do the analysis. If Santer were to give Steve all of the processed data and/or his codes, Steve is guaranteed to simply get the same result.

  Eric, I, as a layperson, did some of my own statistical analyses of the RSS and UAH tropical troposphere temperature data up to the present and found that the trend differences between the GISS land and ocean in the tropics and the troposphere where negative (troposphere – surface) for the UAH data and showed no trend with the RSS data. I could only compare those results to that reported by Santer et al. (2008) for the climate models up to 1999 and found that difference in surface to troposphere trends to be positive. It would be great to obtain the latter data from Santer so that the model comparison could be brought up to date.

  Why would any scientists be so mean-spirited that they would not provide that data for an easy comparison? Or why would not that data already be available? Science is supposed to be about seeking the truth and not playing games as Santer and now Jones are doing.

Dodgy Geezer

Artifexis right – “..Sigh. I think these guys are their own worst enemies. By reacting with a knee jerk instead of taking a moment to engage their brains, they come out of this looking pretty poorly…”

But they’re also brave/foolhardy. Courageous in the Sir Humphery sense. I would only have the guts to respond this way to an auditor if I was DEAD sure that my work contained NO errors. If there is ANY kind of issue with Santer’s work, this response will be blown up out of all recognition.

Why does he not play the usual team game of agreeing politely and then refering Steve to a web site where only half the required information is stored? Is this a clever double-take, and has he already secretly submitted his work to three independent sets of auditors…?

Steve McIntyre

#72. If papers are presented as being policy-relevant, then scientists had better get used to the idea that people other than their pals are entitled to examine the data. If you’re suggesting that documentation in these fields is done in a “fashion that others can easily interpret”, you’ve obviously not read this blog. Methods are routinely mis-described.

And make no mistake, it’s not simply being rude. Phil Jones said on another occasion – “We have 25 years invested in this. Why should we let you see our data when your only objective is to find something wrong with it?”

In my own case, when I first looked at the Mann corpus, I had no expectation of finding anything wrong with it and approached such a conclusion very gingerly; however, when Mann said that he had “forgotten” where his data was located and his associate said that it wasn’t in any one place, it was clear that no one had ever checked his work.

Policies at econometrics journals require authors to archive working code and data as a condition of submission. So we’re not talking about a policy that is not already “best practices”.

• Eric
  Posted Nov 11, 2008 at 11:12 AM | Permalink

  Re: Steve McIntyre (#74),
  I am not interested in defending Phil Jones or Micheal Mann. If methods are not properly documented or raw data is being shared only with “friends” then those are problems that need to be exposed and corrected. But do not confuse a lack to willingness to aid a refutation with a lack of willingness to be refuted. I can only say this so many times: replication of an experiment for the purpose of verification should be done independently. As a practicing scientist, I am under no obligation to help you replicate my experiment beyond properly documenting my procedures. Nor would such help be desirable if the goal if the goal is to independently verify a scientific result.

  • Craig Loehle
    Posted Nov 11, 2008 at 12:18 PM | Permalink

    Re: Eric (#77), Perhaps you misunderstand what the data are? The data are the output of a bunch of climate models. There is no experiment and these models are mostly not available to be run by anyone. It is like saying go run your own atom smasher when you think the detection of some particle may be spurious (and it has happened).

    • Eric
      Posted Nov 11, 2008 at 12:42 PM | Permalink

      Re: Craig Loehle (#84),
      I know precisely what we are talking about. In fact, we shouldn’t be talking about “data” because output from a climate model is not “data”. My understanding is that the model runs cited by the IPCC are archived by LLNL in the PCMDI database, however I do not know what their policy is for updating it, so perhaps the runs do not extend as far as those used by Santer et al.(?), it seems that maybe that is what Kenneth is asserting. If that is the case, then Santer et al. would be best served by coughing them up. Also note that many climate models codes are publicly available. I know for certain that NCAR CAM, NASA GISS, and NOAA GFDL codes are all available for download from the web. But all of this misses the point I was trying to make. Indeed, if you do want to independently verify a particle maybe you should build your own atom smasher. I recognize that this may take decades and hundreds of millions of dollars, but new accelerators do get built every now and then. And presumably the predictions of the standard model, as well as the results of past accelerator experiments, are evaluated against the results from newer accelerators. All I am suggesting is that you view climate science in the same way. I don’t think it is really necessary in this case, since the modeling groups have decided to make their output publicly available. But if you don’t think that is sufficient, then download the source codes and run them yourself. If you don’t think that the community’s models are right, or sufficiently independent from one another, then build your own.

    • PhilH
      Posted Nov 11, 2008 at 1:22 PM | Permalink

      Re: Eric (#89), I’m probably missing something here. Suppose X’s atom smasher produces some data. X’s algorythim, which he does not reveal, produces a conclusion based on that data. Are you saying that if Y legitimately suspects that the mathematical method applied to the data has problems, he has to go build another smasher, run it, produce data (that may or not be the same), then guess what X’s math was, and run that… before he is allowed to critique it intelligently? That the proper scientific approach would not be for X to simply tell Y, or the world, what the mathematical method was and let Y, and the world, check it?

    • Eric
      Posted Nov 11, 2008 at 1:34 PM | Permalink

      Re: PhilH (#92),
      I would expect that the mathematical basis for X’s algorithm would be described in X’s paper.

    • Willis Eschenbach
      Posted Nov 11, 2008 at 1:56 PM | Permalink

      Re: Eric (#94), you say:

      I would expect that the mathematical basis for X’s algorithm would be described in X’s paper.

      Heck, so would we all, that’s fundamental to science. However, even the most cursory reading of this blog shows that in climate science this is the exception rather than the rule.

      Which in turn reveals to us that you haven’t done your homework by reading the blog a bit before posting …

      w.

    • Jonathan
      Posted Nov 11, 2008 at 2:03 PM | Permalink

      Re: Willis Eschenbach (#97)
      In my experience most scientific papers don’t clearly describe their algorithms; at best they describe the situation well enough that with a bit of persistence you can work out roughly what they did. The only way to find out what they really did is to get hold of code, sample inputs, and so on. Generally my competitors are willing to provide this if I ask nicely. Most of the journals I publish in require me to do this on request, though there is a sort of gentleman’s agreement not to take advantage of this to acquire unpublished insights.

      To repeat what has been said many times before: climate science is being used to justify multi-billion (multi-trillion?) dollar expenditures. This is not like most science, and we really can (and should) demand higher standards.

    • lucia
      Posted Nov 11, 2008 at 4:33 PM | Permalink

      Re: Eric (#94),

      The precise method for determining the trends in the lower troposphere based on model data are not described in the Santer paper. The current paper describes what they did by pointing to a five page article in Science. I have not downloaded that article, but Science articles tend toward, shall we say, brevity?

      In his response to Steve, Santer suggests algorithms (plural) to do figure out the trends are in other papers.

      As there are multiple algorithms, and they can be tweaked, it will be difficult to reverse engineer what Santer unless Santer explains further. Providing his code or post processed data would accomplish this.

      The situation that has arisen is not uncommon in research. This is why, ordinarily, scientists do grant requests for information or codes that support the major results they present in peer reviewed articles. I don’t know if they are required to do so, but many grant these willingly.

    • Kenneth Fritsch
      Posted Nov 11, 2008 at 4:17 PM | Permalink

      Re: Eric (#89),

      I do not know what their policy is for updating it, so perhaps the runs do not extend as far as those used by Santer et al.(?), it seems that maybe that is what Kenneth is asserting.

      Eric, the calculation complexity comes from converting model output to give the integrated areas of the troposphere that RSS and UAH measure and report as T2 and T2LT. You could pick that up from Santer’s nasty and mocking reply.

      By the way, Eric, what would you, as scientist, do in a case like this if the data could be emailed with little of no effort. Just like to know what kind of person I am talking to, you know.

  • jae
    Posted Nov 11, 2008 at 12:24 PM | Permalink

    Re: Eric (#77),

    As a practicing scientist, I am under no obligation to help you replicate my experiment beyond properly documenting my procedures. Nor would such help be desirable if the goal if the goal is to independently verify a scientific result.

    As a scientist, then you understand that you have the obligation to provide EVERYTHING that is necessary for another person to repeat your experiment. In that case, that has to include (working) computer code, all the data that were used, and any assumptions that were made. Documenting procedures is not enough, when you are using data from somewhere else in forming your conclusions.

    • Eric
      Posted Nov 11, 2008 at 12:59 PM | Permalink

      Re: jae (#85),

      As a scientist, then you understand that you have the obligation to provide EVERYTHING that is necessary for another person to repeat your experiment. In that case, that has to include (working) computer code, all the data that were used, and any assumptions that were made. Documenting procedures is not enough, when you are using data from somewhere else in forming your conclusions.

      I guess I’ll just say it again. It is my opinion that for the purposes of routine validation of a scientific result, I am not obligated to make my computer codes or the results of intermediate analysis steps available to anyone who asks. If you want to verify my results, then read the paper to learn the procedure, then acquire the data and do the calculation yourself. My codes and intermediate steps are not required for you to reproduce the result. Again, this assumes that the paper sufficiently documents the procedure. I do not know if that is the case. If it is not, then there is a problem with the paper.

    • JS
      Posted Nov 11, 2008 at 1:20 PM | Permalink

      Re: Eric (#90),
      There is always a problem with the paper. I have yet to come across a published paper that completely documents the steps used. Publishing space constraints are such that the technical details are the first to be cut. Instead, in the absence of such information, one must try a process of trial and error to work out what was done – and if there are more than a few details missing it becomes near on impossible to work out exactly what was done. Different data vintages can affect results in non-trivial ways – even the ‘raw’ data keeps changing in some areas.

      I expect that the standards should be the equivalent to these:

      “Authors of accepted papers that contain empirical work, simulations, or experimental work must provide to the [journal], prior to publication, the data, programs, and other details of the computations sufficient to permit replication.”
      “For econometric and simulation papers, the minimum requirement should include the data set(s) and programs used to run the final models, plus a description of how previous intermediate data sets and programs were employed to create the final data set(s). Authors are invited to submit these intermediate data files and programs as an option; if they are not provided, authors must fully cooperate with investigators seeking to conduct a replication who request them. The data files and programs can be provided in any format using any statistical package or software. Authors must provide a Readme PDF file listing all included files and documenting the purpose and format of each file provided, as well as instructing a user on how replication can be conducted.”

    • Craig Loehle
      Posted Nov 11, 2008 at 1:27 PM | Permalink

      Re: Eric (#90), Perhaps you could look around this site. You would find that there has been almost not a single case where the documented procedures in a paper have been sufficient to replicate, verify, audit or test a result. In the case of the GISS (official NASA data for US and world temperatures–you can’t go “build your own”) there were errors in the code and when the code was coughed up by Hansen, no one has been able to get it to run. I would remind you that in fields like biochemistry or tricky surgery techniques, it is common to go visit a lab that innovated a technique to see how it was done.

  • Ron Cram
    Posted Dec 24, 2008 at 9:47 AM | Permalink

    Re: Eric (#77),

    You write:

    As a practicing scientist, I am under no obligation to help you replicate my experiment beyond properly documenting my procedures. Nor would such help be desirable if the goal if the goal is to independently verify a scientific result.

    Where do you get this nonsense? Is this what they are teaching in schools these days?

    Perhaps we need to define terms here. Replication can mean different things to different people. In some cases, replication may require obtaining new samples from the field. That seems to be what you have in view. That is not what Steve is requesting. He did call on people to update certain tree ring series relating to the Divergence Problem. When they didn’t, he went into the field and did it himself. The main issue is that these people have not “properly documented procedures” in the first place. The NSF requires researchers to archive their source code, but not everyone does. In this case, you are required to turn over your methodological source code whether you think you should have to or not.

    • Mark T.
      Posted Dec 24, 2008 at 11:43 AM | Permalink

      Re: Ron Cram (#224), The point that Eric is missing is that with the studies like these, properly documenting methods is not sufficient for replication. The input data needs to be made available, too, else replication is not possible.

      Mark

    • Ron Cram
      Posted Dec 24, 2008 at 11:52 AM | Permalink

      Re: Mark T. (#227),

      I do think it is theoretically possible to properly document methods and archive all your code prior to publication, but it is not common practice. Even if someone attempted it, he would probably leave out one or two important elements unintentionally. That is the point I was trying to make. At that point, the researcher cannot say “Take my word for it!” He has cooperate with replication attempts or he is not a scientist.

    • Mark T.
      Posted Dec 24, 2008 at 12:33 PM | Permalink

      Re: Ron Cram (#229), What I was getting at is that some experiments do not require an external data but others do. For example, when I describe a method for detecting multipath signals in noise, anyone that is knowledgeable w.r.t. communication theory can easily generate their own data to test my theory. Santer’s specific example, however, cannot be replicated without access to the actual data used. Simply saying “do the runs yourself” is not sufficient as it would generate different results (at least, that’s my understanding of this issue), so documenting the methods is not sufficient.

      Mark

    • Ron Cram
      Posted Dec 24, 2008 at 2:30 PM | Permalink

      Re: Mark T. (#231),

      Now I understand you and agree completely.

Soronel Haetir

Eric,

Producing a new GCM does nothing to verify or refute the existing GCMs, it merely provides another model. Given that this very topic started from trying to either verify or refute existing GCMs I don’t see how producing another GCM helps in that quest.

anonymous

The idea that you can deny access to important intermediate calculations of a peer reveiwed paper makes a mockery of science.

If that were really a justifiable notion, you could have two seperate studies on the same raw data doing in general the same thing but coming up with contradictory results with both sides saying “the raw data is there, if you filter out the ‘correct’ items and work it out properly for yourselves your results would comply with ours”. How does one compare the two papers for validity ? By doing your own study and coming up with a third result (of which you will with-hold the intermediate steps again) ?

What an embarassment for climate science.

Sam Urbinto

In summary, you have access to all the raw information that you require in order to determine whether the conclusions reached in our IJoC paper are sound or unsound.

What does the raw data have to do with conclusions?

Raw information lets you recreate something to see what your conclusions are, and if they match the conclusions of others.

Conclusions are checked by checking the conclusions.

I see no reason why I should … provide you with derived quantities … which you can easily compute yourself.

What does being able to compute (easily or not) have to do with not providing the existing supporting data for comparison?

I see no reason why I should do your work for you…

Sorry, Steve, Dr. Benjamin D. Santer (http://www.nersc.gov/news/annual_reports/annrep03/advances/5.1.fingerprints.html) isn’t going to let you copy his homework out on the playground.

aurbo

Re #82

However, by looking too closely at the trees (statistics) we don’t see the forest (the premise).

I think you’ve got it exactly backwards. In the topic under discussion, it is precisely the nature of the trees that needs to be examined. Santer, by accumulating 16 other authors to bear witness to his premises has, in effect, created a forest to obscure the tree that is a product of his own construction. And to extend the metaphor a bit further, he has also attracted additional supple saplings [accent on the first syllable] who are willing to bend over backwards and beyond to rationalize and justify his…dare I say…”egregious” behavior.

Sam Urbinto

It’s like math class; show your work to get credit for the solution to the problem.

Re: Eric (#94),

Even if a paper describes the mathematical basis for an algorithm in the first place, that may or may not detail it well enough to be replicated. The same steps and methods that leads to the same results. A non issue, that’s not the subject.

We are not discussing replicating or investigating a study, but rather the examination of the results.

Noblesse Oblige

I strongly support your pursuit of requested ‘data’ under the FOI Act. The behavior and arrogance of these people are appalling and indeed seems to be worsening as evidence at odds with the IPCC central position accumulates.
Meanwhile I am scratching my head over a simple conceptual problem I have with the Santer et al approach. As I understand it, the authors defined a class of models which have somewhat differing treatments of the climate system, including presumably differing climate sensitivities. This class of models produced an ensemble of runs for the historical period in question. A statistical analysis of this ensemble is presented that purports to show that the ensemble is indistinguishable from the actual historical record, when ‘weather noise’ is included. Therefore, the conclusion we are presented with is that the class of models as a whole is ‘consistent’ with observed trends. And by implication, attacks on the models per se are ill founded.
If I understand this correctly, it seems strange to define a class of differing predictive methods (models) with the purpose of determining that the class as a whole is consistent with observation. One might argue that it is more logical to consider the predictions of specific models, one model at a time, and statistically rule in or rule out these specific models based on how close their predictions come to observation. I suspect that if you do this, you will find that those with lower computed climate sensitivity to greenhouse gases come closer to the observed record, providing evidence that the actual sensitivity is lower than the bulk of the models.
In practice, different models are probably not completely independent of each other, so the actual number of ‘degrees of freedom’ probably lies somewhere between one and the number of models. Still, it seems that the scientific objective ought to be to determine what the results of different models, when projected against the historical record, can tell us about the climate sensitivity, not whether models as a methodology class are ok or not.
Comments welcome.

• jae
  Posted Nov 11, 2008 at 4:17 PM | Permalink

  Re: Noblesse Oblige (#107),

  FWIW, I completely agree. I never could see any kind of reality in an “ensemble of models.” I even think the whole idea is funny. If you have enough models, you get enough variation to cover any historical record. And I’ll bet that the models are so inter-bred that the df are quite close to 1.

• Lucy Skywalker
  Posted Nov 11, 2008 at 6:47 PM | Permalink

  Re: Noblesse Oblige (#107), “the scientific objective ought to be to determine what the results of different models, when projected against the historical record, can tell us about the climate sensitivity”

  It would be wonderful if Bishop Hill (or someone) would consider writing up the Santer – Douglass & Christy story like you did with the Hockey Stick. It seems that the sensitivity issue, and the nonexistent troposphere heat spot, is an important link in the serial auditing of Climate Science. This can greatly help those like myself who care deeply about what is happening in Science at present, as Michael Crichton did, but may have difficulty understanding scientific/mathematical language at times, and cannot follow every blog here, yet want to grasp the science and the story correctly and adequately.

• Alan Wilkinson
  Posted Nov 11, 2008 at 7:40 PM | Permalink

  Re: Noblesse Oblige (#107),

  I agree and have made these points more than once. Also until climate modellers can show proof their model predictions are convergent rather than divergent both individually and as a group it seems invalid to use them to predict anything at all and certainly not for any runs that lie outside the experimental confidence limits of the observed data.

• craig loehle
  Posted Nov 11, 2008 at 10:23 PM | Permalink

  Re: Noblesse Oblige (#107), A number of posts have raised exactly the same point. It seems sensible to deal with an ensemble generated by a single model perturbed with different initial conditions, stochastic blah blah. BUT an ensemble of different models? I don’t think so.

Steve McIntyre

One point that I’d like to remind readers is that climate authors seldom use “standard” methods described in statistical texts. Articles usually studied here are seldom “methodological” or “applied” – but a quirky mix of the two in which the authors are seeking to establish a result on applied data, but generally do something a bit novel in their data handling.

Steve McIntyre

#114. That’s a nice observation. Santer has described a procedure for converting model outputs into synthetic MSU series, through “static weighting functions” – the synthetic PCMDI runs would appear to be within the scope of this licence. Maybe someone could send a request to Santer specifically requesting the synthetic T2LT series obtained from Hadley Center data, referring to the Hadley licence.

Spam

Do they make ethics classes mandatory in science degrees nowadays? I certainly don’t recall any.

William Harrison

As a government (DOD) scientist who has been FOIAed I can assure you that Ben won’t enjoy the process. Where he used to be in control he will quickly find that a FOIA lawyer at NNSA is actually in charge. And the FOIA attorney works for the public and not him. Imagine his surprise when computer tapes are made, emails are collected and in general some attorney makes his life uncomfortable until the letter and spirit of the request are satisfied. It is much better to give it up willingly and quickly – unless you are a masochist. And the ultimate criminal act is to delete files. Do that and go to jail or hope that POTUS will parden you.

Noblesse Oblige

Re: Lucy Skywalker (#120). I think the Bishop would do a great job of telling this story. In fact someday, someone will write a definitive scholarly history of the whole global warming business. This and the hockey stick will be part of it. Bishop writes beautifully about PROCESS, while Lindzen is masterful at narrating the SCIENCE and science politics. Some combination of the two capabilities would be commanding.
It is too early to write the entire history because we don’t yet know the ending, but Volume 1 would be welcome.

Jeremy, Alabama

When real science, such as mathematics, use algorithms as part of a proof, the algorithms are subject to rigorous review. For instance, the 4-color map theorem.

For a scientist to say, “I have the proof. It is proven. I have proved it. Proven, it is. Work out the details yourself”, is hardly science.

Soronel Haetir

Eric,

I would argue that the code should be the ultimate authority on what the method actually is. It might not be pretty, but working code provides an unambiguous definition of whatever matter is under discussion. It is much harder to provide that level of clarity with other forms of language.

Steve McIntyre

Eric, once agian, I refer you to the mission statement of PCMDI where Santer is employed. It states that “PCMDI’s mission demands that we work on both scientific projects and infrastructural tasks. … Examples of ongoing infrastructural tasks include … the assembly/organization of observational data sets for model validation.”

Much of the effort in Santer et al appears to have been the assembly/organization of data sets, which is what PCMDI is funded to do. The employees have been compensated for the assembly/organization of the data sets used in Santer et al 2008 and, given this mission statement – which is over and above any general obligations as a scientist – it is absurd for him to say that I should do the infrastructural work myself and the managers of the laboratory have an obligation to deal with this refusal to adhere to the approved PCMDI mission.

John Baltutis

javascript:edInsertContent(edCanvas,’Re:%20%3Ca%20href=%22#comment-310953%22%3EEric%20(%2368)%3C/a%3E,%20′);updateLivePreview();

As long as the unprocessed data is available and there is sufficient documentation for an independent person or group to replicate the work, then his responsibility as a scientist is met.

But there isn’t sufficient documentation to replicate the work. That would have been crystal clear if you had read the threads on this blog.

Gerry Morrow

There’s something interesting, if not important in Phil Jones’ note, he starts with:

“Ben,
Your response is already up on the Climate Audit site, and when I looked there were over 50 comments.”

Read it carefully and it becomes obvious that this isn’t the first e-mail on the subject. “Your response is already up on the Climate Audit site…” Response to what? We have clearly been included in an already started e-mail conversation, it looks to me like Dr. Santer has been looking to Prof Phil for advice on how to stonewall the requests. Important? Maybe not, but interesting nonetheless. Could we not put an FOI request in for the rest of this correspondence now they’ve copies Steve on it?

Lucy Skywalker

Was it The Firm that told the story of an innocent who found himself in a firm of corrupt lawyers, realized he’d been framed and compromised so he could not come out with the issue direct but collected (audited) hundreds of chits that showed their record of serial falsification, each one tiny by itself, but together massive?

Perry Debell

I’ve arrived late at this debate, but the comment by Eric #77 strikes me as bizarre, namely “As a practicing scientist, I am under no obligation to help you replicate my experiment beyond properly documenting my procedures. Nor would such help be desirable if the goal if the goal is to independently verify a scientific result.”

If one were to have discovered a method of cold fusion, what would be the advantage of saying to other researchers “Yah boo, sucks to you! I discovered cold fusion, but I’m not going to let you know how to do it.” That’s my reading of your comment & others have also noted some strangeness in your way of thinking. Now, if there were to be some huge financial reward at the end of your research, naturally it would make sense to protect your investment in your efforts, but as far as I can tell, that’s not the issue here. You just seem to think it’s a good thing to obstruct othesr from verifying any of your work, which begs the question “How good is your work?”.

You may not like my opinion, but your attitude is not nice and, so far, your more recent comments have failed to change my mind. You are trying to defend the indefensible and I would suggest you remember the adage, “When you are in a hole, the first thing to do is STOP digging”. BTW, stop practicing to be a (pompous) scientist, just be one!

Good eh?

Smokey

I see no reason why I should do your work for you.

Providing the requested information would probably entail no more than a handful of mouse clicks.

That is surely less ‘work’ than Santer’s lengthy response entailed. So Santer is hiding something. We can only speculate what it is. But if his work was straightforward and honest, Santer would have no reason to stonewall like this.

• Sunsettommy
  Posted Nov 12, 2008 at 9:13 PM | Permalink

  Re: Smokey (#144), Smokey writes:

  “Providing the requested information would probably entail no more than a handful of mouse clicks.

  That is surely less ‘work’ than Santer’s lengthy response entailed. So Santer is hiding something. We can only speculate what it is. But if his work was straightforward and honest, Santer would have no reason to stonewall like this.”

  I am not so sure he is hiding something.It could just fear of being “audited” by a perceived skeptic and be exposed for it.

  A groupthink mentality that has been apparent for sometime now.A decided reluctance to share the data with others who are clearly interested in the research.It could be self preservation.

  We read of Phil Jones telling (post # 59) Santer about Steve’s effort for data.Did you note the exasperation? The same Dr. Jones who refused to share data with Warwick Hughes.As McIntire points out in post # 74:

  “And make no mistake, it’s not simply being rude. Phil Jones said on another occasion – “We have 25 years invested in this. Why should we let you see our data when your only objective is to find something wrong with it?””

  When I read of such comments from scientists about honest requests for data.I am thinking they are afraid that their entire belief system will come crashing down.

  It is fear that prevents them from doing the obvious.Sharing the data.

craig loehle

I would like to point to a scientist giving a proper response. In the following paper (of interest to readers here by the way):
Springer, G.S., H.D. Rowe, B. Hardt, R.L. Edwards, and H. Cheng. 2008. Solar forcing of Holocene droughts in a stalagmite record from West Virginia in east-central North America. Geophysical Research Letters 35, L17703, doi:10.1029/2008GL034971.
The data is available from the author (and he sent it right away). I found a discrepancy in Fig. 1 graph vs the data (Ha! beat you to it Steve!) and notified the author. In a few hours he replied that I was right and he will send a notice to the journal. Quick, painless, honest. Wow. Good job Greg!

Mark T.

It’s got nothing to do with having something to hide. It has everything to do with Steve being their pariah. Steve has made life hard for the Team, or at least harder than it otherwise would have been, not through malfeasance on his part, but simple attention to details that they would have preferred went unnoticed. Santer’s reply, as well as much of the stonewalling Steve runs into (though not all), is payback.

Mark

• Sam Urbinto
  Posted Nov 13, 2008 at 10:25 AM | Permalink

  Re: Mark T. (#150),

  AFAIC, Steve’s just as much, if not more, “a climatologist” as they are. But since he’s not on “The Team and Associates, Inc”, well.

  Who is this brash interloper!

  Re: Robinedwards (#157),

  Exactly so. If some work is done on data that is “unique” and then conclusions are reached using that data, the only way to validate/verify/recreate the conclusions is to have the unique data.

  Unless the process is explained well enough to replicate and the contention is that the results give the same basic answer every time.

  I don’t know how these sorts of issues can be determined and then worked out without cooperation.

Jeff Alberts

So, in the absence of evidence that any one of the climate models is right, it has to be assumed that all of them are wrong. At issue is how wrong they are.

I would add that even if a model gets it “right”, there’s absolutely no way anyone can know if they got it “right” for the right reasons. Maybe the replication of climate patterns happened by accident, maybe it didn’t. Unless you know absolutely 100% everything there is to know about global climate and all it’s internal and external inputs, you simply can’t know if you got it “right”.

Regular lurker

data –> (tp1) processing –> (tp2) stat analysis –> (tp3) conclusion

Tp = testpoint.

1. Do we want to have robust conclusions?

2. The more testing, the more robust are the conclusion.

3. A conclusion that gives access to testdata is easier to test, compared to a conclusion where only inputdata is given.

4. The more important the conclusion is, the more important the testing becomes.

5. If lots of work are required to test, then less testing will be done.

Eric, do you disagree on any of the points above?

Michael Sirks

I remember an old analysis of Santer’s earlier work by the late John Daly.
http://www.john-daly.com/sonde.htm

That is why he doesn’t like people to analyse his work. Some even called his handling of data “to santerize data”.

So now you can “santerize” data and “mann handle” data.

• jae
  Posted Nov 12, 2008 at 3:45 PM | Permalink

  Re: Michael Sirks (#155),

  LOL. Deja Vu all over again. Looks like truncation of datasets is established methodology among some climate scientists.

• Alan Wilkinson
  Posted Nov 12, 2008 at 5:32 PM | Permalink

  Re: Michael Sirks (#155),

  A damning link, Michael. Santer obviously has a track record as do Nature’s climate science referees.

• Kenneth Fritsch
  Posted Nov 12, 2008 at 5:56 PM | Permalink

  Re: Michael Sirks (#155),

  Chris Knappenberger was here at CA several days ago pointing to the problem that Ben Santer has with short time series. Santer et al (2008) was evidently not the first instance. But Santer’s time periods used, for whatever reason, are trumped by the longer time periods — so let us move on.

Larry

As a layman from a science standpoint, yet as a lawyer who is deeply concerned with the impact of so-called scientific “findings” upon public policy in connection with climate change, I see the debate generated in the posts about Ben Santer’s refusal to provide Steve with the requested data as just another example of how science in this country and around the world has become overly politicized. The fact that “Eric” just doesn’t seem to get what the true problem is constitutes further proof.

If you want to publish findings that reach a certain conclusion, and you are so confident about your conclusion, what is the problem with producing the data? What do you have to hide? And what is the problem with someone whom you think is “trying to find something wrong with your work?” I have to deal with that all of the time in my own work, but I don’t worry about it, because I know my work will withstand scrutiny. But withholding data seems to be the way of government scientists today, along with those other “scientists” in the alarmist camp. Let’s see the evidence, let’s have the debate.

Steve, keep up the good work. You need to continue to question as much as you can, because the road to hell is pretty greased these days.

bill-tb

Perhaps this Data Quality Act may be of assistance … http://en.wikipedia.org/wiki/Data_Quality_Act

It applies to all government sources, government grants and government contractors.

Colin Davidson

Science is all about replication. If you can’t replicate it, it isn’t Science. We have a word for that: Art.

Santer is an Artist as his work cannot at present be replicated.

David L Hagen

Interesting reviewer comment on Santer in NAS 2005 Review of Ch 2, OCR for page 18

In fact, the method of Santer et al. (2000) seems to rely on the assumption that after subtracting trends, the time series is of AR1 form, which can indeed be characterized by the first-order autocorrelation. However, the AR1 assumption may not be correct and is certainly unnecessary as it is possible to fit a general ARMA (autoregressive, moving average) model with scarcely any more work. The “arima” function in the freely available R statistical package allows for fitting a linear regression component with ARMA errors, where the autoregressive and moving average components are of arbitrary order.

Review of the U.S. Climate Change Science Program’s Synthesis and Assessment Product on Temperature Trends in the Lower Atmosphere (2005), p 15 THE NATIONAL ACADEMIES PRESS

http://books.nap.edu/openbook.php?isbn=030909674X&page=31

Nicolas J.

Hello,

I wonder if, apart from Eric, people here have actually worked in labs. As a former graduate Physics student, and son of a research director, I have some small experience of laboratory research and the use of data.

It’s easy to understand why scientists are not all keen on delivering all their intermediate results. After all, no company would be keen on delivering its intermediate results that lead to their final products, and not only because of fear of competition. But mainly because contrarily to what has been said several times, most of the intermediate results are uninteresting and therefore discarded. Sometimes, they are not very pretty either, as is most of the source code that is produced daily everywhere. Actually, everyone who has done it before knows that presenting intermediate results in a complete and easily reproducible manner is very time consuming, while not adding much value.

Besides, producing all the intermediate results would mean keeping Gb of simulation data, which is impractical and wasteful. It would also mean that one has to document details like all the softwares used (and sometimes, there are many, with a lot of ugly data transfer between), their configuration, etc, which no scientist has time, not to mention patience, for.
The article is supposed to provide the mathematical tools and assumptions that led the data analysis, which is a much better way to convey the relevant information in order to replicate the analysis with your own tools.

All this being said, it is good that people try to independently examine these results, and the tone of Dr Santer certainly was unwelcome even though understandable.

Nicolas J.

The fact that they store Tb of data doesn’t mean that they store intermediate results, but only meaningful ones, and also the datasets that allow to reconstruct them (hopefully), and as per your quote, datasets that are used for model validation.

Again, I wish good luck with your request.

Steve McIntyre

#177. First of all, I happen to know what’s involved in collecting data from remote locations as I have many years of experience in mining exploration stocks.

Second, thousands of businesses have to deal with auditors every year. I have experience with this as well. You can’t just tell an auditor – go do your own accounts. I realize that this example is foreign to scientists, but the link comes through public policy and dealing with the public. If scientists expect their results to be applied in public policy, then the public has a right to expect that scientists properly document and archive their results and methods so that third parties can examine them.

Journal peer review is not an audit process; it’s a cursory screening, and all too often it’s concerned more with defending a POV than ensuring replicability of results. When, as a reviewer for Climatic Change, I asked for supporting data and code, the editor said that I was the first reviewer in the 28-year history of the journal to do so and he refused to require the author to comply.

Andy

Steve: “First of all, I happen to know what’s involved in collecting data from remote locations as I have many years of experience in mining exploration stocks.”

OK. I didn’t say “all”… …out of interest, would you have just handed those data over to whoever asked for them?

“If scientists expect their results to be applied in public policy, then the public has a right to expect that scientists properly document and archive their results and methods so that third parties can examine them.”

First off, I think that it’s up to politicians to decide whether or not the standards of documentation and archiving of results are sufficient to base policy decisions on. If they don’t want to make an unpopular decision (“say goodbye to your Hummers” or somesuch) then they can say “you didn’t do a good enough job convincing me”. Most of the time the public is outraged when politicians refuse to act on evidence of increased risk to life and health (Katrina being a recent case-in-point). It seems to me that the right balance is struck when those that have to make the policy decisions define the standards of documentation and archive of the information on which they have to act.

“Journal peer review is not an audit process; it’s a cursory screening, and all too often it’s concerned more with defending a POV than ensuring replicability of results. When, as a reviewer for Climatic Change, I asked for supporting data and code, the editor said that I was the first reviewer in the 28-year history of the journal to do so and he refused to require the author to comply.”

It seems we are somewhat in agreement on this point. I have also experienced the occasional problem with reproducibility of results from the information given in journal papers. But subsequent contact with the lead author has often led to fruitful collaboration. My biggest problem with the peer-review process in journals is that editors seldom do their job these days. The reviewers are supposed to assist the editor in making his decision but the latter often abdicates his responsibility, essentially saying “unless you can satisfy the reviewer, the paper won’t be published”. Perhaps it’s because there are so many papers these days that editors just don’t have the time. A review is certainly not an audit but “cursory screening” belittles the work that many reviewers do. With reading the manuscript, making sure one understands the main concepts, checking the key references, examining the plots, maybe testing a few cases for oneself and ensuring that the conclusions do not overreach the data, I’d say a good two days’ (unpaid) effort is the norm for a first round review.

• UK John
  Posted Dec 23, 2008 at 3:46 PM | Permalink

  Re: Andy (#184),

  Should we rely on Peer Review or do we need an Audit?

  Should we trust “Policy Makers” to get things right?, shouldn’t we always question them, as they can and do get it wrong, just like anybody else.

  In my view too few questions are asked, we wouldn’t be in a mess if more people questioned consensus, as Steve does.

  Or do you just want people not to question your scientific consensus?

• Dave Dardinger
  Posted Dec 23, 2008 at 3:48 PM | Permalink

  Re: Andy (#184),

  …out of interest, would you have just handed those data over to whoever asked for them?

  So what’s your favorite game, softball? Steve is going to have fun knocking that one out of the park.

Steve McIntyre

#184. As a start, there is a very clear policy statement requiring federal recipients to archive data after a limited period of exclusive use. If NSF enforced these policies on climate scientists that I deal with, that would go a long way to resolving some of my issues. Or if journals enforced their stated policies. To a considerable degree, NSF has become a cheerleader and has abdicated its compliance responsibilities. This happens from time to time in business situations and it seldom is to the benefit of shareholders and the public.

If you notice, I spend far more time and attention trying to get journals and agencies to administer their stated policies. I actually take more exception to mealymouthed answers from bureaucrats and editors than to obstruction by climate scientists.

But if a scientist like Santer chooses to make a spectacle of himself as in the present case, Ill see what happens under FOI, under journal policies and under agency policies.

gens

Sorry for the bad formatting – I clearly haven’t mastered the “Link” function.

mikep

Replication has quite a long history in economics – and the AER polices arise in part from past problems. See e.g. this interesting paper by Daniel Hamermesh – a well respected US Labor Economist.

Click to access dp2760.pdf

Economics shares with paleoclimate studies the problem of limited data sources and being for the most part non-experimental. When it is possible to repeat experiments then simple replication of other’s results is not necessary because you just do your own experiment. But in non-experimental fields replication is often the only way of ensuring that results are indeed robust, and in exploring why the results came out the way they do. For example it was only because of Steve’s attempt to replicate the MBH results that it emerged that the method Mann et al described as principal components was anything but standard principal components analysis.

Andy

OK I’ve had a skim-read of the Ababneh thesis and what I could find on the Bona Churchill ice core. I would’ve expected a well-oiled (a.k.a. well-funded) machine like Thompson’s to turn out the results quite quickly if they had been easy to understand (i.e. fitted the original hypothesis). So it looks at tho’ they have a lot of thinking to do. In fact both cases indicate that local climate is a significant issue when interpreting proxy data. I guess that, in turn, means there’s always likely to be pickings for anyone of a skeptical disposition…

Back to the original point. It seems that Santer et al. are, umm, disinclined to cooperate. It seems that they are being defensive and this is perhaps somewhat understandable. I’m not sure how to overcome the divide but I don’t think it’s too hard to undertake the analysis steps that Santer refers to, if you’re concerned that there’s something rotten in the state of CA. That could legitimately lead to a “comments on” paper in IJoC, assuming you find something wrong. But I s’pose there’s also the (not widely accepted, in this field at least) principle of making datasets available. It may be worth Santer’s (or maybe NOAA’s) while to give you the data since, as #28 points out, the data availability issue is potentially more harmful anyway.

• Ron Cram
  Posted Dec 24, 2008 at 8:17 AM | Permalink

  Re: Andy (#200),

  You talk about Santer et al being “umm, disinclined to cooperate.” Is that acceptable behavior in your mind?

  I found an online reference on theImportance of Replication. Quote:

  Students are taught that a research report should include all necessary details to permit replication. However, it is often impossible to find all relevant details about how research is conducted in a published report of research. Sometimes, to carry out an exact replication, one must contact earlier researchers to learn details of a procedure. Gasparikova-Krasnec and Ging (1987) found that earlier researchers were generally cooperative in providing needed information for later researchers. A month’s wait was normally all that was required.

  Researchers typically are willing to provide details of their procedures to a scientist planning a replication, If a replication fails, the original findings are cast into doubt. However, serious researchers realize this process of double-checking is important to science. Naturally they hope for confirmation of their earlier efforts. However, if the attempted replication fails, this is not entirely bad news (as long as no fraud is involved). A failed replication may have a stimulating effect on a field of research, inspiring new studies to figure out why an attempt to use the “same” procedures led to different results on different occasions or with different populations.

  If “researchers are typically willing to provide details” and journals and funding sources require it, how or why should climate science be exempt?

  To my mind, the claim of an exemption to the standards of science brings dishonor on the researcher, the journal and his place of employment. I am sorry, but I do get a little worked up over this issue. Let me say it plainly. Any researcher who refuses to provide the information necessary to replicate is not a scientist. He is a pseudoscientist and his work should be treated as such.

  • Craig Loehle
    Posted Dec 24, 2008 at 9:07 AM | Permalink

    Re: Ron Cram (#221), Tell us how you really feel, Ron…

    • Ron Cram
      Posted Dec 24, 2008 at 9:34 AM | Permalink

      Re: Craig Loehle (#222),

      Craig, I know, I know. It is the Cram temper coming out. I hope none are offended except those who need offending.

Andy

Dave: I was really expecting a physical science journal – Beilstein is a reference compilation. I don’t think the original authors were expected to verify the procedures. Some might have done so, of course. The Handbook authors obviously had an interest in replicating the experiments they had gleaned from the literature prior to inclusion – very rigorous (one might say Germanic) of them. But if you really want to document, for example, the Unified Model, there’s an awful lot of code and documentation to review. Are you serious…?

• Dave Dardinger
  Posted Dec 23, 2008 at 6:24 PM | Permalink

  Re: Andy (#201),

  if you really want to document, for example, the Unified Model

  Are we talking a Climate Model or a GUT / TOE here? In any case, why shouldn’t the data and code and Run outputs be available for such things? Obviously they aren’t in the papers themselves, but the SIs should have them.

  Your whole position seems to be, “We’ve never done it that way before.” Not exactly a confidence-building position.

Andy

“…really want to document…” should be “…really want to audit…”

trevor

Andy,

Good on you for engaging here at CA. You can expect to be treated cordially, and your points addressed with respect. I have two reactions in response to your points.

Steve: “First of all, I happen to know what’s involved in collecting data from remote locations as I have many years of experience in mining exploration stocks.”

OK. I didn’t say “all”… …out of interest, would you have just handed those data over to whoever asked for them?

The Australian JORC Code provides quite stringent requirements on mining and exploration companies to properly disclose their results. At the risk of repeating material that has already been posted, here are the principles under which the JORC Code is framed:

“4. The main principles governing the operation and application of the JORC Code are transparency,
materiality and competence.

• Transparency requires that the reader of a Public Report is provided with sufficient information, the presentation of which is clear and unambiguous, to understand the report and is not misled.

• Materiality requires that a Public Report contains all the relevant information which investors and their professional advisers would reasonably require, and reasonably expect to find in the report, for the purpose of making a reasoned and balanced judgement regarding the Exploration Results, Mineral Resources or Ore Reserves being reported.

• Competence requires that the Public Report be based on work that is the responsibility of suitably qualified and experienced persons who are subject to an enforceable professional code of ethics.”

The JORC Code is enshrined in Australian Corporations Law and also Australian Stock Exchange Listing Rules, and it is mandatory that all listed mining and exploration companies comply. These rules have been developed over the past 20-25 years, and have become accepted good practice that all serious industry participants accept as being fair and reasonable. The outcome is that mining and exploration companies, especially the smaller ones for whom exploration results are decidedly material, report their exploration results in abundant detail. The JORC Code provides quite detailed guidance on the level of disclosure expected, and practice has improved markedly over the years.

The other major mining countries have adopted similar codes that are either based on the JORC Code, or on similar practice adopted for the same reasons.

The other point I want to make is that those concerned about man’s CO2 emissions, and consider that those CO2 emissions will cause catastrophic global warming, are in effect asking the world population to spend trillions of dollars to address the problem. And those trillions will affect the hip pocket of many millions of people through tax imposts, and higher charges for energy. Whole industries will be shut down, and hundreds of thousands of people will be either dislocated or put out of work, or both. Now it is not to say that this isn’t a reasonable request of the world population if, as you so clearly believe, that anthropogenic CO2 is the worst problem facing mankind.

I hope that you will forgive the more sceptical of us for asking what seem to me to be reasonable qquestions before we simply agree to your requests. It is surely reasonable for us to ask whether the temperature record that you are using really can be relied upon – have the urban heat island effects been properly adjusted for? Are the temperature readings reliable? Have the changing list of temperature stations been properly accounted for? And surely it is reasonable for us to ask whether the proposition that man’s CO2 emissions will lead to a doubling of atmospheric CO2 levels is really understood and substantiated. Do we really understand the CO2 cycle and processes on the planet? And why shouldn’t we ask how it is that you are so confident that a doubling of atmospheric CO2 levels (if it happens) will lead to an increase in Global Mean Temperature of 3 degrees C (the latest figure that James Hansen seems to be using) when it is evident that the science indicates that the sensitivity is more like 1 deg C per doubling of CO2, with the extra 2 degrees coming from assumptions about feedback processes that it seems could just as well be negative (thus reducing the 1 deg C impact) as it could be positive.

It seems to me that the real climate scientists have not covered themselves in glory. They fail to provide clear evidence in support of their propositions, fail to address questions, fail to provide data, processes, programs that would support their conclusions, and generally seek to confuse and obfuscate the issues.

As well, you have Stephen Schneider and Al Gore both admitting that they consider it OK to “exaggerate” the concerns in order to raise public concern to the point where action is taken to address the problem. In their minds the end justifies the means.

It seems to me that it would be helpful if the real climate scientists would comply with the terms of their publicly funded grants in relation to reporting results and supporting information. It would be helpful if they would comply with the policies publishing their “peer reviewed” papers requiring disclosure of supporting information. It would be helpful if they would seriously address the questions directed to them. And it would be helpful if they desisted from the emotional ad hominem comments that seem to characterise the debate.

None of this is unusual or unreasonable. It is normal accepted practice in most other fields. Why should climate science be immune from requirements to adhere to sound practice?

Andy

Dave: OK, I mean the UKMO Unified Model, a version of which has been ported to Linux and used to be available (was last time I checked) but now looks as tho’ it requires a specific agreement with the UK Met Office. Still, there’s the Community Climate Model, which is available at:

http://www.earthsystemgrid.org/browse/browse.htm?uri=http://datagrid.ucar.edu/metadata/cgd/ccsm/thredds/ccsm.thredds

after a simple registration and a wait of, well, I’ll find out (I just registered, but it is Christmas, so…).

As I understand it (having typed “PCDMI” into Google a minute ago and clicked on the 2nd-from-top link) output data from a number of models are available for download after registration.

Steve McIntyre

#210. My request to Santer was for the data as used by him. It’s easy to register at PCMDI. But you won’t find the data as used by Santer. I’m handy at extracting data from archives and don’t mind working at it, and the public record is pretty clear on that -so that’s not the issue. However, if you are able to replicate Santer’s data from the information in the article, then bully for you. Please send it to me.

But y’know, I’m not going to argue about this. There are policies governing this. Let’s see what happens. I’ve sent FOI requests to the Department of Energy and NOAA; I’ve sent a request to the journal under their policies; I’ve sent a request to DOE under their mission statement. It seems to me that it would have been simpler for Santer to send me the data, but, as I say, if he wants to be a small center of attention in this corner of the blogosphere, I’ll oblige him.

I’ve had some success with this with other obstructionist climate scientists. We’ve been getting information from Jones and Briffa inch by inch under the British FOI legislation. Science magazine hated the bad press about Esper and eventually made him provide most of his data (though Thompson is too big for them to deal with.)

If the climate science community chooses to cheer for obstructionism, as Phil Jones cheered Santer on, this is not a strategy that I would recommend to them. IF they are confident in their results, then they should provide data instantly. Take this sort of issue off the table.

Craig Loehle

It is my understanding that if you publish a paper describing a protein sequence/structure you must publish your data to back up the claim. Andy seems to dismiss the argument that safety critical studies like those upon which drug acceptance and toxicology standards are based have high standards and that this should apply to climate science. The legislation for GHG regulation makes regulating a single industrial chemical or pesticide seem totally trivial in terms of its potential economic impact. If this diverts money from health care, from research, from education, then yes it can kill. It seems odd to me to act like the people doing key research here have no responsibility to the public, when many of them in fact work in government labs and produce official data sets like GISS or CRU.

Ron Cram

Steve,

I was wondering if you had seen this “opportunity to comment.”

Steve McIntyre

#224. The NSF does NOT require paleoclimate scientists to archive source code. They don’t even require them to archive data. They have abdicated their responsibilities, saying it’s up to the journals. IPCC has likewise abdicated their responsibilities to the journals – both somewhat ironic in that the editor of the journal that published Santer didn’t even know whether his journal had a policy. What a comedy.

• Ron Cram
  Posted Dec 24, 2008 at 11:47 AM | Permalink

  Re: Steve McIntyre (#225),

  I am not so sure. If true, it is probably changing. NSF policy for Social, Behavior and Economic Sciences requires researchers to archive or share mathematical or computer models they have established. I would be surprised if NSF would not like to see a uniform policy on these questions.

  Strangely, Geosciences has three different data policies. The Atmospheric Sciences division is still under development. The most promising is from Ocean Sciences.

  The Division of Ocean Sciences says it “expects investigators to share with other researchers, at no more than incremental costs and within a reasonable time, the data, samples, physical collections and other supporting materials created or gathered in the course of the work. It also encourages grantees to share software and inventions or otherwise act to make the innovations they embody widely useful and usable.”

  If paleoclimate scientists get any funding from Division of Ocean Sciences, they would be bound to share software.

  The Division of Earth Sciences requires “researchers and organizations to make results, data, derived data products, and collections available to the research community in a timely manner and at a reasonable cost.” A good lawyer might argue that source code created to analyze data is a “derived data product” and should be archived. But you might have a fight on that.

  And of course, you always have the possibility of non-compliance even when the policy is very clear.

• Ron Cram
  Posted Dec 24, 2008 at 12:08 PM | Permalink

  Re: Steve McIntyre (#225),

  I have communicated with climate scientists who published in AGU publications and did not know about AGU data archiving policies, but I really am surprised Santer’s editor did not know if they had policy or not. What a comedy indeed.

Andy

Ron: I don’t see the requirement for making source code available in public archive in the AGU data policy. The sentence you have highlighted refers to AGU publications and the role they play in disseminating new techniques to the community. A new technique can be described in a paper without having to provide the source code. Their data policy really deals with just two issues. Firstly, AGU spell out their requirement for referencing a dataset, i.e., if a dataset is referenced in a paper, it must be publicly available. If it is not publicly available then it cannot be referenced explicitly in the text and must be treated in a similar manner to that of a personal communication in the manuscript. Secondly, if a paper essentially describes the creation of a new dataset (what AGU refers to as a “data paper”) and not with scientific analysis, etc., then the dataset which is being described must be made publicly available.

As far as I can see, NSF Ocean Sciences does not require that software be made available, although they do encourage it. It is my recollection that NASA expects this but I looked at their policy on STI and couldn’t find anything specific (I may have missed something tho’).

• Ron Cram
  Posted Dec 24, 2008 at 5:50 PM | Permalink

  Re: Andy (#238),

  I misread that line. I see you are correct. The AGU policy requires data archiving but not really anything on source code. You are right again that NSF Ocean Sciences “encourages” researchers to make software available, but does not use the word “expects.” I am sure why the difference in wording. If the government paid for the development of the source code, they can surely require it be shared to other researchers for the purpose of replicating published science papers.

Mark T

It shouldn’t be, but sadly, it is a very partisan issue.

Mark

John M

Steve Mc,

Speaking of the AGU and requirements for source code documentation, perhaps you have a soulmate in the AGU.

I wonder how his talk was received.

link

Steve McIntyre

GRL and JGR do not require paleoclimate authors to observe AGU policies. I wrote one GRL editor about this a couple of years ago and he said that they were already asking reviewers to do enough without putting them to the additional burden of asking them to ensure that authors archived their data.

During IPCC AR4 review, D’Arrigo et al was then pending at JGR. I asked IPCC for the supporting data and they blew me off saying that they would not do “secretarial” duties for me, even though I had contacted a “secretariat”. They told me that archiving was the responsibility of the journals. I wrote to the journal (JGR)asking for the data and additionally asking them to ensure that the data was archived in accordance with AGU policies.

Instead of doing that, they complained back to IPCC. IPCC said that I only knew about this article through IPCC and this knowledge was subject to confidentiality and that requesting data was a breach of confidentiality and that, if it happened again, I would be expelled as a reviewer. I still don’t have the data.

• Ron Cram
  Posted Dec 26, 2008 at 12:29 PM | Permalink

  Re: Steve McIntyre (#246),

  Stories like these… wow. Some of them I have read before but am still shocked. I think I will become a member of AGU so I can speak out on this.

  BTW, as I mentioned above, EOS was requesting comments on whether AGU should update its 2003 statement on “Human Impacts on Climate.” I left my comment but learned it will probably not be posted on the website because I am not an AGU member. There’s another reason to join.

  Also, I just wrote an article for Citizendium on “Ocean heat content.” Citizendium was founded by Larry Sanger, one of the founders of Wikipedia who left because he wanted to improve the quality of the articles. I hope readers will find it interesting.

ZT

Just observed the following comment on real climate, I thought I’d record it here:

In:

http://www.realclimate.org/index.php/archives/2010/02/close-encounters-of-the-absurd-kind/

Comment: 72
Pasteur01 says:
25 February 2010 at 8:01 AM

Dr. Santer,

Did your immediate supervisor decide to release the data before you did?

“…preparation of the datasets and documentation for them began before your FOIA request was received by us.” Dr. David Bader, January 2009

“A little over a month after receiving Mr. McIntyre’s Freedom of Information Act requests, I decided to release all of the intermediate calculations I had performed for our International Journal of Climatology paper.” Dr. Ben Santer, February 2010

• Jimchip
  Posted Feb 25, 2010 at 5:41 PM | Permalink

  Re: ZT (Feb 25 15:29),

  Ya but he sure whined about it in the emails. I think he even threatened to quit LLNL, if I recal correctly.

• Pasteur01
  Posted Feb 26, 2010 at 2:38 PM | Permalink

  Also posted on RC…

  Further to my earlier post, I created the following timeline from the leaked emails and from CA posts.

  On November 10, 2008 McIntyre receives refusal to release intermediate data from Dr. Santer.

  McIntyre files FOIA request for the intermediate data and related emails on the same day, November 10, 2008.

  November 10 or November 11, 2008 Thomas Karl informs Dr. Santer of Mr. McIntyre’s request.

  Dr. Santer cc’d his immediate superior, Dr. David Bader, among others in a response to Thomas Karl in an email dated November 11th, 2008.

  Thereafter, but before December 16, 2008, Dr. Santer sends an email to his co-authors indicating that he learned of Mr. McIntyre’s FOIA request “earlier this morning.” In that same email Dr. Santer indicates that he had been discussing the “FOIA Issue” with others including Dr. Bader for “several weeks.”

  On January 30, 2009 Dr. Bader sends an email to Mr. McIntyre indicating that the decision to release the intermediate data was made before the FOIA request was received.

  In his recent statement on this blog Dr. Santer indicated that the decision to release the intermediate data was made after receiving the FOIA request.

  • Jimchip
    Posted Feb 26, 2010 at 3:26 PM | Permalink

    Re: Pasteur01 (Feb 26 14:38),

    I got a little confused. Perhaps Dr. Santer needs to perform a time series reconstruction.

    My favorite email from that time period is where Gavin Cawley thinks he found a Santer boo boo (Wigley calms the team down). Published at RealClimate: group @ 12 December 2007
    
    “Once more unto the breach, dear friends, once more!” (King Henry, no less)

    From: Tom Wigley
    To: Ben Santer , Phil Jones
    Subject: [Fwd: Re: Possible error in recent IJC paper]
    Date: Sat, 01 Nov 2008 18:50:12 -0600

    Hi Ben & Phil, No need to push this further, and you probably realize this anyhow, but the
    RealClimate criticism of Doug et al. is simply wrong.

  • Jimchip
    Posted Feb 26, 2010 at 4:31 PM | Permalink

    Re: Pasteur01 (Feb 26 14:38),

    OK, here’s a timeline:

    1224005421.txt
    From: Ben Santer
    To: David Douglass
    Subject: Response
    Date: Tue, 14 Oct 2008 13:30:21 -0700

    Prof. Douglass,

    You have access to EXACTLY THE SAME radiosonde data that we used in our
    recently-published paper in the International Journal of Climatology
    (IJoC). You are perfectly within your rights to verify the calculations
    we performed with those radiosonde data. You are welcome to do so…
    —–

    1225140121.txt
    From: Phil Jones
    To: santer1@llnl.gov
    Subject: Re: End of the road…
    Date: Mon Oct 27 16:42:01 2008

    Ben,

    It seems that Climate Audit has been discussing the paper. I ad
    a look whilst I was in Iceland as I had nothing better to do a few times.
    It was cold and snowy outside, there was internet…..
    Seems as though they are making some poor assumptions; someone
    is trying to defend us, but gets rounded upon and one of the co-authors
    on the paper is in touch with McIntyre.
    As it isn’t me, and I can rule out a number of the others, my list of who
    it might be isn’t that long….
    Looking forward to next week !!
    Cheers
    Phil
    —–

    This one is the interesting one:

    1225412081.txt
    From: Ben Santer
    To: “‘Philip D. Jones'”
    Subject: [Fwd: Re: [Fwd: Typo in equation 12 Santer.]]
    Date: Thu, 30 Oct 2008 20:14:41 -0700

    Dear Phil,

    I thought you’d be interested in my reply to Gavin (see forwarded email).

    Cheers,

    Ben

    Dear Gavin,

    There is no typo in equation 12. The first term under the square root in
    equation 12 is a standard estimate of the variance of a sample mean
    (see, e.g., “Statistical Analysis in Climate Research”, Zwiers and
    Storch, their equation 5.24, page 86). The second term under the square
    root sign is a very different beast – an estimate of the variance of the
    observed trend….
    —–

    1225462391.txt
    From: Ben Santer
    To: “Thorne, Peter” , Peter.Thorne@noaa.gov, Leopold Haimberger , Karl Taylor , Tom Wigley , John Lanzante , Susan.Solomon@noaa.gov, Melissa Free , peter gleckler , “‘Philip D. Jones'” , Thomas R Karl , Steve Klein , carl mears , Doug Nychka , Gavin Schmidt , Steven Sherwood , Frank Wentz
    Subject: [Fwd: Santer et al 2008]
    Date: Fri, 31 Oct 2008 10:13:11 -0700

    Dear folks, While on travel in Hawaii, I received a request from Steven McIntyre for all of the model data used in our IJoC paper (see forwarded email). After some conversation with my PCMDI colleagues, I have decided not to respond to McIntyre’s request…
    —–

    [The other Gavin but in the timeline: 1225465306.txt
    From: “Cawley Gavin Dr \(CMP\)”
    To:
    Subject: RE: Possible error in recent IJC paper
    Date: Fri, 31 Oct 2008 11:01:46 -0000
    Cc: “Jones Philip Prof \(ENV\)” , “Gavin Schmidt” , “Thorne, Peter” , “Tom Wigley”

    Dear Ben,
    many thanks for the full response to my query. I think my confusion arose from the discussion on RealClimate (which prompted our earlier communication on this topic), which clearly suggested that the observed trend should be expected to lie within the spread of the models,…]
    —–

    1226337052.txt
    From: Ben Santer
    To: Steve McIntyre
    Subject: Re: FW: Santer et al 2008
    Date: Mon, 10 Nov 2008 12:10:52 -0800

    Dear Mr. McIntyre,

    I gather that your intent is to “audit” the findings of our
    recently-published paper in the International Journal of Climatology
    (IJoC)…I gather that you have appointed yourself as an independent arbiter of
    the appropriate use of statistical tools in climate research. Rather
    that “auditing” our paper, you should be directing your attention to the
    2007 IJoC paper published by David Douglass et al., which contains an
    egregious statistical error….
    —–

    Then,

    1226451442.txt
    From: Ben Santer
    To: “Thomas.R.Karl”
    Subject: Re: [Fwd: FOI Request]
    Date: Tue, 11 Nov 2008 19:57:22 -0800

    …Providing Mr. McIntyre with the quantities that I derived from the raw
    model data (spatially-averaged time series of surface temperatures and
    synthetic Microwave Sounding Unit [MSU] temperatures) would defeat the
    very purpose of an audit…McIntyre’s request (2) demands “any correspondence concerning these
    monthly time series between Santer and/or other coauthors of Santer et
    al 2008 and NOAA employees between 2006 and October 2008”. I do not know
    how you intend to respond this second request. You and three other NOAA
    co-authors on our paper (Susan Solomon, Melissa Free, and John Lanzante)…
    —–

    Then, from Wigley,

    1226456830.txt
    From: Tom Wigley
    To: santer1@llnl.gov
    Subject: Re: [Fwd: FOI Request]
    Date: Tue, 11 Nov 2008 21:27:10 -0700

    …Hmmm. I note the following ,,,

    “at which I can be contacted between 9 and 7 pm Eastern Daylight Time”

    Is this a 22 hour, or, for people with time machine, a negative 2 hour
    window?

    Joking aside, it seems as a matter of principle (albeit a principle yet
    to be set by the courts) that provision of primary data sources that are
    sufficient to reproduce the results of a scientific analysis is all that
    is necessary under FOI.

    It also seems that judgment of what correspondence is central to the
    analysis can only be made by the persons involved…
    —–

    Then, of course, Phil is involved:

    1226500291.txt
    From: Phil Jones
    To: santer1@llnl.gov
    Subject: Re: [Fwd: FOI Request]
    Date: Wed Nov 12 09:31:31 2008
    
    Ben,
    Another point to discuss when you have your conference call – is
    why don’t they ask Douglass for all his data…
    —–

    It goes on into December…

    1228249747.txt
    From: wigley@ucar.edu
    To: santer1@llnl.gov
    Subject: Re: Further fallout from our IJoC paper
    Date: Tue, 2 Dec 2008 15:29:07 -0700 (MST)
    Cc: santer1@llnl.gov, “Thorne, Peter” , peter.thorne@noaa.gov, “Leopold Haimberger” , “Karl Taylor” , “Tom Wigley” , “John Lanzante” , susan.solomon@noaa.gov, “Melissa Free” , “peter gleckler” , “‘Philip D. Jones'” , “Thomas R Karl” , “Steve Klein” , “carl mears” , “Doug Nychka” , “Gavin Schmidt” , “Steven Sherwood” , “Frank Wentz”

    Ben,

    I support you on this. However, there is more to be said than
    what you give below. For instance, it would be useful to note
    that, in principle, an audit scheme could be a good thing if done
    properly. But an audit must start at square one (your point). So,
    one can appear to applaud McIntyre at first, but then go on to
    note that his modus operandi seems to be flawed…
    —–

    Here’s Schmidt with political advice

    1228258714.txt
    From: Gavin Schmidt
    To: santer1@llnl.gov
    Subject: Re: Further fallout from our IJoC paper
    Date: 02 Dec 2008 17:58:34 -0500
    Cc: “Thorne, Peter” , Peter.Thorne@noaa.gov, Leopold Haimberger , Karl Taylor , Tom Wigley , John Lanzante , Susan.Solomon@noaa.gov, Melissa Free , peter gleckler , “‘Philip D. Jones'” , Thomas R Karl , Steve Klein , carl mears , Doug Nychka , Steve Sherwood , Frank Wentz

    Ben, there are two very different things going on here. One is technical
    and related to the actual science and the actual statistics, the second
    is political, and is much more concerned with how incidents like this
    can be portrayed. The second is the issue here…
    —–

    1228330629.txt
    From: Phil Jones
    To: santer1@llnl.gov, Tom Wigley
    Subject: Re: Schles suggestion
    Date: Wed Dec 3 13:57:09 2008
    Cc: mann , Gavin Schmidt , Karl Taylor , peter gleckler

    Ben,
    When the FOI requests began here, the FOI person said we had to abide
    by the requests. It took a couple of half hour sessions – one at a screen, to convince
    them otherwise
    showing them what CA was all about. Once they became aware of the types of people we were
    dealing with, everyone at UEA (in the registry and in the Environmental Sciences school
    – the head of school and a few others) became very supportive. I’ve got to know the FOI
    person quite well and the Chief Librarian – who deals with appeals. The VC is also
    aware of what is going on – at least for one of the requests, but probably doesn’t know
    the number we’re dealing with. We are in double figures.

    One issue is that these requests aren’t that widely known within the School. So
    I don’t know who else at UEA may be getting them. CRU is moving up the ladder of
    requests at UEA though – we’re way behind computing though. We’re away of
    requests going to others in the UK – MOHC, Reading, DEFRA and Imperial College.
    So spelling out all the detail to the LLNL management should be the first thing
    you do. I hope that Dave is being supportive at PCMDI.
    The inadvertent email I sent last month has led to a Data Protection Act request sent by
    a certain Canadian, saying that the email maligned his scientific credibility with his
    peers!

    This one is interesting:

    1233326033.txt
    From: Ben Santer
    To: Smithg
    Subject: Re: data request
    Date: Fri, 30 Jan 2009 09:33:53 -0800

    Dear Mr. Smith,

    Please do not lecture me on “good science and replicability”. Mr.
    McIntyre had access to all of the primary model and observational data
    necessary to replicate our results. Full replication of our results
    would have required Mr. McIntyre to invest time and effort. He was
    unwilling to do that…
    —–

    Also,

    1233586975.txt
    From: Ben Santer
    To: P.Jones@uea.ac.uk
    Subject: Re: [Fwd: data availability]
    Date: Mon, 02 Feb 2009 10:02:55 -0800
    Reply-to: santer1@llnl.gov

    Dear Phil,

    Yes, this is the same Geoff Smith who wrote to me. Do you know who he
    is? From his comments about the RMS, he seems to be a Brit…
    —–

    • Pasteur01
      Posted Feb 27, 2010 at 11:18 AM | Permalink

      Love the hit parade.

      As for my posts I finally understand that the 2 FOIA requests, NOAA and DOE, found their way to Dr. Santer at different times.

      But I still don’t understand why Dr. Bader found it necessary to insist for the record that they decided to post the data before receiving Mr. McIntyre’s FOIA request. I mean they already knew he was trying to get it through NOAA employees. And very obviously (even without the benefit of the leaked emails) they were preparing and posting the data in response to that request.

      Steve: It’s sometimes hard to figure out why they say what they do.

    • Jimchip
      Posted Feb 27, 2010 at 12:59 PM | Permalink

      Re: Pasteur01 (Feb 27 11:18),

      I don’t think the “hit parade” is even complete (just based on my search algorithm) but it gives context. I think the Bader statements are sorta administratively political, trying to hold off on a flood of “harassing” FOIA requests, which is a claim that Jones and Santer frequently made. There are so many times when many could have just followed standard practice and published what they were supposed to…At the minimum, it made it look like there was something rotten, some “dirty laundry”, perhaps.