NAS Report on Data and Methods Disclosure

Aug 20, 2009 – 8:17 AM

Jeff Id on the Air Vent has written a post pointing out the recent publication online of a report by the Committee on Ensuring the Utility and Integrity of Research Data in a Digital Age from the National Academy of Sciences: Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age. I am starting this thread so that it can also be discussed here

The committee and its objectives were discussed on CA, for example, here and in a number of other places. You can find the other threads easily by using the search CA feature at the top right of the page for the phrase “NAS Committee”.

An executive summary of the report is available in pdf format and access to an online version of the full text is available here.

It makes for interesting light reading. From the summary:

Legitimate reasons may exist for keeping some data private or delaying their release, but the default assumption should be that research data, methods (including the techniques, procedures, and tools that have been used to collect, generate, or analyze data, such as models, computer code, and input data), and other information integral to a publicly reported result will be publicly accessible when results are reported, at no more than the cost of fulfilling a user request. This assumption underlies the following principle of accessibility:

Data Access and Sharing Principle: Research data, methods, and other information integral to publicly reported results should be publicly accessible.

(bold in report)

Maybe the folks at HadCru should pay attention…

Update (using my Comment 1:)

It appears that there may be some caveats on those for whom the the data should be accessible. From page 2 of the summary chapter (page 3 of the pdf) (all bold mine):

Documenting work flows, instruments, procedures, and measurements so that others can fully understand the context of data is a vital task, but this can be difficult and time-consuming. Furthermore, digital technologies can tempt those who are unaware of or dismissive of accepted practices in a particular research field to manipulate data inappropriately.

On the next page, this seems to be clarified somewhat:

The most effective method for ensuring the integrity of research data is to ensure high standards for openness and transparency. To the extent that data and other information integral to research results are provided to other experts, errors in data collection, analysis, and interpretation (intentional or unintentional) can be discovered and corrected. This requires that the methods and tools used to generate and manipulate the data be available to peers who have the background to understand that information.

The “public” appears to be those who are deemed to deserve it by the owners of the data and methods. After all, who knows what damage can be done when an examination of the data and methods is carried out by someone who doesn’t “understand the information” or associated “accepted practices”. 😉

By RomanM | Posted in Archiving, Data, Disclosure and Diligence, NAS Panel | Comments (93)

Svalbard’s Lost Decades

Aug 17, 2009 – 3:06 PM

In a 2006 article in JGR, Aslak Grinsted, John Moore, Viejo Pohjola, Tonu Martma and Elisabeth Isaksson study several climate indicators from the Lomonosovfonna ice field in Svalbard, shown below with their caption:

Svalbard climate proxies
Figure 5. Fifteen-year moving averages of Lomonosovfonna ice core data. (a) Oxygen isotopes, (b) continentality proxy (A), (c) stratigraphic melt indices (SMI), and (d) washout indices (solid line is W_NaMg, and dashed line is W_ClK).

They conclude,

In the oldest part of the core (1130-1200), the washout indices are more than 4 times as high as those seen during the last century, indicating a high degree of runoff. Since 1997 we have performed regular snow pit studies [Virkkunen, 2004], and the very warm 2001 summer resulted in similar loss of ions and washout ratios as the earliest part of the core. This suggests that the Medieval Warm Period [Jones and Mann, 2004] in Svalbard summer conditions were as warm (or warmer) as present-day, consistent with the Northern Hemisphere temperature reconstruction of Moberg et al [2005].

Although the Svalbard ice core record extends back to 1130, a 2009 paper in Climate Dynamics, by Grinsted and 3 of the same authors plus M. Macias Fauria, S. Helama, M. Timonen, and M. Eronen, utilizes the same ice core record to infer winter sea extent, yet omits the distinctively “warm” first 7 decades of the record. It concludes, “The twentieth century sustained the lowest sea ice extent values since A.D. 1200.”

My question for Dr. Grinsted and any of his co-authors who might drop in is, why did the first 7 decades of the core disappear between 2006 and 2009? Is it because they contradict the IPCC/AIT line that there was no MWP to speak of?

The 2006 paper has already been commented on by World Climate Report, while the inconsistency of the 2009 paper has already been noted by Steve McIntyre on Climate Audit.

Dr. Grinsted does occasionally visit CA, and contributed several helpful comments clarifying his smoothing algorithm on the 7/3 thread The Secret of the Rahmstorf ‘Non-Linear Trend Line’.

BTW, has the Lomonosovfona core data ever been archived? I gathered from Steve’s post that it has not.

I might add that Craig Loehle and myself (see Loehle 2007, Loehle and McCulloch 2008) have reconfirmed the existence of a MWP, using twice as many proxies as Moberg et al. Craig selected the proxies and did the smoothing, while I contributed standard errors to the 2008 correction, showing that the MWP and LIA were both significant relative to the bimillenial average. We did not use Lomonosovfona, but it could be a useful addition to future such studies, if calibrated to temperature and archived.

Update: I would like to thank Dr. Grinsted for responding at length in comment #25 below, as follows:

It is curious that we find that it was warm in Svalbard during the MWP but we do not see a low sea ice extent in our sea ice reconstruction. I would have expected it to be lower even though it does not extend quite as far back. When i was interviewed by the danish press then I pointed this out as the most surprising result. But I do not see a conflict between those two observations.

We had several considerations that led us to restrict the sea ice reconstruction to 1200.

We knew that the oldest data 1100-1200 was influenced by melt to such a degree that ions were being flushed from the ice (Grinsted et al. 2006 and the figure shown on this blog). That made us cautious of whether the isotope data might be influenced by post-depositional processes as well. 1200 seemed a natural choice for the cut-off (see above fig).

The dating-model is expected to perform poorer near the bed. We believe the Lomonosovfonna dating to be quite accurate around 1200, since we have identified a sulphuric peak that we believe to be the 1259 eruption (Moore et al., JGR 2006). However, it is very important for the recontruction procedure that the dating is correct to within 5 years. Otherwise we might try to reconstruct the past ice extent using a lag between ice core and tree ring data that is inconsistent with the one used in the calibration period. The primary reason to do the 5-year smoothing was to make the reconstruction more robust against small dating errors. The dating could still be good prior to 1200AD, however we did not have confidence that the errors would be in an acceptable range for the treatment we were planning and therefore excluded this data. Note that, 1200AD is only 2m above the max depth of the ice core.

The layers gets compressed near the bed and the temporal resolution decreases back in time. For the reconstruction we needed atleast 5 year resolution, because that is what we chose in the calibration period. At 1200AD the d18o temporal resolution is 3-4years per sample. That is OK, but not very good when we want to resample to 5year averages.

@Hu (4): it is also asked why I only showed post-1400 d18O in my JGR 2006 paper. The reason is that E. Isaksson wanted to publish this herself before anybody else could get access to it. That simple. This is also where I will redirect all requests for the isotope data.

He also thoughtfully replies to several questions posed by readers in comments #25, 27, 30, 49, and 59 below.

Update 2: In a subsquent paper with K. Virkkunen et al, Dr. Grinsted and co-authors report on the washout factor from two pits at the summit of Lomonosovfonna that update the original core, which was drilled in 1997. An announcement, entitled “Present day summers in Svalbard are as warm as those during the medieval warm period”, is on Dr. Grinsted’s website, with a link to the full paper. Note that since the horizontal scale is depth in meters rather than inferred calendar date, the present is at the left, while the 12th century is at the right and highly compressed.

It is not clear that this is the same as either of the washout measures shown in figure 5 from Grinsted 2006 above, however, since neither of those has conspicuous up-spikes corresponding to the ones this update shows at around 25 and 32 meters.

By Hu McCulloch | Posted in General | Tagged | Comments (95)

Rejected … by RC!

Aug 15, 2009 – 3:20 PM

I have been rejected at RealClimate!
My first rejection! I have not posted there in about two years although I have occasionally read some of their consensus defences when they were relevant to what I have been looking at. They have been running a Steig Corrigendum thread concurrent with ours and I have followed it sporadically. I got a little irritated by a comment from what appears to be one of their regulars and decided to clarify something that had been puzzling me for several days. So I posted the following:

#52 Chris

Some scientists do some research and publish a paper. A correction is made. In the meantime various wannabe’s, armchair numerologists and self-important grandstanders pick and hack at the work, insult the scientists at a distance and play second-guessing games about their motives.

With all this talk about motives, nobody seems to have taken a close look here at the Corrigendum itself. In my role as one of the wannabe numerologists, my examination of the correction has lead me to some puzzling questions.
In his original post on CA, Dr. McCullough detrended the sequence of Antarctic temperatures and estimated the first order autocorrelation of the residuals to be .318. It was not difficult to verify that this result is correct and that the methods used were in fact the usual ones from the statistics literature for doing such a calculation.
On the other hand, in the Corrigendum, Dr. Steig states

We report in Table 1 the corrected values, based on a two-tailed t-test, with the number of degrees of freedom adjusted for autocorrelation, using Neffective = N(1 – r)/(1 + r), in which N is the sample size and r is the lag-1 autocorrelation coefficient of the residuals of the detrended time series. The median of r is 0.27, resulting in a reduction in the degrees of freedom from N = 600 to Neffective = 345 for the monthly time series.

giving a value of .27 for the same autocorrelation. The puzzling question is why is this value different? Since the method for calculating this value was not specified, and using the clue of the word “median”, I was able to pick and hack a possible means by which this answer could be arrived at.
If one takes the original 5509 individual monthly sequences and applies the methods used by Dr. McCullough to each sequence: detrend (with a different trend) each sequence and calculate the autocorrelation in each case, with the median of the resulting 5509 correlations is 0.2692838 which rounds to .27.
What I don’t understand is why this result has anything to do with the actual autocorrelation of the residuals of the averaged sequence from which the temperature trend is calculated. If one realizes that the 5509 sequences are all calculated from the same three principal components, then it is easy to surmise that they are strongly correlated to each other making the use of that value quite inappropriate without establishing a theoretical reason for its use.

However, in the original post by “group”, it states

The corrected calculations were done using well-known methods, the details of which are available in myriad statistics textbooks and journal articles. There can therefore be no claim on Dr. McCulloch’s part of any originality either for the idea of making such a correction, nor for the methods for doing so, all of which were discussed in the original paper.

I am not aware of any justification for Dr. Steig’s calculation above in statistics textbooks or statistics journal articles. If the method that I used gives the same answer by coincidence, and the answer can be arrived by some other appropriate method, I am also not aware of that. Perhaps, Chris or dhog or “group” could point me to a reference for how the calculation was done – I would appreciate that.
By the way, the difference in the final result is a reduction in confidence interval length of slightly more than five percent without a reduction also in the probability value as well making the temperature trend “more significant”.

About 45 minutes later, it had disappeared from moderation status.

Hey, I thought they would like it because it showed that Dr. Steig did not plagiarize this from Hu (yeah, I notice I misspelled your last name twice in the comment – 😦 – but I’ll leave the comment verbatim) since the correction wasn’t done by the known-to-be-appropriate method Hu had used. I would definitely be interested in a valid explanation and/or justification of the difference, but apparently it won’t come from RC. Maybe someone else can do that for me.

My second question is, if the corrigendum has a corrigendum, can I get credit? 😉

By RomanM | Posted in Spurious, Statistics, Steig at al 2009 | Tagged | Comments (198)

Erice 2009

Aug 15, 2009 – 9:52 AM

I’m off to Italy for the next two weeks, starting tonight.

I suggested that the Erice seminar of the World Federation of Scientists have a session on Water Cycle Feedback, as this seemed to me to be the most important scientific issue affecting climate sensitivity. I had a couple of motives for suggesting this – first, it seemed to me to be the most important scientific issue involved in climate sensitivity; and second, I wanted to do something constructive for the seminar so that I would be invited back. It was a great visit last year and both my wife and I wanted to go again.

Fortunately, Zichichi and Manoli liked the suggestion and asked me to suggest terms and help with speakers. My description of the issue was:

The big question in climate is the “sensitivity” of global climate to doubled CO2. Estimates in AR4 GCMs vary from 2 to 4.5 deg C. Within the IPCC GCMs, the primary source of sensitivity uncertainty is cloud feedbacks, and, in particular, the shortwave response of low-level clouds (marine boundary layer). Despite a wide range of sensitivity within IPCC GCMs, IPCC GCMs do not necessarily span both sides of relevant characteristics of critical cloud types, e.g in tropical and midlatitude regions, simulations systemically yield clouds that are too optically thick, and not abundant enough in the mid-troposphere and in large-scale subsidence regimes.

Questions:
– IPCC AR4 pinpointed the largest source of intermodel variability between IPCC models to SW response of low-level clouds. What are the prospects for narrowing this source of uncertainty?
– IPCC AR4 identified several areas of systemic bias in cloud modeling GCMs e.g in tropical and midlatitude regions by simulating clouds generally too optically thick, and not abundant enough in the mid-troposphere and in large-scale subsidence regimes. Do these (or other biases) result in any material risk of under- or over-estimation of climate sensitivity? How could this be tested?
– are cloud (and other GCM) properties sufficiently constrained to confidently exclude the possibility of either very high climate sensitivity (above 4.5 deg C) or low climate sensitivity (under 2 deg C)?
– if not, which modeling components (cloud or otherwise) are mostly likely to result in a large enough systemic bias to yield either a higher or lower climate sensitivity? What steps can be taken to reduce such areas of uncertainty?
– Can paleoclimate and/or other indirect studies add any confidence to constraining climate sensitivity?

I suggested a number of speakers on both sides of the debate. My own hope was that I’d be able to learn something from the debate. The roster has ended up being over-weighted towards “skeptic” speakers: Lindzen, Arking, Choi, Kininmonth, Paltridge, but it is a blue-ribbon “skeptic” session. Invited but unable to attend: Peter Webster, Judy Curry, Andrew Dessler, Stephen Schwartz, Sandrine Bony, Tom Crowley. The invitations were only made in May and schedules had unforunately filled in for many people by then. The final session is:

CLIMATE & CLOUDS
FOCUS: Sensitivity of Climate to Additional CO2 as indicated by Water Cycle Feedback Issues
Chairman A.Zichichi – Co-chair R. Lindzen

10.30 – 12.00
SESSION N° 9
* Dr. William Kininmonth
Australasian Climate Research, Melbourne, Australia
A Natural Constraint to Anthropogenic Global Warming
* Professor Albert Arking
Earth and Planetary Sciences Dept., John Hopkins University, Baltimore, USA
Effect of Sahara Dust on Vertical Transport of Moisture over the Tropical Atlantic and its Impact on Upper Tropospheric Moisture and Water Vapor Feedback.
* Dr. Yon-Sang Choi
Earth, Atmospheric and Planetary Sciences Dept., MIT, Cambridge, USA
Detailed Properties of Clouds and Aerosols obtained from Satellite Data
* Professor Richard S. Lindzen
Department of Earth, Atmospheric & Planetary Sciences, MIT, Cambridge
On the Determination of Climate Feedbacks from Satellite Data
* Professor Garth W. Paltridge
Australian National University and University of Tasmania, Hobart, Australia
Two Basic Problems of Simulating Climate Feedbacks

I’ll check in from time to time from Erice, but my blog attendance will be spotty. We’re going to be traveling in Italy for a few days on either side of the conference – in east Sicily around Siracusa for the next couple of days and Florence for a few days after the conference.

I’ve asked a few regulars to keep an eye on things while I’m away.

By Stephen McIntyre | Posted in General | Comments (29)

Steig Professes Ignorance

Aug 14, 2009 – 4:07 PM

On Feb. 26, I wrote a post on CA, “Steig 2009’s Non-Correction for Serial Correlation”, commenting on the Jan. 22 letter in Nature by Eric Steig et al. On Feb. 28, I sent Steig and his 5 co-authors an e-mail alerting them to my post and its content.

On Aug. 6, Steig and co-authors published a Corrigendum in Nature replicating my findings, but without mentioning my prior post. I wrote the editors of Nature a letter complaining that if the Corrigendum was received after Feb. 28, it would constitute plagiarism under Nature‘s definition as “when an author attempts to pass off someone else’s work as his or her own.” My letter to Nature, together with my e-mail to Steig and co-authors, is in Comment 60 of the CA thread on the Steig Corrigendum.

On Aug. 10, Steig wrote Nature Associate Editor Michael White the following letter, with a copy to myself:

Date: Mon, 10 Aug 2009 08:31:16 -0700 (PDT)
From: Eric Steig
To: mwhite@nature.com
Cc: mcculloch.2@osu.edu
Subject: regarding Dr. McCulloch’s claim

Department of Earth and Space Sciences
University of Washington, Seattle
August 10th, 2009

Dr. Michael White, Editor
Nature

Dear Dr. White,

I understand that Nature has received a letter from Prof. Hu McCulloch, claiming that the August 6th Corrigendum (to my paper in the January 22nd issue of Nature) is a plagiarism of his work. He makes this claim on the grounds that he posted an article on the blog climateaudit.org in February showing the same information as in our Corrigendum, and that he informed me of this in an email on February 28th. Any such email sent to me in February would have received notification that I was in the field in Antarctica until late March, and not receiving email. I was unaware of his post, and did not read it.

The accusation of plagiarism implies I have presented Prof. McCulloch’s ideas as my own. His ‘ideas’ in this case are the recognition that we did not account for autocorrelation in our data when calculating the significance of trends. While I regret that the published version of the paper didn’t include such a correction, it is obvious that I was aware of the need to do so, since in the text of the paper we state that we did in fact make this correction. Once I recognized that we had neglected to make the correction properly, we re-did the calculations using well-known methods, the details of which are available in myriad statistics textbooks and journal articles.

There can therefore be no claim on Dr. McCulloch’s part of any originality either for the idea of making such a correction, nor for the methods for doing so, all of which were discussed in our original paper.

Had Dr. McCulloch been the first person to make me aware of the error in our paper, or had he written directly to Nature at any time prior to the submission of our Corrigendum, it would have been appropriate to acknowledge him in the Corrigendum and we would have been happy to do so. To suggest that correcting an error in my own work, using standard methods, constitutes plagiarism is specious.

Sincerely,

Eric J. Steig, Professor
University of Washington

On August 12, White wrote me as follows:

Date: Wed, 12 Aug 2009 13:00:14 +0100
From: “White, Michael”
To: mcculloch.2@osu.edu
Cc: Eric Steig

Dear Dr McCulloch (cc to Dr Steig)

Thank you for your letter of 7 August 2009. Dr Steig sent us the below correspondence (which he cc’d to you) in which he outlined to us the sequence of events leading up to the submission of his corrigendum. In the light of this information, we see no need for further action on our part. If you feel that the description provided is not accurate then this is something that you would need to take up with Dr Steig directly or with his institution.

Yours sincerely,

Dr Michael White
Associate Editor
Nature

Since Steig professes ignorance of my post and claims that he had not read it, I can only take him at his word. Accordingly, I wrote White today thanking him for his prompt attention to the matter and withdrawing my complaint.

However, while ignorance may be an iron-clad defense against plagiarism, it is a rather dicey position academically speaking. Surely Steig and co-authors would at least read the vigorous and serious discussion of their paper on Climate Audit, the Air Vent, and other blogs, even if they do not deign to participate.

If Steig doesn’t follow CA, he must be the only person in all of climate science. It is well known that within 24 hours of when Steve McIntyre hinted on CA that there was a problem with the Harry AWS data used by Steig et al, Gavin Schmidt of NASA and RealClimate had reported the problem to the British Antarctic Survey, who manages the data. It is less well known that at 1:22 AM on Feb. 2, way down in Comment 171 of the Dirty Harry 4 thread, I noticed that there was also a problem with Steig’s AWS station #52. “Andy” quickly identified this as Racer Rock in Comment 173, and within 12 hours the BAS had already corrected the data!

Steig maintains that he did not receive my Feb. 28 e-mail to him, as he was in Antarctica, and that I should have received an automatic reply to that effect. While it is possible I did receive such an automatic reply from him, I can’t find it in my in box, spam box, or even trash box. I did receive such an automatic reply from him on March 17, in reply to another e-mail, requesting data. This indicated that he would in Antarctica until March 19, and could be contacted until then only by a special e-mail address posted on his website. There was no indication in the message that he would not catch up on e-mails received in his absence on his return.

I did receive an automatic reply from Michael Mann on Feb. 28, indicating that he was out of town and that important e-mails should be resent after his return, or he was not likely to read them. However, I had no reason to believe that Steig’s other 4 co-authors had not received my e-mail. Nevertheless, a new post on Real Climate does state that “Had Dr. McCulloch been the first person to make Steig et al. aware of the error in the paper, or had he written directly to Nature at any time prior to the submission of the Corrigendum, it would have been appropriate to acknowledge him and the authors would have been happy to do so. ” The key addition here is “et al”, indicating that none of the 6 authors learned of the error from my post, and that therefore all were as ignorant of the discussion here as was Steig himself.

By Hu McCulloch | Posted in General | Tagged , | Comments (137)

Steig and the International Man of Mystery

Aug 14, 2009 – 1:04 PM

Real Climate has defended Steig against a plagiarism complaint from Hu McCulloch, covered by Pielke Jr here and Jeff Id here. Hu’s original post is here and the most recent CA discussion here. Hu’s complaint is here.

Note: Hu’s email to the Steig coauthors is here. Steig was not the only recipient. All Steig authors were copied – Stieg, D Schneider, Rutherford, Mann, Comiso and Shindell.

Date: Sat, 28 Feb 2009 15:51:44 -0500
To: steig@ess.washington.edu, dschneid@ucar.edu, srutherford@fox.rwu.edu,mann@psu.edu, josefino.c.comiso@nasa.gov,Drew.T.Shindell@nasa.gov
From: Hu McCulloch
Subject: Comment on serial correlation in Steig et al 2009
Dear Dr. Steig and co-authors,
FYI, I have recently posted a comment on your 2009 paper in Nature
on Climate Audit, at http://www.climateaudit.org/?p=5341 .
While I was able to replicate or virtually replicate the 1957-2006 trends you report
on p 460 for the three regions and the continent as a whole, the 95% Confidence
Intervals you report appear to have taken no account of serial correlation
in the regression errors. When this is done, the CI’s are substantially wider
than you report.
Any reactions, by comments there or by e-mail, would be welcome!
— Hu McCulloch

J. Huston McCulloch [email and other particulars]
Economics Dept.
Ohio State Univ.
URL: http://econ.ohio-state.edu/jhm/jhm.html

RC argues that Steig was in Antarctica when the McCulloch email was sent. Steig claimed the following:

Response: McCulloch’s email, which provided no details but pointed me to his post, was sent while I was in the field in Antarctica, and would have received notification that I was in gone, and not receiving email for the next month.

Jeff Id sent a contemporary email to Steig and that is NOT the answer that he received. He received a message stating that Steig had limited email in Antarctica with a 30K file size max. (Hu’s message was 6K and well within the 30K that Steig was receiving.)

Hello,
I am in Antarctica until the middle of March. I will have email access via satellite, but text only is permitted, and 30 kB maximum file size. Please do not write except for essential matters. The email address is posted on my website at the University of Washington. Any issues pertaining to the lab should be directed to Andrew Schauer (contact information at —–). Don Grayson (Archaeology) is acting Chair of the Quaternary Research Center while I’m away.
Best wishes,
Eric Steig

Hureceived an out-of-office reply from Mann as follows:

Hu To: mcculloch.2@osu.edu
Subject: Re: Comment on serial correlation in Steig et al 2009
Date: Sat, 28 Feb 2009 15:53:36 -0500 (EST)
From: mann@meteo.psu.edu (Michael Mann)
I will be away from my email through March 16, 2009.
Any email sent before then may remain unread and be discarded. If your message
is important, you will need to resend after that date.

Notwithstanding this caveat, this does not preclude Mann processing his inbox shortly after March 16. An RC reader grasps at this particular straw arguing that this exonerates all the Steig coauthors.

Before considering their defence of Steig, let me review their previous defence of Gavin the Mystery Man, originally discussed here here. I had noticed curious properties in the Harry station in Antarctica, which were subsequently identified as resulting from the splice of two different stations. Based on information provided at Climate Audit, Gavin Schmidt spent his Sunday night determining that Harry had been spliced with Gill and notified the British Antarctic Survey (but not GISS) of the error. By the next morning, BAS had erased the incorrect series used in Steig et al, adding one further difficulty to replication of this study. Gavin said that a mystery scientist had “independently” discovered the problem and notified BAS.

Gavin stated:

It would have been nice had SM actually notified the holders of the data that there was a problem (he didn’t, preferring to play games instead). If he hadn’t left it for others to work out, he might even have got some credit 😉

and :

People will generally credit the person who tells them something. BAS were notified by people Sunday night who independently found the Gill/Harry mismatch. SM could have notified them but he didn’t. My ethical position is that it is far better to fix errors that are found than play around thinking about cute names for follow-on blog posts. That might just be me though. – gavin]

and:

I’m sure BAS are grateful that someone told them about the data mess up – it just didn’t happen to be SM.

After being outed as his own “Mystery Man”, Gavin stated:

i) discovering that Gill was mismatched with Harry was found independently by at least three people (SM, myself, and a poster on CA). ii) the source of the confusion was indeed found and not given to me by anyone else, iii) we are all dependent on many things, including that SM had alluded to data problem at Harry – I don’t see anywhere that I denied this. And BAS were notified by ‘people’ (plural) – not just by me. – gavin]

An RC reader asked:

So let me get this straight, Gavin. You found the very same problem that SM alluded to, on the very same day he did, with no direct or indirect input from CA? Is this correct?

Gavin:

[Response: Huh? Let’s try again. He alluded to an unspecified problem, and I looked into it. I found the source of the problem with no further input from anyone. This isn’t that complicated. – gavin]

I refer to the Gavin and the Mystery Man episode to demonstrate what Gavin’s idea of discovering a problem “independently” meant. It meant reading comment threads at CA, where issues were being discussed openly, spending his Sunday evening trying to figure out where we were going and notifying the British Antarctic Survey the next morning before we woke up.

This also needs to be borne in mind when Real Climate authors claim to be unaware of posts at Climate Audit. Obviously this isn’t the first such incident. Various changes were made at the Mann et al 2008 Supplementary Information responding to CA comments, without citing CA. On Feb 26, Hu McCulloch posted on Steig’s failure to allow for serial correlation in their confidence interval calculations (even though they had said that they had done so.) The recent Steig corrigendum made precisely the same correction – without acknowledging McCulloch.

RC acknowledged Hu’s priority, but stated that Steig became aware of the error “independently”.

This brings us to the recent claim by Hu McCulloch that a post on ClimateAudit.org, detailing an error in Steig et al’s paper in Nature on Antarctic temperature change, was not given due credit by Steig et al. when they published a Corrigendum earlier this month. In this case, McCulloch’s comment on the paper were perfectly valid, but he chose to avoid the context of normal scientific exchange — instead posting his comments on ClimateAudit.org — and then playing a game of ‘gotcha’ by claiming plagiarism when he wasn’t cited.

McCulloch accuses Steig et al. of appropriating his ‘finding’ that Steig et al. did not account for autocorrelation when calculating the significance of trends. While the published version of the paper didn’t include such a correction, it is obvious that the authors were aware of the need to do so, since in the text of the paper it is stated that this correction was made. The corrected calculations were done using well-known methods, the details of which are available in myriad statistics textbooks and journal articles. There can therefore be no claim on Dr. McCulloch’s part of any originality either for the idea of making such a correction, nor for the methods for doing so, all of which were discussed in the original paper. Had Dr. McCulloch been the first person to make Steig et al. aware of the error in the paper, or had he written directly to Nature at any time prior to the submission of the Corrigendum, it would have been appropriate to acknowledge him and the authors would have been happy to do so. Lest there be any confusion about this, we note that, as discussed in the Corrigendum, the error has no impact on the main conclusions in the paper.

RC’s arguments here raise a couple of issues.

They concede that the “first person” to make Steig et al aware of the error deserves acknowledgement. However, they did not say who that “first person” was nor is another “first person” acknowledged in the post in question. OK, I’ll bite. Who was the first person to make Steig et al aware of the error in the paper? Who is this mysterious person, unacknowledged in the Corrigendum?

Was it another “independent” discovery by the International Man of Mystery?

Update: In the first comment below, a reader observes that Steig said in an RC comment:

In any case, I had already recognized the error in our paper before I heard anything about McCulloch.

This could be interpreted as meaning that Steig suddenly realized that they had goofed in their Nature paper and that there was no “first person”. However, I don’t think that it precludes the possibility of him being alerted to the problem by a non-McCulloch first person, perhaps even by the International Man Of Mystery. Steig could easily clarify this. If he suddenly realized the problem all by himself, I, for one, would be interested in knowing what led him to this sudden realization after the months of article preparation, discussion and peer review.

By Stephen McIntyre | Posted in Steig at al 2009 | Tagged , | Comments (55)

The CRU Gong Show: Refusing Ross McKitrick

Aug 13, 2009 – 5:57 PM

Today brought in some CRU refusals- their rejections of Ross Mc, Roman M, myself. (They’re going to have to re-do their Roger Pielke rejection, since they replied to the wrong request in his case.) Each one deserves to be savored. So today I’ll post up their obstruction of Ross McKitrick.

FOI officer Palmer denied the request on the grounds that the request is “manifestly unreasonable” as the data is “available elsewhere”, that its disclosure would have an “adverse effect on international relations” and would have an adverse impact on the institutions supplying the data. Continue reading →

By Stephen McIntyre | Posted in CRU, FOIA | Tagged , | Comments (169)

More Check Kiting at Nature

Aug 12, 2009 – 9:21 PM

Nature has published another remarkable example of academic check kiting by Michael Mann et al, this time “Atlantic hurricanes and climate over the past 1,500 years”. (Prior examples of academic check kiting discussed at CA are Ammann and Wahl, the story of which is well told by Bishop Hill’s Caspar and the Jesus Paper and “Mann, Bradley and Hughes 2004”, cited in Jones and Mann 2004.)

Mann et al 2009 reconstructs Atlantic tropical cyclone counts resulting in a curve that looks pretty much like every other Mannian curve. Atlantic tropical cyclone counts as a linear combination of reconstructed Atlantic SST in the east tropical Atlantic “main development region” (MDR), reconstructed El Nino and reconstructed North Atlantic Oscillation, using a formula developed in (3,16) – which surprisingly enough turn out to be articles by Mann himself (Mann and Sabatelli, 2007; Sabatelli and Mann 2007) previously discussed at CA here. This is summarized in the article as follows:

An independent estimate of past tropical cyclone activity was obtained using a statistical model for Atlantic tropical cyclone counts. This previously developed and validated 3,16 statistical model conditions annual Atlantic tropical cyclone counts on three key large-scale climate state variables tied to historical variations in Atlantic tropical cyclone counts: (1) the SST over the main development region (MDR) for tropical Atlantic tropical cyclones, which reflects the favourability of the local thermodynamic environment; (2) the El Nino/Southern Oscillation (ENSO), which influences theamount of (unfavourable) vertical wind shear; and (3) the North Atlantic Oscillation (NAO), which affects the tracking of storms, determining how favourable an environment they encounter. The statistical model was driven by proxy-based reconstructions17,18 of these three state variables (Fig. 2), yielding a predicted history of Atlantic tropical cyclone counts for past centuries.

One doesn’t necessarily expect much clarification from Mannian methodology and this time Mann surpasses himself. Remarkably the Methods Summary is almost word-for-word the same as the article. It’s actually a little less. Mann explains once again that they reconstructed hurricane counts by using reconstructions of the SST in the Atlantic Main Development Region, El Nino and the North Atlantic Oscillation, with the only new information being that they only used the North Atlantic Oscillation reconstruction over the past 500 years from Luterbacher (18), but it didn’t matter. Mann again cites (3,16) Mann and Sabatelli; Sabatelli and Mann.

We used a statistical model of tropical cyclone counts as conditioned on3,16: the MDR SST, the ENSO (measured by the boreal winter Nino3 SST index), and the boreal winter NAO index The statistical model, which is trained on the modern historical record, has been shown in independent statistical validation experiments3,16 to resolve roughly 50% of the interannual and longer-term variations in Atlantic tropical cyclone counts. The model, in this study, was driven by decadally smoothed proxy reconstructions of the three required climate indices to yield predictions of tropical cyclone activity over past centuries. The MDR SST and Nino3 reconstructions were derived from proxy-based surface temperature patterns spanning the past 1,500 years17. Though an NAO reconstruction was available only for the past 500 years18, the NAO influence was found to be very minor (Supplementary Information).

The Full Methods in the online version adds little additional information. Again we are told that they used a statistical reconstruction using Atlantic SST and El Nino from the enigmatic ref 17. Confidence intervals appear to be done using the recipes of Mann et al 2008 with lots of “decadally smoothed” series.

Statistical prediction of tropical cyclone counts using proxy reconstructions. Here the model was applied to decadally resolved reconstructions of MDR SST and Nino3 described by ref. 17 and the decadally smoothed winter NAO index of ref. 18. For the instrumental interval (1851 to present), standard errors due to uncertainties in the model coefficients were calculated from the residual decadal variance diagnosed from the validation residuals (standard errors were averaged for the early and late intervals of the split calibration/validation procedure). For the pre-1851 statistical model estimates, which are driven by reconstructed climate indices, there is an additional component of uncertainty due to the uncertainties in the climate indices themselves. This contribution was estimated by Monte Carlo simulations in which the statistical model was driven with an ensemble of 2000 randomly perturbed versions of the statistical predictors consistent with their estimated uncertainties17, and an additional random term due to the uncertainties in the model coefficients.

The Supplementary Information sheds no light on the methodology or the proxies.
The Supplementary Information contained no data sets. The proxies used for the Mann et al submission are not even listed.

The edifice is built on the SST and Nino3 reconstructions, both of which are references to the enigmatic reference 17, which turns out to be an unpublished submission of Mann et al.

17. Mann, M. E. et al. Global signatures of the Little Ice Age and the medieval climate anomaly and plausible dynamical origins. Science (submitted).

At the time that Nature published this article, there was precisely NO information available on what proxies were used in the reconstruction of Atlantic SST or El Nino or how these reconstructions were done. Did any of the Nature reviewers ask to see the other Mann submission? I doubt it. I wonder if it uses Graybill bristlecone pines.

UPDATE: Roger Pielke Jr observes below that Mann has provided a “grey” Supplementary Information at his website here http://holocene.meteo.psu.edu/~mann/Nature09/ , which commendably includes source code and data from the check-kited paper on i.e. we still don’t know anything about the reconstruction proxies for Atlantic SST or El Nino. To my knowledge, there is no reference in the original article or Nature SI to the Supplementary SI at Mann’s website; there is no link on Mann’s website to the Supplementary SI and the directory hosting the Supplementary SI is not readable or searchable. Unless you know the precise name of the subdirectory, you can’t there. Right now, I don’t know how Roger found the Supplementary SI, but once located, the documentation looks at first glance to be very commendable.

References:
Mann, M.E. , Jonathan Woodruff, Jeffrey P. Donnelly & Zhihua Zhang. NAture 2007. Vol 460| 13 August 2009| doi:10.1038/nature08219 http://www.meteo.psu.edu/~mann/shared/articles/MannetalNature09.pdf
3. Mann, M. E., Sabbatelli, T. A. & Neu, U. Evidence for a modest undercount bias in early historical Atlantic tropical cyclone counts. Geophys. Res. Lett. 34, doi:10.1029/2007GL031781 (2007)
16. Sabbatelli, T. A. & Mann, M. E. The influence of climate state variables on Atlantic Tropical Cyclone Occurrence Rates. J. Geophys. Res. 112, doi:10.1029/2007JD008385 (2007).
17. Mann, M. E. et al. Global signatures of the Little Ice Age and the medieval climate anomaly and plausible dynamical origins. Science (submitted).
18. Luterbacher, J. et al. Extending North Atlantic Oscillation reconstructions back to 1500. Atmos. Sci. Lett. 2, 114–124 (2002)

By Stephen McIntyre | Posted in General | Comments (183)

The "Confidentiality Agreements"

Aug 12, 2009 – 1:35 PM

CRU refused my FOI request for CRU data stating that:

Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics

I asked to see the precise language of the underlying agreements because I very much doubted that agreements specifically prohibited “further transmission to non-academics”. If there were such a term in an agreement, it seemed far more likely that the term would be for “academic use” or something like that, and, given that my interest was scholarly rather than commercial, I doubted that the language of any applicable agreement would be applicable.

CRU has only managed to locate three documents pertaining to their agreements with NMSs: an application to Spain and letters from Norway and Bahrain, all from 1993-4. They also include a letter from CRU to the Met Office and, inexplicably, a copy of a current webpage from NERC governing Met Office data.

In their discussion of data availability, they state (in contorted logic) that:

In some of the examples given, it can be clearly seen that our requests for data from NMSs have always stated that we would not make the data available to third parties. We included such statements as standard from the 1980s, as that is what many NMSs requested.

Let’s examine the agreements. If the effect of these and other agreements is to prohibit them from supplying me the data, it is under language that has far more broad reaching consequences than merely preventing the delivery of data to me.

The British Territories Agreement
Here is the language from the CRU request to the UK Met Office regarding information about British Territories. It asks for data in connection with the construction of climatological normals; it makes no mention of the construction of a gridded temperature index nor the construction of a merged land-sea index. It says that the data would be used “unauthorized for any project” – which would obviously include its use in CRUTEM and HadCRU unless such authorization had been obtained. It doesn’t prohibit delivery of the data to “non-academics”; it prevents delivery of the data to “third parties” -which includes all the recipients of Advance 10K data, those people who downloaded cruwlda2 or newcrustnsall and any other academic.

If this language is representative of agreements with NMSs, then it prohibits the delivery of the data to CDIAC at the US Department of Energy (who published versions in the mid1980s and placed a version online in the early 1990s.) For that matter, it would prohibit the delivery of NMS data to the Met Office for use in HadCRU as the Met Office is a “third party” to CRU.

The Norway Correspondence
The Norway correspondence is also from 1993-1994 (pre WNO Resolution 40) and also is about 1961-1990 climatological normals, rather than temperature data. But viewing this as an example of the “lost” agreements, as CRU invites us to do, it says that CRU must not give the data “to a third party”. It doesn’t say “non-academic” – it says “third party.” If CRU seriously holds that this clause is in effect, then, as above, it prevents them from delivering data to the Advance-10K academics, to CDIAC and to the UK Met Office for use in HadCRU.

The Bahrain Correspondence
The language may well not constitute a binding agreement. But let’s stipulate that it does. Again it doesn’t refer to “non-academics”; it refers to “third parties”, including academics, CDIAC and the Met Office.



The Spanish Correspondence
As others have noted, the Spanish correspondence contains no prohibition on delivery to third parties – it merely asks for citation. I suppose that CRU might argue that there was an implied term of the agreement regarding further transmission of the data – but on the available evidence, such implied term would extend to all “third parties” and not merely to Climate Audit readers.

By Stephen McIntyre | Posted in General | Comments (26)

Nature Reports on CRU Stonewalling

Aug 12, 2009 – 12:16 PM

Nature reported today on the CRU data requests. I was interviewed at length last Thurs, followup Friday by Olive Heffernan of Nature. They even asked for a photograph. I haven’t seen the article yet. More after I see the story.

Update: There is an additional discussion at the Nature Blog. Behave nicely.

Update- my picture is in the article :). Photo credit is to my sister.

By Stephen McIntyre | Posted in Archiving, CRU, Nature | Comments (72)