Update: Next instalment here
On March 16, Science sent me 10 (out of 14) measurement data sets used by Esper; one gridcell temperature series used by Osborn-Briffa and caused Briffa to archive annual data versions at WDCP in addition to the smoothed versions. The new information has been extremely helpful to me.
However, the information remains incomplete. I can’t imagine why Esper would only send 10 of 14 measurement data sets; you’d think that he’d send them all when he picked up the file. Osborn-Briffa used different versions in several cases and these remain outstanding. Anyway, I’ve re-iterated my request. There are some puzzles in the data provided.
For example, Briffa calculated foxtail correlations for the period 1888-1990 even though there was gridcell information available from 1870. Using the full record, the correlation falls from 0.18 to 0.045. (I can’t replicate many other reported correlations, but that’s a different story.)
To keep you posted (and to give a flavor of how each step is frustrated by incomplete data, here’s my reply to Science, re-capping the status of outstanding data requests.
Dear Dr Hanson,
Thank you for your email of March 16, 2006, enclosing 10 measurement (rwl) files, one temperature file and providing notice of the amendment of the Briffa archive at WDCP. I have already found them very helpful. While you stated that “The tree ring data are archived also at the NOAA ITRDB (at least several that I checked but I haven’t gone through each one; e.g., the pol site not included here is there)”‘?, I can assure you that (1) not all the data is at the ITRDB site; (2) where there are ITRDB versions, it would be (was) impossible to match the combinations of ITRDB sites with the Esper data. (3) comparing the Esper versions to ITRDB versions shows some errors in the ITRDB versions.
I won’t go through all the issues, but will mention a couple. Osborn and Briffa cite two ITRDB measurement data sets for Mangazeja (russ067 and russ068). However there is no ring width data set at russ067w. The missing russ067 series can be located in the Birminsdorf format data at ITRDB. However, Esper uses 4 ITRDB data sets (not two) for Mangazeja. In addition to the two ITRDB data sets labelled Mangazeja, Esper used two nearby datasets (Moma). In the absence of a detailed SI, this could not be guessed at. To further complicate matters, WDCP has inadvertently reversed one of the Moma series with one of the Mangazeja series in the ITRDB version. In another case, Esper cites Luckman et al [1997] for the Athabaska site, but uses a data set not used in the Luckman article, while not using a data set used in the Luckman article. In another case, the ITRDB identification (germ21) for Tirol provided in Osborn and Briffa does not match either the Esper measurement data or the site chronologies. I’m in the process of reconciling these matters. I would very much encourage the provision of a more adequate Supplementary Information and would be happy to provide you with notes on this in case any one else inquires. So please be assured that the information is essential to replication efforts given the present Supplementary Information and the effort is not wasted.
As you recognize, some of the information which you had undertaken to obtain has not yet been provided. I have re-stated these requests below to review the bidding:
1. In February, Esper provided 13 of 14 site chronologies, omitting the Mongolia series for some reason. Could you please provide the 14th chronology.
2. In your March email, Esper, through your assistance, provided 10 of 14 site measurement (rwl) files, omitting Mongolia, Polar Urals, Boreal and Upperwright. Boreal and Upperwright are definitely not at ITRDB. The Polar Urals data set at ITRDB does not have 157 radii (as indicated in the chronology summary). I don’t understand why these were not provided concurrently with the other 10, but could you please provide the remaining 4 rwl files.
3. In 4 cases, the Osborn site chronology differs from the Esper site chronology, although in the other cases the versions are identical. In some cases, the date ranges do not match. I do not believe that it is possible to replicate the Osborn version from the Esper measurement data in these 4 cases and surmise that Osborn used a different measurement data set. I therefore request measurement data used by Osborn for the following sites: Polar Urals, Tornetrask, Taymir and Athabaska.
4. In 4 cases (Athabaska, Jaemtland, Quebec,Zhaschiviersk), Esper’s site chronology says that not all of the data in the data set is used. This is not mentioned in the original article. What is the basis for de-selection of individual cores?
5. Esper et al. [2002] do not provide a clear and operational definition distinguishing “linear”‘? and “nonlinear”‘? trees. As previously requested, could you please provide an operational definition of what they did, preferably with source code showing any differences in methodology. (I’ve enclosed a copy of a policy on source code which has been implemented at the American Economic Review by Ben Bernanke, its former editor who is now Chairman of the Federal Reserve Board. Several other economics journals have adopted similar policies. I urge Science to consider a comparable policy.)
6. I acknowledge receipt of a temperature data set from 1888-1990. The HadCRU2 data set contains temperature data for the gridcell 37.5N, 117.5W commencing in 1870. However, the gridcell information provided by Osborn commenced only in 1888 and the differences are material to the final result (0.045 versus 0.18 reported). What is the reason for commencing this comparison in 1888 rather than the available 1870? Why is there no notice of this in the SI? Since there is a material difference in this example, could you please provide the gridcell temperature sets in a comparable format for the other 13 Osborn and Briffa series/
7. The long-standing request for a complete archive of Thompson data, especially Dunde and Guliya ice cores, including both isotope and chemical data, remains outstanding.
I acknowledge that the following request has been completed and thank Science for its efforts:
1. Digital versions of the Osborn and Briffa data sets on an unsmoothed basis. The WDCP archive appears satisfactory.
Again, I express my appreciation for your efforts to extract data from the various authors. As mentioned above, I anticipate that I will be able to provide some useful Supplementary Information and will provide this to you for your records in case there are similar inquiries in the future.
Yours truly,
Stephen McIntyre
6 Comments
Isn’t the process of science a glorious, nay, a beautiful thing ? Just reading that made my knees go weak 🙂
I am sure that the folk at Science are not that stupid. They have seen corrigenda for mbh and moberg, and they know that the hockey-stick publications in their journal are the basis of public policy. I am sure that they don’t like it, but they are slowly enforcing compliance from their authors to meet the basic standard of providing enough information to enable replication. They make all that effort, and wham- it could easily start to look like they have opened a can of worms.
If the folk at science are too busy firefighting, they won’t think about this. If they do, the implications for the credibility of their publication will be worth considering.
cheers
per
WOW, again. You start thinking “charade,” when you read your post. I hope it is not, but am thinking it is so….
I hope that they interpret the moral as being that these articles have to be buttoned up correctly at the outset. In a way, it’s too bad that Hanson has to be in the middle of the process, but it’s lax administration by the journal (together with lax administration by NSF) that results in the problem. So I think that it’s useful for him to work through one of these cases in baby steps.
He should be pissed off at Esper for sending 10 of 14 rwl files and 13 of 14 crn files; and wasting his time having to keep picking up the file. Although he probably blames it on me rather than Esper. But it’s really nice to see them being constructive about this and hopefully the exercise will be salutary for all parties.
The truncation of the temperature series by Briffa (AR4 lead author) is unfortunately all too typical. I don’t see how he’s going to be able defend the truncation. Some of the other results appear even worse, but I’ve haven’t doble-checked them yet. On a preliminary basis, Osborn-Briffa report a correlation of the Tirol series to gridcell temperature of 0.45, but I can only reproduce 0.07. (Since I can pretty much replicate one gridcell, I’m sure that I’m apples and apples in terms of picking off gridcell temperatures. I’ll report on this in a weeek or so.
Ross may not always appreciate your pedantic eye for detail Steve but it is the correct way to deconstruct the argument. Methodology is the foundation for science and if those requests for information continue to be only partially answered then anybody must have to question the robustness of the process, let alone the conclusions. I admire your persistance and methodical approach. Stay patience and continue to ask the awkward questions.
Steve: assuming there is a linear temperature signal and assuming the temperature is increasing, shouldn’t the grass plots show upward bending curves?
Ooops. My comment is misplaced. Should have been in connection with the grass plots.